Compose an AI system
smarter than any single model
and prove it where everyone can see.

01How a result becomes real

Most leaderboards ask for trust. Scores are self-reported, the test set is frozen the day it's published, and there is no way to check the numbers. This board is built so you never have to trust it.

1. A submission arrives carrying its recipe, its benchmark, and its claimed score.
2. It is validated and deduplicated by content hash.
3. The run is repeated independently. Benchmark answer keys and judges sit behind the Engine, where neither the submitter nor the operator can reach them.
4. Only then does it rank, with the recipe, the cost, and a rerun link attached.

Every published entry ships with its recipe, cost, and methodology, so anyone can re-run the result and check us.

Evaluation is nondeterministic, so scores carry variance and close ranks are effectively tied. At launch the board hosts fusions only. The full board publishes as open JSONL, so anyone can audit the board as well as the entries on it.

02The recipe travels with the result

Every entry stores its method as one line of url4, an open text grammar for describing a composed system: which models, with what parameters, asked to do what. The line compiles to a graph any engine can execute. Rerunning it reproduces the result. Changing one thing and resubmitting is how the next entry starts.

Nobody starts from a blank page. The board is seeded and every recipe is public; a newcomer's first act is a remix.

03Your keys, your machine

Keys

Stored encrypted by the local Engine on your machine. The Engine is the only component that ever talks to a model provider.

Grading

Answer keys and judges never leave the Engine. Your code can't see the answers. Neither can ours. Nobody grades their own homework.

Network

No hosted service required, no broker in the middle of your calls. What leaves your machine: provider requests made with your own keys, and any result you choose to publish. If ScreamingFace disappeared tomorrow, every published fusion would still run — nothing requires us in the path.

Cost

Every run reports its tokens, dollars, and time while it happens, totaled per model and per fusion.

Source

Free and open source, at github.com/screamingface.

04The toolkit

Client

The Python library, live now: pip install screamingface. Compose a fusion, run a benchmark, and get back the score, the bill, and the url4 line that reruns it all.

Studio

A desktop app that does the same job without code. Drag models into slots, set weights and a judge, and inspect every answer. In development; the Client ships first.

Engine

The process inside both that does the trusted work. The identical software also runs hosted for sponsored cohorts, and inside an institution's own walls.

05For researchers

Grading is independent by construction, so a result holds up in review without extra apparatus, and a finding exports as one line a reviewer can rerun on their own keys. Fusion Monsters, a cohort program with subsidized compute, is taking applications on a rolling basis; results published through it carry the researcher's name.