LetTokenBurn
Sign in

BenchHub · an open registry of 6,820 AI benchmarks

Reproduce any AI benchmark with the subscription you already pay for.

Find a benchmark, run it on your machine with the ltb CLI and Harbor, submit the score, and see it next to what the vendor self-reported.

Browse benchmarksRun one locally
6,820
Benchmarks indexed
718
Runnable with Harbor
1,634
Models tracked
6,039
Scores, each with a source

Snapshot updated Oct 13, 2026 · 386 benchmarks verified, 161 of them runnable today · how records are collected

Leaderboard preview

Who scores what, and who said so

12 models across 10 verified, current-generation benchmarks, grouped by capability. Each cell is the best score we hold with a source link; S marks numbers the vendor reported about its own model.

Open the full leaderboard
Best sourced score per model and benchmark, grouped by capability axis. Higher is darker green to brighter green.
ModelKnowledge & ReasoningCodingAgentic / Tool UseAbstract Reasoning
GPQA DiamondHLESimpleQA VerifiedSWE-bench VerifiedGSOSciCodeTerminal-BenchDeepResearch BenchOSWorldARC-AGI-2
GPT-5.4933645773157823575S74
Claude Sonnet 4.5 (2025-09-29)7414247115454748634
Claude Opus 4.691S53S4781S41805573S69S
Gemini 3.1 Pro94467478S2359804877
Claude Sonnet 4.690S49S3680S47535573S60
GPT-586255074743505010
GPT-5.1882448681443484318
Claude Opus 4.79036528444558076
Claude Opus 4.894S58S5389S47545072
GPT-5.2-chat9128377428654153
Claude Opus 4.5 (2025-11-01)81257727635566S8
Kimi K2.588S243477S49436312
0100S self-reported by the vendorunmarked: independent· no sourced scoreSwipe the grid to see every column.

Models and benchmarks here are picked for coverage, not ranked. 105 of 120 cells (88%) have a sourced score: 17 self-reported, 88 independent and 0 reproduced so far. Empty cells are never estimated; each one is a run waiting to happen.

How it works

From a benchmark page to a reproduced score

Four commands. Everything runs on your machine with the agent and model plan you already have; the platform only stores the score summary.

CLI and Harbor reference
  1. 1

    Discover

    Search the registry for the benchmark that matches what you care about. Each page lists the paper, data, license, metric and every known score with its source.

    $ npm i -g lettokenburn-cli
    $ ltb bench search mmlu
  2. 2

    Run it locally

    Pull it as Harbor tasks and run them in Docker with your own agent and subscription. Start with the oracle agent to check the tasks themselves.

    $ ltb bench pull mmlu-pro --n 50
    $ ltb bench run ./mmlu-pro -a oracle
    $ ltb bench run ./mmlu-pro -a claude-code -m anthropic/claude-opus-4-8
  3. 3

    Submit the score

    Push the run summary: score, task count, per-task rewards and a hash of the results file. Transcripts and model outputs stay on your machine.

    $ ltb bench push ./jobs
  4. 4

    Compare

    Your run joins the Reproduced column for that model and benchmark, next to the self-reported and independent numbers, each linked to its source.

    $ ltb bench info mmlu-pro

Start here

Verified benchmarks you can run today

Reviewed records with a working Harbor adapter or native Harbor tasks, ordered by how many sourced scores already exist to compare against.

All 6,820 benchmarks

Browse by capability

A note on data quality

Every score links to its source. Benchmark metadata is collected automatically from public channels and checked by an LLM reviewer, so some fields can be missing or wrong. Each record carries a quality tier (Verified, Auto or Incomplete) and lists what is missing. Only 386 of 6,820 records are verified today, and community reproductions are just starting. Spot an error? Email [email protected].

How data is collected Why BenchHub exists