LetTokenBurnA benchmark of 413 open-ended U.S. legal research questions with gold answers and binary rubrics to measure end-to-end reliability of agentic models.
$ npm i -g lettokenburn-cli$ ltb bench info legal-research-bench
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Claude Opus 4.8 | 42.9 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| GPT-5.5 | 40.0 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| Claude Sonnet 4.6 | 38.5 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| GLM-5.2 | 31.2 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| Gemini-3.5-flash | 30.8 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| MiniMax-M3 | 29.8 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| GLM-5.1 | 27.6 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| Qwen3.7 Max | 25.2 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| DeepSeek V4 Pro | 23.2 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| Gemini 3.1 Pro | 20.8 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| Kimi K2.6 | 15.5 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| Grok-4.3 | 15.3 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| GPT-5.4-mini | 12.6 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
LRB comprises 413 expert-authored U.S. legal research questions spanning eight areas of law, each paired with verified authorities and binary grading rubrics. It evaluates tool-using agents on long-horizon tasks requiring web and case-law search.
Scoring uses all-pass grading: a response is correct only if it satisfies every rubric criterion and cited authorities verify. The best model, Claude Opus 4.8, achieved 42.9% accuracy, indicating significant room for improvement in agent reliability.