LetTokenBurnOpenProblemBench: Benchmarking AI on Open Problems in the Foundational Theoretical Sciences
A benchmark of 82 unresolved mathematics and theoretical physics problems assessing AI research capabilities via evaluator models judging correctness and progress without reference solutions.
$ npm i -g lettokenburn-cli$ ltb bench info openproblembench
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| GPT-6 Astra | 18.3 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| GPT-5.6 Sol | 8.5 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| Kimi K3 | 3.7 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| Qwen3.8 Max | 2.4 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| DeepSeek V4.1 Flash | 1.2 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| GLM-5.3 | 1.2 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| GLM-5.3 Flash | 1.2 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
OpenProblemBench comprises 82 literature-grounded open problems from mathematics and theoretical physics, providing context, assumptions, and prior progress for each. It evaluates AI systems' ability to contribute to foundational science by proposing solutions to unresolved questions.
Four independent evaluator models assess submissions for correctness, completeness, and degree of progress. GPT-6-Astra achieved the highest mean judged solve rate of 14.0%, demonstrating superior capability in handling open scientific problems compared to other evaluated models.