LetTokenBurnSELF-POT: Benchmark for Test-Time Computation Potential
A benchmark evaluating test-time computation gains across mathematics, programming, and agentic workflows using a unified budget rule to compare direct inference against parallel sampling and self-revision.
$ npm i -g lettokenburn-cli$ ltb bench info self-pot
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| DeepSeek V4.1 Flash | 0.8 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| GLM-5 | 0.7 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| GPT-oss 120B | 0.5 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| Qwen3-Next-80B | 0.5 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| MiniMax M2.5 | 0.4 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Content & Construction: SELF-POT is an evaluation framework featuring a sealed four-domain test set (mathematics, competitive programming, AppWorld, Terminal-Bench) containing 350 tasks. It uses five low-cost reasoning models compared against a stronger anchor model (Claude Opus 5.5), employing frozen protocols that charge every API call including selection and critique.
Scoring & Findings: The benchmark measures test-time potential by tracking correctness transitions under revision and protocol completion alongside task success. Key findings indicate that failure handling significantly impacts gains, public evidence can improve submission rates while lowering costs in programming tasks, and agent workflows primarily measure interface reliability due to stage failures.