LetTokenBurnA benchmark of multi-solution tasks across five executable domains measuring solution mode diversity and correctness under RLVR.
$ npm i -g lettokenburn-cli$ ltb bench info modebench
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| DeepSeek V4 Pro | 0.3 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| Kimi K3 | 0.3 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| GPT-5.4 | 0.3 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| Grok-4.3 | 0.2 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| GPT-5.6 Sol | 0.2 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| Claude Opus 4.8 | 0.2 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| Claude Opus 5 | 0.1 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
ModeBench comprises questions with multiple valid solutions across five domains: graph coloring, normalized computation trees, executable Python code, algebraic problems, and constraint optimization. Each task includes an executable verifier that determines both correctness and unique solution-mode identity.
Scoring uses pairwise correct-mode diversity (PCMD) to measure diversity conditional on correctness, alongside standard accuracy metrics. Experiments show RLVR post-training reduces solution diversity even as accuracy improves, while the proposed Re:Max method mitigates this collapse.