LetTokenBurnA multimodal benchmark of 23,196 inputs evaluating spatial reasoning from reportable facts to usable states across four levels (L1-L4) using a Supported/Contradictory/Unknown judgment interface.
$ npm i -g lettokenburn-cli$ ltb bench info spaceconflict
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Qwen3.6 27B | 65.2 | N/A | Independent | medium | Oct 2, 2026 | arxiv.org |
| qwen3.5-27B | 62.5 | N/A | Independent | medium | Oct 2, 2026 | arxiv.org |
| InternVL3.5-14B | 60.5 | N/A | Independent | medium | Oct 2, 2026 | arxiv.org |
| qwen3.5-9B | 59.3 | N/A | Independent | medium | Oct 2, 2026 | arxiv.org |
| InternVL3.5-8B-Instruct | 57.8 | N/A | Independent | medium | Oct 2, 2026 | arxiv.org |
| google/gemma-4-31B-it | 57.0 | N/A | Independent | medium | Oct 2, 2026 | arxiv.org |
| Qwen3.8 27B | 56.6 | N/A | Independent | medium | Oct 2, 2026 | arxiv.org |
| Qwen3.5 4B | 55.8 | N/A | Independent | medium | Oct 2, 2026 | arxiv.org |
| Qwen2.5-VL-32B | 54.2 | N/A | Independent | medium | Oct 2, 2026 | arxiv.org |
| Qwen3-VL-8B-Instruct | 51.0 | N/A | Independent | medium | Oct 2, 2026 | arxiv.org |
| Qwen3-VL-2B | 25.7 | N/A | Independent | medium | Oct 2, 2026 | arxiv.org |
| Qwen2.5-VL-7B | 24.4 | N/A | Independent | medium | Oct 2, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
SpaceConflict comprises 23,196 inputs organized into four levels: local fact binding, relational composition, cross-observation consistency, and state judgment under transformation. It uses controlled conflicts among visual evidence and spatial claims to test if models can use recovered facts in subsequent computation.
Scoring uses a unified Supported/Contradictory/Unknown classification interface, with PairAcc computed over matched pairs. The benchmark reveals an availability-utilization gap where models recover initial states but fail to use them for transformations, a gap narrowed by Operational State Supervision.