LetTokenBurnWhen Charts Deceive Web Agents: A Paired Benchmark for Misleading Visualization-Grounded Tasks
A paired executable benchmark with 280 task instances evaluating how misleading visualizations affect Web Agent execution and reliability.
$ npm i -g lettokenburn-cli$ ltb bench info misvisagentbench
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Claude Opus 4.8 | 50.7 | N/A | Independent | medium | Oct 9, 2026 | arxiv.org |
| GPT-5.6 Sol | 49.3 | N/A | Independent | medium | Oct 9, 2026 | arxiv.org |
| Gemini 3 Pro | 45.7 | N/A | Independent | medium | Oct 9, 2026 | arxiv.org |
| Claude Opus 4.7 | 45.0 | N/A | Independent | medium | Oct 9, 2026 | arxiv.org |
| Claude Opus 4.6 | 41.4 | N/A | Independent | medium | Oct 9, 2026 | arxiv.org |
| Qwen3.5 Plus | 40.0 | N/A | Independent | medium | Oct 9, 2026 | arxiv.org |
| Kimi K2.6 | 37.9 | N/A | Independent | medium | Oct 9, 2026 | arxiv.org |
| Qwen3.6 Plus | 37.9 | N/A | Independent | medium | Oct 9, 2026 | arxiv.org |
| GPT-5.4 | 35.7 | N/A | Independent | medium | Oct 9, 2026 | arxiv.org |
| Claude Haiku 4.5 | 34.3 | N/A | Independent | medium | Oct 9, 2026 | arxiv.org |
| Claude Sonnet 4.6 | 34.3 | N/A | Independent | medium | Oct 9, 2026 | arxiv.org |
| GPT-5.5 | 34.3 | N/A | Independent | medium | Oct 9, 2026 | arxiv.org |
| Gemini 3.1 Flash | 27.9 | N/A | Independent | medium | Oct 9, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Comprises 280 matched clean–misleading pairs of executable web-agent tasks that differ only in chart evidence. It isolates misleading-induced failures from baseline difficulty using a diagnostic framework.
Scoring relies on task success rate and analysis of browser traces for extra steps or hesitation. Misleading charts reduced average success by over 34 percentage points across thirteen agents.