LetTokenBurnFactual Integrity and Grounded Support
A dual-axis evaluation framework using an adaptive 10-turn conversational simulator to measure sycophancy versus calibrated validation across 500 diverse scenarios.
$ npm i -g lettokenburn-cli$ ltb bench info figs
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Gemini 3.8 Flash | 65.5 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| Kimi K3 | 54.5 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| GLM-5.3 | 50.0 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| Claude Fable 5.1 | 44.0 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| DeepSeek V4.1 Flash | 37.3 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| GPT-5.6 Sol | 29.1 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| Grok-4.6 | 24.5 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| GPT-6 Astra | 14.5 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Content & Construction: FIGS is a benchmark suite consisting of 500 frozen multi-turn dialogue scenarios spanning six everyday domains and three severity levels. It utilizes an automated judge with ten rules (seven for Sycophancy, three for Calibrated Validation) to score interactions against a separate target model.
Scoring & Findings: The framework scores on two independent axes: 'Sycophancy' (holding firm to truth vs yielding) and 'Calibrated Validation' (empathetic understanding without overdoing it). Evaluations reveal that current systems either drift toward sycophancy under pressure or over-correct into cold, dismissive rigidity.