ARC-AGI-2
Harbor nativeARC-AGI-2: 更难的抽象推理
Verified100%Abstract Reasoning
- Samples
- 167
- Released
- 2025
- Scores
- 120
Trapped in Your Story
Trapped in Your Story: Evaluating Narrative-Induced Sycophancy in Large Language Models
A benchmark of 5,500 instances measuring narrative-induced sycophancy in LLMs using 500 debatable questions across 10 domains with varied narrative justifications.
How to run
ltb · trapped-in-your-story
$ npm i -g lettokenburn-cli$ ltb bench info trapped-in-your-story
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
7 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Claude Fable 5 | 18.8 | N/A | Independent | medium | Oct 16, 2026 | arxiv.org |
| Gemini 3.8 Flash | 21.8 | N/A | Independent | medium | Oct 16, 2026 | arxiv.org |
| GPT-5.5 | 55.6 | N/A | Independent | medium | Oct 16, 2026 | arxiv.org |
| GPT-5.1 | 69.1 | N/A | Independent | medium | Oct 16, 2026 | arxiv.org |
| Llama-3.1-8B-Instruct | 83.0 | N/A | Independent | medium | Oct 16, 2026 | arxiv.org |
| Qwen3 8B | 99.6 | N/A | Independent | medium | Oct 16, 2026 | arxiv.org |
| DeepSeek V4 Pro | 99.8 | N/A | Independent | medium | Oct 16, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsLoading community runs…
About
The benchmark comprises 500 English debatable questions across ten domains, instantiated into 5,500 prompts. Each question features a neutral baseline and five mirrored narrative levels for two opposing positions, controlling for stance while varying justification structure.
Scoring measures direction-balanced agreement and the Narrative Sycophancy Gap. Results indicate that causal framing increases model agreement with user stances more than personal experience, highlighting how narrative structure influences sycophancy beyond simple stance alignment.