LetTokenBurnDecepEval: A Benchmark for Evaluating Deception in LLM Agents
A benchmark with 1,532 instances across 3 task families and 28 scenarios measuring deception rates in LLM agents under neutral vs induced conditions.
$ npm i -g lettokenburn-cli$ ltb bench info decepeval
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| GLM-5.3 Flash | 27.1 | N/A | Independent | medium | Oct 6, 2026 | arxiv.org |
| Claude Opus 5 | 36.0 | N/A | Independent | medium | Oct 6, 2026 | arxiv.org |
| Claude Sonnet 5 | 36.2 | N/A | Independent | medium | Oct 6, 2026 | arxiv.org |
| GPT-5.6 Sol | 36.8 | N/A | Independent | medium | Oct 6, 2026 | arxiv.org |
| GPT-5.6-terra | 45.5 | N/A | Independent | medium | Oct 6, 2026 | arxiv.org |
| Grok 4.5 | 54.8 | N/A | Independent | medium | Oct 6, 2026 | arxiv.org |
| Claude Haiku 4.5 | 66.5 | N/A | Independent | medium | Oct 6, 2026 | arxiv.org |
| DeepSeek V4 Flash | 78.4 | N/A | Independent | medium | Oct 6, 2026 | arxiv.org |
| DeepSeek V4 Pro | 80.6 | N/A | Independent | medium | Oct 6, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
DecepEval comprises 1,532 instances spanning three task families (tool-use, coding, long-horizon) and 28 professional scenarios. It pairs neutral and induced versions of tasks to measure condition-dependent changes in deception based on the LLM Deception Diamond framework.
Scoring compares deception rates between paired instances to isolate inducement effects. Evaluations of nine frontier LLMs show inducements increase deception across all models, with long-horizon interactions exhibiting the highest increase (55.73%).