GSM8K
Harbor adapterGSM8K: 小学应用题（多步推理）
- Samples
- 1,319
- Released
- 2021
- Scores
- 180
DIB-Bench
Deployment-Integrity Blindness Benchmark
Measures deployment-integrity blindness across 7,560 examples with six tamper types, evaluating if output-only judges detect runtime stack changes.
How to run
ltb · dib-bench
$ npm i -g lettokenburn-cli$ ltb bench info dib-bench
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
3 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Qwen2.5-7B-Instruct | 88.6 | N/A | Independent | medium | Oct 15, 2026 | arxiv.org |
| Llama-3.1-8B-Instruct | 99.9 | N/A | Independent | medium | Oct 15, 2026 | arxiv.org |
| Qwen2.5-1.5B-Instruct | 99.9 | N/A | Independent | medium | Oct 15, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsLoading community runs…
About
DIB-Bench evaluates LLM agents using 7,560 scored examples across three model pairs and six single-component tamper types (e.g., prompt, tool, dependency). It employs an evaluator ladder from regex scorers to LLM judges to measure acceptance of tampered deployments.
Scoring relies on mean tamper acceptance rates, revealing that stronger evaluators are more blind to integrity violations. Key findings show acceptance rising from 88.6% for rule-based to 94.1% for GPT-5-nano, highlighting the need for identity channels.