LetTokenBurn
Sign in
Auto71%Harbor: pendingSafety & AlignmentKnowledge & Reasoning

RAGStress

RAGStress: A Controlled Benchmark for Evaluating Retrieval-Augmented Generation Under Knowledge-Base Degradation

Evaluates RAG robustness under 4 KB corruption types and 3 severity levels using 182,546 documents across 57 MMLU subjects.

PaperRepo

How to run

ltb · ragstress
$ npm i -g lettokenburn-cli
$ ltb bench info ragstress

External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.

Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs

Scores

5 sourced entries
HarnessSourceConfidenceLink
Qwen3-32B100.0N/AIndependentmediumOct 3, 2026arxiv.org
DeepSeek R199.9N/AIndependentmediumOct 3, 2026arxiv.org
Claude Opus 4.699.7N/AIndependentmediumOct 3, 2026arxiv.org
Claude Sonnet 4 (2025-05-14)99.4N/AIndependentmediumOct 3, 2026arxiv.org
Llama 3.1 8B Inst.98.6N/AIndependentmediumOct 3, 2026arxiv.org

Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.

Reproduced by the community

All runs
Loading community runs…

About

RAGStress stress-tests RAG systems by applying factual corruption, numeric typos, relevance poisoning, and contradiction injection at subtle, moderate, and obvious levels. It uses a metadata-filtered design on a single vector index with 182,546 documents derived from 57 MMLU subjects.

The benchmark evaluates model arbitration between parametric memory and corrupted context across 52,500 evaluations. Key findings indicate semantic-fidelity corruptions are more harmful than signal-utility perturbations, and clean retrieval accuracy does not predict robustness.