SciSlopBench
Science or Slop?: Benchmarking and Mitigating Scientific Slop in AI-Generated Papers
SciSlopBench paper link SciSlopBench is a paired benchmark for scientific slop: failures of global scientific reasoning in AI-generated research papers that token-level AI detectors do not see.
How to run
$ npm i -g lettokenburn-cli$ ltb bench info scislopbench
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
6 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| SciSlop | 85.4 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| CycleReviewer | 68.9 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| Binoculars | 68.3 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| DetectGPT | 62.3 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| AI Scientist | 61.3 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| NTS | 60.7 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsAbout
SciSlopBench
paper link
SciSlopBench is a paired benchmark for scientific slop: failures of global scientific reasoning in AI-generated research papers that token-level AI detectors do not see. It holds 390 pairs. Each pair is one AI-generated paper and one human-written, peer-accepted paper (its anchor) of the same contribution type on the same or a neighboring research problem. The AI papers come from two sources, and the same pairing rule is applied to both:…