Aider Polyglot
Harbor nativeAider Polyglot: 6种语言diff编辑
Verified71%Coding
- Samples
- 225
- Released
- 2024
- Scores
- 61
Contradiction Benchmark for Review Evaluation
A benchmark evaluating LLM-assisted peer review systems' ability to detect logical contradictions in AI conference papers via synthetic error insertion.
How to run
ltb · contradiction-benchmark-for-review-evaluation
$ npm i -g lettokenburn-cli$ ltb bench info contradiction-benchmark-for-review-evaluation
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
6 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| GPT-4.1 | 100.0 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| Gemini 3 Pro | 99.9 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| o3 | 99.9 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| o4-mini | 99.5 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| GPT-5.2-chat | 96.3 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| Gemini 2.5 Pro | 93.1 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsLoading community runs…
About
The benchmark consists of AI conference papers with synthetically inserted logical contradictions, creating unambiguous targets for error detection. It uses a knowledge-graph-based approach to quantify contradiction severity.
Scoring focuses on the system's ability to identify these specific errors rather than just imitating human reviews. The framework demonstrates high error detection performance and alignment with human scores, though vulnerabilities to adversarial manipulation persist.