ARC-AGI-2
Harbor nativeARC-AGI-2: 更难的抽象推理
Verified100%Abstract Reasoning
- Samples
- 167
- Released
- 2025
- Scores
- 118
LPHB
The Labeling Problem in Hallucination Detection Benchmarks
A 900-sample human-annotated dataset for auditing automated hallucination labelers in open-domain QA, focusing on factual correctness.
How to run
ltb · lphb
$ npm i -g lettokenburn-cli$ ltb bench info lphb
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
7 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| GPT-5.4 | 0.8 | N/A | Independent | medium | Oct 6, 2026 | arxiv.org |
| GPT-5-mini | 0.8 | N/A | Independent | medium | Oct 6, 2026 | arxiv.org |
| Claude Opus 4.7 | 0.8 | N/A | Independent | medium | Oct 6, 2026 | arxiv.org |
| GPT-5 nano (2025-08-07) | 0.7 | N/A | Independent | medium | Oct 6, 2026 | arxiv.org |
| Gemma 2 9B IT | 0.6 | N/A | Independent | medium | Oct 6, 2026 | arxiv.org |
| Qwen2.5-7B-Instruct | 0.6 | N/A | Independent | medium | Oct 6, 2026 | arxiv.org |
| LLama-3 8B | 0.5 | N/A | Independent | medium | Oct 6, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsLoading community runs…
About
This benchmark consists of 900 question-answer pairs spanning three common QA datasets and three generator models, labeled by humans for factual correctness. It evaluates the alignment of automated labeling strategies (lexical metrics, NLI, LLM judges) with human judgments.
Scoring relies on agreement with human annotations as ground truth. Key findings reveal substantial disagreement between automated labels and humans, with faithfulness-oriented prompts causing false-positive dominance compared to factual-correctness prompts.