Model
Claude Fable 5.1
Anthropic · closed weights
Knowledge & Reasoning
Knowledge & Reasoning| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| ArcticAbstain | 41.5 | abstention_rate | N/A | Independent | medium | arxiv.org |
| CritPt | 31.1 | accuracy | N/A | Independent | low | epoch.ai |
| HLE | 46.5 | accuracy | N/A | Independent | low | epoch.ai |
| ScholarCatalyst | 0.5 | r@20 | tool-calling agent | Independent | medium | arxiv.org |
| SimpleQA Verified | 70.8 | accuracy | N/A | Independent | high | epoch.ai |
Coding
Coding| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| A2Z GameSpec-Bench | 77.0 | gdd_fidelity | N/A | Independent | medium | arxiv.org |
| GSO | 88.2 | score_opt@1 | OpenHands | Independent | medium | gso-bench.github.io |
| SciCode | 63.1 | score | N/A | Independent | medium | artificialanalysis.ai |
| Video2World | 48.5 | v2wscore | N/A | Independent | medium | arxiv.org |
Agentic / Tool Use
Agentic / Tool Use| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| DAYJOB | 9.6 | success_rate | N/A | Independent | medium | arxiv.org |
| DrivingBench | 45.0 | progress | claude code | Independent | medium | arxiv.org |
| Incident-Arena | 0.4 | pass@1 | claude code | Independent | medium | arxiv.org |
| RoboQuest | 11.4 | success_rate | N/A | Independent | medium | arxiv.org |
Multimodal
Multimodal| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| GDP.pdf | 29.6 | gdp.pdf_score | N/A | Independent | medium | surgehq.ai |
| HSS | 40.8 | pass@1 | N/A | Independent | medium | arxiv.org |
Instruction Following
Instruction Following| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| Design Creativity Bench | 48.6 | overall | N/A | Independent | medium | arxiv.org |
Safety & Alignment
Safety & Alignment| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| DistressBench | 0.7 | score | N/A | Independent | medium | arxiv.org |
| FIGS | 44.0 | automated_judge_applying_a_10-rule_taxonomy_separating_sycophancy_from_calibrated_validation | N/A | Independent | medium | arxiv.org |
Abstract Reasoning
Abstract ReasoningReproduced by the community
Loading community runs…