Model
Claude Opus 4.8
Anthropic · closed weights
Knowledge & Reasoning
Knowledge & Reasoning| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| CritPt | 20.9 | accuracy | N/A | Independent | low | epoch.ai |
| EnigmaEval | 23.5 | accuracy | N/A | Independent | medium | labs.scale.com |
| GPQA Diamond | 93.6 | accuracy | N/A | Self-reported | low | artificialanalysis.ai |
| GPQA Diamond | 91.0 | accuracy | N/A | Independent | high | epoch.ai |
| HLE | 57.9 | accuracy | N/A | Self-reported | low | lastexam.ai |
| HouseholdBench | 0.9 | relmae | N/A | Independent | medium | arxiv.org |
| MOF-VERIFY | 62.8 | macro-f1 | N/A | Independent | medium | arxiv.org |
| SimpleQA Verified | 53.0 | accuracy | N/A | Independent | high | epoch.ai |
| TasteVal | 0.8 | compute_multiplier | N/A | Independent | medium | arxiv.org |
Coding
Coding| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| A2Z GameSpec-Bench | 56.8 | gdd_fidelity | N/A | Independent | medium | arxiv.org |
| CRBugBench | 45.6 | success_rate | N/A | Independent | medium | arxiv.org |
| D2K-Bench | 2.3 | s_perf | N/A | Independent | medium | arxiv.org |
| DeepSWE | 59.0 | pass@1 | mini-swe-agent | Independent | medium | deepswe.datacurve.ai |
| E2E-SWE | 51.9 | pass@1 | N/A | Independent | medium | arxiv.org |
| GSO | 47.1 | score_opt@1 | OpenHands | Independent | medium | gso-bench.github.io |
| MathVis | 20.9 | pass@1 | N/A | Independent | medium | arxiv.org |
| ModeBench | 0.2 | pcmd | N/A | Independent | medium | arxiv.org |
| NP-Bench | 100.0 | clean-integration_success_rate | np-bench | Independent | medium | arxiv.org |
| SciCode | 53.5 | score | N/A | Independent | medium | artificialanalysis.ai |
| SimuVerity | 42.9 | score | claude code | Independent | medium | arxiv.org |
| SWE-bench Verified | 88.6 | resolved_rate | N/A | Self-reported | low | swebench.com |
| SWE-Duel | 2.5 | bradley_terry_log_strength | mini-swe-agent | Independent | medium | arxiv.org |
Agentic / Tool Use
Agentic / Tool Use| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| CASE longitudinal multimodal benchmark | 65.8 | overall_(%) | N/A | Independent | medium | arxiv.org |
| DeepResearch Bench | 50.2 | average_score | N/A | Independent | medium | evals.futuresearch.ai |
| DelegationBench | 65.5 | accuracy | N/A | Independent | medium | arxiv.org |
| EurekaBench | 0.1 | tau_b | N/A | Independent | medium | arxiv.org |
| IGMBench | 71.8 | success_rate | N/A | Independent | medium | arxiv.org |
| KlinikeBench | 16.8 | strict_pass@1 | N/A | Independent | medium | arxiv.org |
| Legal Research Bench | 42.9 | accuracy | N/A | Independent | medium | arxiv.org |
| MisVisAgentBench | 50.7 | pra | N/A | Independent | medium | arxiv.org |
| MMPostTrainBench | 40.6 | mean_task_score_(%) | claude code | Independent | medium | arxiv.org |
| OpenGameEval | 51.5 | pass_rate | N/A | Independent | medium | arxiv.org |
| StoreBench | 0.7 | composite | N/A | Independent | medium | arxiv.org |
Multimodal
Multimodal| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| ChartBmkAgent | 80.3 | accuracy | N/A | Independent | medium | arxiv.org |
| GDP.pdf | 24.0 | gdp.pdf_score | N/A | Independent | medium | surgehq.ai |
Instruction Following
Instruction Following| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| UXBench Pro | -0.0 | turn_bt | N/A | Independent | medium | arxiv.org |
| WISE | 80.3 | compliance | N/A | Independent | medium | arxiv.org |
Safety & Alignment
Safety & Alignment| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| DistressBench | 0.7 | score | N/A | Independent | medium | arxiv.org |
| DRYLAB-Bench | 0.1 | floor-included_brs_s2 | N/A | Independent | medium | arxiv.org |
| ObligationBench | 49.0 | recall | N/A | Independent | medium | arxiv.org |
| SWE-DeceptionBench | 3.3 | active_deception_rate | N/A | Independent | medium | arxiv.org |
Abstract Reasoning
Abstract ReasoningReproduced by the community
Loading community runs…