Model
Claude 3 Opus
Anthropic · closed weights
Knowledge & Reasoning
Knowledge & Reasoning| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| EnigmaEval | 0.8 | accuracy | N/A | Independent | medium | labs.scale.com |
| GPQA Diamond | 47.2 | accuracy | N/A | Independent | high | epoch.ai |
| LiveBench | 49.2 | global_average | N/A | Independent | medium | livebench.ai |
| MMLU | 84.6 | exact_match | N/A | Independent | medium | crfm.stanford.edu |
| PubMedQA | 75.8 | accuracy | N/A | Independent | medium | anthropic.com |
| PubMedQA | 74.9 | accuracy | N/A | Independent | medium | anthropic.com |
| SimpleQA Verified | 12.6 | accuracy | N/A | Independent | high | epoch.ai |
| WinoGrande | 88.5 | accuracy | N/A | Self-reported | medium | assets.anthropic.com |
| WinoGrande | 88.5 | accuracy | N/A | Independent | medium | anthropic.com |
Coding
Coding| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| MBPP | 86.4 | accuracy | N/A | Independent | medium | anthropic.com |
Long Context
Long Context| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| MMNeedle | 12.3 | 1_image,_4*4_stitching,_exact_accuracy | N/A | Independent | medium | anthropic.com |
Safety & Alignment
Safety & Alignment| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| Cybench | 10.0 | unguided_%_solved | N/A | Independent | medium | cybench.github.io |
| SynthPAI | 71.1 | average_accuracy_in_% | N/A | Independent | medium | arxiv.org |
Reproduced by the community
Loading community runs…