Model
Claude 3 Sonnet
Anthropic · closed weights
Knowledge & Reasoning
Knowledge & Reasoning| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| GPQA Diamond | 40.6 | accuracy | N/A | Independent | high | epoch.ai |
| MMLU | 79.0 | average_(%) | N/A | Independent | medium | anthropic.com |
| MMLU | 75.9 | exact_match | N/A | Independent | medium | crfm.stanford.edu |
| WinoGrande | 75.1 | accuracy | N/A | Self-reported | medium | assets.anthropic.com |
| WinoGrande | 75.1 | accuracy | N/A | Independent | medium | anthropic.com |
Coding
Coding| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| MBPP | 79.4 | accuracy | N/A | Independent | medium | anthropic.com |
Multimodal
Multimodal| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| VideoDB's OCR Benchmark Public Collection | 67.7 | average_accuracy | N/A | Independent | medium | arxiv.org |
Safety & Alignment
Safety & AlignmentReproduced by the community
Loading community runs…