Model
Claude 3.5 Haiku (2024-10-22)
Anthropic · closed weights
Knowledge & Reasoning
Knowledge & Reasoning| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| Balrog | 19.3 | average_progress | N/A | Independent | medium | balrogai.com |
| CritPt | 0.0 | accuracy | N/A | Independent | low | epoch.ai |
| GPQA Diamond | 38.1 | accuracy | N/A | Independent | high | epoch.ai |
| LiveBench | 43.5 | global_average | N/A | Independent | medium | livebench.ai |
| MMLU | 74.3 | exact_match | N/A | Independent | medium | crfm.stanford.edu |
Coding
Coding| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| Aider Polyglot | 28.0 | percent_correct | N/A | Independent | medium | aider.chat |
| SciCode | 27.4 | score | N/A | Independent | medium | artificialanalysis.ai |
| Web-Bench | 15.4 | pass@1 | N/A | Independent | medium | arxiv.org |
Multimodal
Multimodal| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| GeoBench | 34.0 | acw_country_% | N/A | Independent | medium | geobench.org |
Reproduced by the community
Loading community runs…