Model
Claude 3.7 Sonnet Thinking (2025-02-19)
Anthropic · closed weights
Knowledge & Reasoning
Knowledge & Reasoning| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| EnigmaEval | 4.2 | accuracy | N/A | Independent | medium | labs.scale.com |
| GPQA Diamond | 66.0 | accuracy | N/A | Independent | high | epoch.ai |
| HLE | 8.0 | accuracy | N/A | Independent | low | epoch.ai |
| LiveBench | 76.1 | global_average | N/A | Independent | medium | livebench.ai |
Coding
Coding| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| Aider Polyglot | 60.4 | percent_correct | N/A | Independent | medium | aider.chat |
| GSO | 3.8 | score_opt@1 | OpenHands | Independent | medium | gso-bench.github.io |
| SWE-bench Verified | 61.0 | resolved_rate | N/A | Independent | high | epoch.ai |
| Web-Bench | 25.1 | pass@1 | N/A | Independent | medium | arxiv.org |
Agentic / Tool Use
Agentic / Tool Use| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| OSWorld | 35.8 | score | claude-3-7-sonnet-20250219 (50 steps) | Independent | medium | os-world.github.io |
| OSWorld | 35.6 | score | claude-3-7-sonnet-20250219 (100 steps) | Independent | medium | os-world.github.io |
| OSWorld | 27.1 | score | claude-3-7-sonnet-20250219 (15 steps) | Independent | medium | os-world.github.io |
| TheAgentCompany | 30.9 | resolved_rate | N/A | Independent | medium | the-agent-company.com |
Multimodal
Multimodal| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| GeoBench | 68.0 | acw_country_% | N/A | Independent | medium | geobench.org |
| SpatialViz-Bench | 33.9 | overall_score | N/A | Independent | medium | arxiv.org |
Safety & Alignment
Safety & Alignment| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| Cybench | 20.0 | unguided_%_solved | N/A | Independent | medium | cybench.github.io |
Abstract Reasoning
Abstract ReasoningReproduced by the community
Loading community runs…