Model
o3
OpenAI · closed weights
Knowledge & Reasoning
Knowledge & Reasoning| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| CL-bench | 17.8 | overall | N/A | Independent | low | epoch.ai |
| Contradiction Benchmark for Review Evaluation | 99.9 | accuracy | N/A | Independent | medium | arxiv.org |
| CritPt | 1.4 | accuracy | N/A | Independent | low | epoch.ai |
| EnigmaEval | 13.1 | accuracy | N/A | Independent | medium | labs.scale.com |
| GDPval | 30.8 | win_rate | N/A | Independent | low | epoch.ai |
| GPQA Diamond | 81.8 | accuracy | N/A | Independent | high | epoch.ai |
| HLE | 20.3 | accuracy | N/A | Independent | low | epoch.ai |
| SimpleQA Verified | 49.4 | accuracy | N/A | Independent | high | epoch.ai |
Math
Math| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| FrontierMath | 25.2 | accuracy | N/A | Independent | medium | paperswithcode.com |
Coding
Coding| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| Aider Polyglot | 81.3 | percent_correct | N/A | Independent | medium | aider.chat |
| GSO | 8.8 | score_opt@1 | OpenHands | Independent | medium | gso-bench.github.io |
| SWE-bench Verified | 62.3 | resolved_rate | N/A | Independent | high | epoch.ai |
Agentic / Tool Use
Agentic / Tool Use| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| DeepResearch Bench | 45.2 | average_score | N/A | Independent | medium | drb.futuresearch.ai |
| OSWorld | 23.0 | score | o3 (100 steps) | Independent | medium | os-world.github.io |
| OSWorld | 17.2 | score | o3 (50 steps) | Independent | medium | os-world.github.io |
| OSWorld | 9.1 | score | o3 (15 steps) | Independent | medium | os-world.github.io |
Multimodal
Multimodal| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| FG-Sudoku | 0.8 | accuracy | N/A | Independent | medium | arxiv.org |
| GeoBench | 74.0 | acw_country_% | N/A | Independent | medium | geobench.org |
| R-GroundBench | 67.5 | accuracy | N/A | Independent | medium | arxiv.org |
Abstract Reasoning
Abstract ReasoningReproduced by the community
Loading community runs…