Model
GPT-4o
OpenAI · closed weights
Knowledge & Reasoning
Knowledge & Reasoning| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| Balrog | 32.3 | average_progress | N/A | Independent | medium | balrogai.com |
| CritPt | 0.0 | accuracy | N/A | Independent | low | epoch.ai |
| EnigmaEval | 0.8 | accuracy | N/A | Independent | medium | labs.scale.com |
| GDPval | 9.9 | win_rate | N/A | Independent | low | epoch.ai |
| GPQA | 53.6 | accuracy | N/A | Independent | medium | arxiv.org |
| GPQA Diamond | 49.2 | accuracy | N/A | Independent | high | epoch.ai |
| HLE | 2.7 | accuracy | N/A | Independent | low | epoch.ai |
| LiveBench | 55.3 | global_average | N/A | Independent | medium | livebench.ai |
| MMLU | 88.1 | exact_match | N/A | Independent | medium | arxiv.org |
| MOF-VERIFY | 62.8 | macro-f1 | N/A | Independent | medium | arxiv.org |
| News QA | 70.2 | exact_match | N/A | Independent | medium | arxiv.org |
| PeerQA | 30.9 | macro_f1 | N/A | Independent | medium | arxiv.org |
| PHRBench | 72.1 | auroc | N/A | Independent | medium | arxiv.org |
| ScienceQA | 88.5 | score | N/A | Independent | medium | arxiv.org |
| SimpleQA Verified | 26.0 | accuracy | N/A | Independent | high | epoch.ai |
Math
Math| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| FrontierMath | 1.0 | accuracy | N/A | Independent | medium | paperswithcode.com |
Coding
Coding| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| Abt-Buy | 92.2 | f1_(%) | N/A | Independent | medium | arxiv.org |
| Aider Polyglot | 23.1 | percent_correct | N/A | Independent | medium | aider.chat |
| Amazon-Google | 63.5 | f1_(%) | N/A | Independent | medium | arxiv.org |
| ArchitectureIQ | 40.0 | accuracy | architectureiq | Independent | medium | arxiv.org |
| GSO | 0.0 | score_opt@1 | OpenHands | Independent | medium | gso-bench.github.io |
| PLCWorld | 92.8 | success_rate | N/A | Independent | medium | arxiv.org |
| SWE-bench Verified | 31.0 | resolved_rate | N/A | Independent | high | epoch.ai |
| VHDL-REPOBENCH | 60.9 | bc_(pass@1) | N/A | Independent | medium | arxiv.org |
| Web-Bench | 17.2 | pass@1 | N/A | Independent | medium | arxiv.org |
| WebApp1k-Duo-React | 53.1 | pass@1 | N/A | Independent | medium | arxiv.org |
| WebApp1K-React | 88.5 | pass@1 | N/A | Independent | medium | arxiv.org |
Agentic / Tool Use
Agentic / Tool Use| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| 6-DoF SpatialBench | 36.2 | total | N/A | Independent | medium | arxiv.org |
| EHR-RobustGym | 80.9 | success_rate | N/A | Independent | medium | arxiv.org |
| GeoNatureAgent (GNA) | 41.6 | accuracy | N/A | Independent | medium | arxiv.org |
| KlinikeBench | 3.9 | strict_pass@1 | N/A | Independent | medium | arxiv.org |
| Talk2Agent | 58.5 | execution | N/A | Independent | medium | arxiv.org |
| TheAgentCompany | 8.6 | resolved_rate | N/A | Independent | medium | github.com |
Multimodal
Multimodal| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| DataVista | 57.1 | accuracy | N/A | Independent | medium | arxiv.org |
| FG-Sudoku | 0.3 | accuracy | N/A | Independent | medium | arxiv.org |
| GeoBench | 71.0 | acw_country_% | N/A | Independent | medium | geobench.org |
| Knossos | 41.2 | f1 | N/A | Independent | medium | arxiv.org |
| R-GroundBench | 69.3 | accuracy | N/A | Independent | medium | arxiv.org |
| SocialNav-SUB | 0.3 | pa | N/A | Independent | medium | arxiv.org |
| Video-MME | 77.2 | accuracy_(%) | N/A | Independent | medium | arxiv.org |
| Video-MME | 71.9 | overall_(no_subtitles) | N/A | Independent | medium | epoch.ai |
| VideoDB's OCR Benchmark Public Collection | 76.2 | average_accuracy | N/A | Independent | medium | arxiv.org |
| Vinoground | 59.2 | text_score | N/A | Independent | medium | paperswithcode.com |
| Vinoground | 54.0 | text_score | N/A | Independent | medium | paperswithcode.com |
| VLM2-Bench | 37.5 | gc-mat | N/A | Independent | medium | arxiv.org |
| Zero-shot Video Question Answering on LongVideoBench | 64.0 | accuracy_(%_) | N/A | Independent | medium | arxiv.org |
Long Context
Long ContextSafety & Alignment
Safety & Alignment| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| Cybench | 12.5 | unguided_%_solved | N/A | Independent | medium | cybench.github.io |
| Jailbreak Guardrails Benchmark | 40.0 | bypass_rate | N/A | Independent | medium | arxiv.org |
| MADBench | 73.1 | amplification_factor | N/A | Independent | medium | arxiv.org |
| OMIT | 0.1 | obr | N/A | Independent | medium | arxiv.org |
| OpenMTB-Audit | 0.7 | accuracy | prompt-checklist | Independent | medium | arxiv.org |
| Sense and Sensitivity | 91.4 | accuracy | N/A | Independent | medium | arxiv.org |
Abstract Reasoning
Abstract ReasoningMultilingual
MultilingualReproduced by the community
Loading community runs…