Model
GPT-6 Astra
OpenAI · closed weights
Knowledge & Reasoning
Knowledge & Reasoning| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| Balrog | 68.3 | average_progress | N/A | Independent | medium | balrogai.com |
| CritPt | 31.7 | accuracy | N/A | Independent | low | epoch.ai |
| FREA | 65.7 | balanced_accuracy | N/A | Independent | medium | arxiv.org |
| GPQA Diamond | 95.8 | accuracy | N/A | Independent | high | epoch.ai |
| HLE | 54.8 | accuracy | N/A | Independent | low | epoch.ai |
| HouseholdBench | 0.9 | relmae | N/A | Independent | medium | arxiv.org |
| SimpleQA Verified | 75.6 | accuracy | N/A | Independent | high | epoch.ai |
| TasteVal | 1.6 | compute_multiplier | N/A | Independent | medium | arxiv.org |
Math
Math| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| OpenProblemBench | 18.3 | mean_judged_solve_rate | N/A | Independent | medium | arxiv.org |
Coding
Coding| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| A2Z GameSpec-Bench | 71.6 | gdd_fidelity | N/A | Independent | medium | arxiv.org |
| ArchitectureIQ | 76.4 | accuracy | architectureiq | Independent | medium | arxiv.org |
| D2K-Bench | 3.3 | s_perf | N/A | Independent | medium | arxiv.org |
| DeepSWE | 74.1 | pass@1 | mini-swe-agent | Independent | medium | deepswe.datacurve.ai |
| GSO | 79.4 | score_opt@1 | OpenHands | Independent | medium | gso-bench.github.io |
| MathVis | 31.3 | pass@1 | N/A | Independent | medium | arxiv.org |
| SciCode | 56.5 | score | N/A | Independent | medium | artificialanalysis.ai |
| TestSuiteBench | 50.1 | score | N/A | Independent | medium | arxiv.org |
| Video2World | 43.5 | v2wscore | N/A | Independent | medium | arxiv.org |
| Zero2Repo | 90.9 | success_rate | codex | Independent | medium | arxiv.org |
Agentic / Tool Use
Agentic / Tool Use| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| AgenticBBO-Bench | 0.6 | hpo | N/A | Independent | medium | arxiv.org |
| AgentPersonaBench | 79.7 | accuracy | N/A | Independent | medium | arxiv.org |
| APTInvestBench | 79.2 | coverage_rate | N/A | Independent | medium | arxiv.org |
| Argo-Bench | 51.8 | score | N/A | Independent | medium | arxiv.org |
| BiGym 2.0 | 76.0 | success_rate | tool-augmented | Independent | medium | arxiv.org |
| BioStudyBench | 94.7 | pass_rate | N/A | Independent | medium | arxiv.org |
| BrickBench | 1297.0 | elo | N/A | Independent | medium | arxiv.org |
| cua-speedrun | 91.6 | success_rate | N/A | Independent | medium | arxiv.org |
| DataSense-Bench | 36.5 | accuracy | N/A | Independent | medium | arxiv.org |
| DAYJOB | 11.6 | success_rate | N/A | Independent | medium | arxiv.org |
| DrivingBench | 100.0 | success_rate | codex | Independent | medium | arxiv.org |
| Incident-Arena | 0.6 | pass@1 | codex | Independent | medium | arxiv.org |
| KlinikeBench | 24.3 | strict_pass@1 | N/A | Independent | medium | arxiv.org |
| LIVIN | 41.5 | success_rate | N/A | Independent | medium | arxiv.org |
| Mine Odyssey | 85.6 | success_rate | N/A | Independent | medium | arxiv.org |
| RoboQuest | 23.2 | success_rate | N/A | Independent | medium | arxiv.org |
| WorldAuditBench | 42.3 | success_rate | vlm auditors | Independent | medium | arxiv.org |
| WorldAuditBench | 16.4 | success_rate | vla–vlm auditors | Independent | medium | arxiv.org |
Multimodal
Multimodal| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| GDP.pdf | 34.2 | gdp.pdf_score | N/A | Independent | medium | surgehq.ai |
| GeoSpatialBench | 96.4 | accuracy | N/A | Independent | medium | arxiv.org |
| HSS | 53.6 | pass@1 | N/A | Independent | medium | arxiv.org |
Instruction Following
Instruction Following| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| Design Creativity Bench | 65.5 | overall | N/A | Independent | medium | arxiv.org |
Safety & Alignment
Safety & Alignment| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| APEX-Harm | 0.0 | success_rate | N/A | Independent | medium | arxiv.org |
| DRYLAB-Bench | 0.1 | floor-included_brs_s2 | N/A | Independent | medium | arxiv.org |
| FIGS | 14.5 | automated_judge_applying_a_10-rule_taxonomy_separating_sycophancy_from_calibrated_validation | N/A | Independent | medium | arxiv.org |
| MADBench | 50.6 | amplification_factor | N/A | Independent | medium | arxiv.org |
Abstract Reasoning
Abstract Reasoning| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| Actionable Indeterminacy Benchmark | 92.6 | accuracy | N/A | Independent | medium | arxiv.org |
| ARC-AGI-2 | 95.0 | score | N/A | Independent | low | epoch.ai |
| DrillBench | 53.2 | accuracy | generative (hosted) | Independent | medium | arxiv.org |
Reproduced by the community
Loading community runs…