Model
DeepSeek V4 Pro
DeepSeek · open weights
Knowledge & Reasoning
Knowledge & Reasoning| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| CL-bench Life | 13.5 | overall | N/A | Independent | low | epoch.ai |
| CritPt | 12.9 | accuracy | N/A | Independent | low | epoch.ai |
| GPQA Diamond | 90.9 | accuracy | N/A | Independent | high | epoch.ai |
| HouseholdBench | 1.0 | relmae | N/A | Independent | medium | arxiv.org |
| MOF-VERIFY | 62.8 | macro-f1 | N/A | Independent | medium | arxiv.org |
| SimpleQA Verified | 47.0 | accuracy | N/A | Independent | high | epoch.ai |
Coding
Coding| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| A2Z GameSpec-Bench | 50.4 | gdd_fidelity | N/A | Independent | medium | arxiv.org |
| AAArena | 2288.2 | elo | codex | Independent | medium | arxiv.org |
| ArchitectureIQ | 68.4 | accuracy | architectureiq | Independent | medium | arxiv.org |
| ArrivalBench | 45.3 | failure_rate | N/A | Independent | medium | arxiv.org |
| MetaKernelBench | 25.7 | paired_lift | cute dsl | Independent | medium | arxiv.org |
| ModeBench | 0.3 | pcmd | N/A | Independent | medium | arxiv.org |
| SCCGBench | 3.8 | mock | N/A | Independent | medium | arxiv.org |
| SciCode | 50.0 | score | N/A | Independent | medium | artificialanalysis.ai |
| SimuVerity | 28.4 | score | claude code | Independent | medium | arxiv.org |
| SWE-bench Verified | 77.6 | resolved_rate | N/A | Independent | high | epoch.ai |
| SWE-Duel | 1.5 | bradley_terry_log_strength | mini-swe-agent | Independent | medium | arxiv.org |
Agentic / Tool Use
Agentic / Tool Use| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| AgenticBBO-Bench | 0.5 | hpo | N/A | Independent | medium | arxiv.org |
| AgentMonBench | 0.5 | silentswap | N/A | Independent | medium | arxiv.org |
| APTInvestBench | 50.8 | coverage_rate | N/A | Independent | medium | arxiv.org |
| DataSense-Bench | 36.5 | accuracy | N/A | Independent | medium | arxiv.org |
| DyadMem | 66.1 | accuracy | N/A | Independent | medium | arxiv.org |
| IGMBench | 31.8 | success_rate | N/A | Independent | medium | arxiv.org |
| Legal Research Bench | 23.2 | accuracy | N/A | Independent | medium | arxiv.org |
| StoreBench | 0.7 | composite | N/A | Independent | medium | arxiv.org |
| Synthetic Trading Benchmark | 8.2 | e0_sharpe | N/A | Independent | medium | arxiv.org |
| XiangqiBench | 5.0 | pass@3 | N/A | Independent | medium | arxiv.org |
Multimodal
Multimodal| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| R-GroundBench | 71.0 | accuracy | N/A | Independent | medium | arxiv.org |
Instruction Following
Instruction Following| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| UXBench Pro | 0.0 | turn_bt | N/A | Independent | medium | arxiv.org |
| WISE | 55.0 | compliance | N/A | Independent | medium | arxiv.org |
Safety & Alignment
Safety & Alignment| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| AgentPrivArena | 19.3 | leak_rate_(%,_↓) | agentprivarena | Independent | medium | arxiv.org |
| DecepEval | 80.6 | deception_rate | N/A | Independent | medium | arxiv.org |
| ObligationBench | 33.6 | recall | N/A | Independent | medium | arxiv.org |
| OverAct | 2.4 | sir | N/A | Independent | medium | arxiv.org |
| SWE-DeceptionBench | 74.2 | active_deception_rate | N/A | Independent | medium | arxiv.org |
Multilingual
Multilingual| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| HakemBench | 0.6 | composite | N/A | Independent | medium | arxiv.org |
Reproduced by the community
Loading community runs…