Model
DeepSeek R1
DeepSeek · open weights
Knowledge & Reasoning
Knowledge & Reasoning| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| Balrog | 34.9 | average_progress | N/A | Independent | medium | balrogai.com |
| CritPt | 1.1 | accuracy | N/A | Independent | low | epoch.ai |
| GPQA Diamond | 71.7 | accuracy | N/A | Independent | high | epoch.ai |
| GPQA Diamond | 71.5 | accuracy | N/A | Self-reported | low | artificialanalysis.ai |
| LiveBench | 71.6 | global_average | N/A | Independent | medium | livebench.ai |
| LMArena | 1398.0 | elo | N/A | Self-reported | low | lmarena.ai |
| MMLU-Pro | 84.0 | accuracy | N/A | Self-reported | low | artificialanalysis.ai |
| News QA | 80.6 | exact_match | N/A | Independent | medium | arxiv.org |
Math
Math| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| AIME 2025 | 87.5 | accuracy | N/A | Self-reported | low | kaggle.com |
| AIME24 | 79.8 | accuracy | N/A | Independent | medium | arxiv.org |
| MATH-500 | 97.3 | accuracy | N/A | Self-reported | low | kaggle.com |
Coding
Coding| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| Aider Polyglot | 56.9 | percent_correct | N/A | Independent | medium | aider.chat |
| LiveCodeBench | 65.9 | pass@1 | N/A | Self-reported | low | livecodebench.github.io |
| SciCode | 35.7 | score | N/A | Independent | medium | artificialanalysis.ai |
| SWE-bench Verified | 49.2 | resolved_rate | N/A | Self-reported | low | swebench.com |
| Web-Bench | 14.5 | pass@1 | N/A | Independent | medium | arxiv.org |
| WebUIProof | 41.9 | success_rate | N/A | Independent | medium | arxiv.org |
Agentic / Tool Use
Agentic / Tool Use| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| XiangqiBench | 1.7 | pass@3 | N/A | Independent | medium | arxiv.org |
Multimodal
Multimodal| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| R-GroundBench | 66.0 | accuracy | N/A | Independent | medium | arxiv.org |
Instruction Following
Instruction Following| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| IFEval | 83.3 | prompt_strict_accuracy | N/A | Self-reported | low | artificialanalysis.ai |
Safety & Alignment
Safety & AlignmentAbstract Reasoning
Abstract ReasoningReproduced by the community
Loading community runs…