Model
DeepSeek-V3
DeepSeek · open weights
Knowledge & Reasoning
Knowledge & Reasoning| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| BBH | 87.5 | average | N/A | Self-reported | medium | arxiv.org |
| CritPt | 0.0 | accuracy | N/A | Independent | low | epoch.ai |
| GPQA Diamond | 56.5 | accuracy | N/A | Independent | high | epoch.ai |
| HellaSwag | 88.9 | overall_accuracy | N/A | Self-reported | medium | arxiv.org |
| LiveBench | 60.5 | global_average | N/A | Independent | medium | livebench.ai |
| MMLU | 87.2 | exact_match | N/A | Independent | medium | crfm.stanford.edu |
| MMLU | 87.1 | exact_match | N/A | Self-reported | medium | arxiv.org |
| PIQA | 84.7 | score | N/A | Self-reported | medium | arxiv.org |
| Trivia QA | 82.9 | exact_match | N/A | Self-reported | medium | arxiv.org |
| WinoGrande | 85.2 | accuracy | N/A | Self-reported | medium | arxiv.org |
Coding
Coding| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| Aider Polyglot | 48.4 | percent_correct | N/A | Independent | medium | aider.chat |
| SciCode | 35.4 | score | N/A | Independent | medium | artificialanalysis.ai |
Agentic / Tool Use
Agentic / Tool Use| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| EHR-RobustGym | 55.5 | success_rate | N/A | Independent | medium | arxiv.org |
| EnterpriseBench | 8.0 | beergame_cost_x10^4_(lower_is_better) | dc | Independent | medium | arxiv.org |
| EnterpriseBench | 7.7 | beergame_cost_x10^4_(lower_is_better) | ace | Independent | medium | arxiv.org |
| EnterpriseBench | 7.6 | beergame_cost_x10^4_(lower_is_better) | gepa | Independent | medium | arxiv.org |
| EnterpriseBench | 7.4 | beergame_cost_x10^4_(lower_is_better) | amem | Independent | medium | arxiv.org |
| EnterpriseBench | 7.1 | beergame_cost_x10^4_(lower_is_better) | single-agent | Independent | medium | arxiv.org |
| EnterpriseBench | 6.8 | beergame_cost_x10^4_(lower_is_better) | discussion | Independent | medium | arxiv.org |
| EnterpriseBench | 6.8 | beergame_cost_x10^4_(lower_is_better) | reflexion | Independent | medium | arxiv.org |
| EnterpriseBench | 6.5 | beergame_cost_x10^4_(lower_is_better) | debate | Independent | medium | arxiv.org |
| EnterpriseBench | 6.4 | beergame_cost_x10^4_(lower_is_better) | self-refine | Independent | medium | arxiv.org |
Multimodal
MultimodalSafety & Alignment
Safety & AlignmentMultilingual
Multilingual| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| Cross-Lingual Unlearning Tensor | 88.9 | residual_access | N/A | Independent | medium | arxiv.org |
Reproduced by the community
Loading community runs…