MMLU
Harbor adapterMMLU evaluates English language models on knowledge and reasoning across 57 diverse subjects using a multiple-choice format with over 14k samples.
- Samples
- 14,042
- Released
- 2020
- Scores
- 209
MMLU-Pro
MMLU-Professional
MMLU-Pro is a robust multi-task language understanding benchmark comprising over 12,000 graduate-level multiple-choice questions across 14 academic disciplines.
How to run
ltb · mmlu-pro
$ npm i -g lettokenburn-cli$ ltb bench pull mmlu-pro --n 50$ ltb bench run ./mmlu-pro -a claude-code -m anthropic/claude-opus-4-8$ ltb bench push
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
13 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| GPT-oss 120B | 90.0 | N/A | Self-reported | low | Jun 1, 2026 | artificialanalysis.ai |
| Qwen 3.5 | 87.8 | N/A | Self-reported | low | Jun 1, 2026 | artificialanalysis.ai |
| Kimi K2.5 | 87.1 | N/A | Self-reported | low | Jun 1, 2026 | artificialanalysis.ai |
| DeepSeek V3.2 | 85.0 | N/A | Self-reported | low | Jun 1, 2026 | artificialanalysis.ai |
| Gemini 3.1 Pro | 85.0 | N/A | Self-reported | low | Jun 1, 2026 | artificialanalysis.ai |
| Gemini 3 Pro | 85.0 | N/A | Self-reported | low | Jun 1, 2026 | artificialanalysis.ai |
| MiMo-V2-Flash | 84.9 | N/A | Self-reported | low | Jun 1, 2026 | artificialanalysis.ai |
| DeepSeek R1 | 84.0 | N/A | Self-reported | low | Jun 1, 2026 | artificialanalysis.ai |
| Claude Opus 4.6 | 82.0 | N/A | Self-reported | low | Jun 1, 2026 | artificialanalysis.ai |
| Llama 4 Maverick | 80.5 | N/A | Self-reported | low | Jun 1, 2026 | artificialanalysis.ai |
| Claude Sonnet 4.6 | 79.1 | N/A | Self-reported | low | Jun 1, 2026 | artificialanalysis.ai |
| MiniMax M2.5 | 76.5 | N/A | Self-reported | low | Jun 1, 2026 | artificialanalysis.ai |
| GLM-5 | 70.4 | N/A | Self-reported | low | Jun 1, 2026 | artificialanalysis.ai |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsLoading community runs…
About
Composition and Scope: MMLU-Pro expands the original MMLU by introducing significantly harder problems derived from professional textbooks and research papers. It covers 14 diverse fields including STEM, humanities, and social sciences.
Scoring and Utility: The benchmark is scored using exact-match accuracy on a multiple-choice format (A-E). Its primary value lies in evaluating large language models' ability to handle complex reasoning tasks that require deep domain knowledge rather than simple pattern matching.