LetTokenBurnHumanity's Last Exam
Humanity's Last Exam is a rigorous knowledge benchmark comprising 2,500 expert-verified questions across multiple disciplines designed to evaluate the upper limits of AI reasoning capabilities.
$ npm i -g lettokenburn-cli$ ltb bench pull hle
External harness. pip install inspect-evals && inspect eval inspect_evals/hle --model <provider/model> https://github.com/UKGovernmentBEIS/inspect_evals
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Claude Opus 4.8 | 57.9 | N/A | Self-reported | low | Jun 1, 2026 | lastexam.ai |
| GPT-6 Astra | 54.8 | N/A | Independent | low | N/A | epoch.ai |
| Claude Opus 4.6 | 53.0 | N/A | Self-reported | low | Jun 1, 2026 | lastexam.ai |
| GLM-5 | 50.4 | N/A | Self-reported | low | Jun 1, 2026 | lastexam.ai |
| Claude Sonnet 4.6 | 49.0 | N/A | Self-reported | low | Jun 1, 2026 | lastexam.ai |
| Claude Fable 5.1 | 46.5 | N/A | Independent | low | N/A | epoch.ai |
| Gemini 3.1 Pro | 46.4 | N/A | Independent | low | N/A | epoch.ai |
| Gemini 3.1 Pro | 45.8 | N/A | Self-reported | low | Jun 1, 2026 | lastexam.ai |
| Gemini 3 Pro | 45.8 | N/A | Self-reported | low | Jun 1, 2026 | lastexam.ai |
| Gemini 3.8 Flash | 44.5 | N/A | Independent | low | N/A | epoch.ai |
| GPT-5.4 Pro | 44.3 | N/A | Independent | low | N/A | epoch.ai |
| Muse Spark | 40.6 | N/A | Independent | low | N/A | epoch.ai |
| Gemini 3 Pro | 37.5 | N/A | Independent | low | N/A | epoch.ai |
| GPT-5.4 | 36.2 | N/A | Independent | low | N/A | epoch.ai |
| Claude Opus 4.7 | 36.2 | N/A | Independent | low | N/A | epoch.ai |
| Claude Opus 4.6 | 34.4 | N/A | Independent | low | N/A | epoch.ai |
| GPT-5 Pro (2025-10-06) | 31.6 | N/A | Independent | low | N/A | epoch.ai |
| Qwen 3.5 | 28.7 | N/A | Self-reported | low | Jun 1, 2026 | lastexam.ai |
| GPT-5.2-chat | 27.8 | N/A | Independent | low | N/A | epoch.ai |
| GPT-5 | 25.3 | N/A | Independent | low | N/A | epoch.ai |
| Claude Opus 4.5 (2025-11-01) | 25.2 | N/A | Independent | low | N/A | epoch.ai |
| Kimi K2.5 | 24.4 | N/A | Independent | low | N/A | epoch.ai |
| GPT-5.1 | 23.7 | N/A | Independent | low | N/A | epoch.ai |
| Gemini 2.5 Pro Preview 06-05 | 21.6 | N/A | Independent | low | N/A | epoch.ai |
| o3 | 20.3 | N/A | Independent | low | N/A | epoch.ai |
| GPT-5-mini | 19.4 | N/A | Independent | low | N/A | epoch.ai |
| Gemini 2.5 Pro Exp 03-25 | 18.2 | N/A | Independent | low | N/A | epoch.ai |
| o4-mini | 18.1 | N/A | Independent | low | N/A | epoch.ai |
| Gemini 2.5 Pro Preview 05-06 | 17.8 | N/A | Independent | low | N/A | epoch.ai |
| Claude Sonnet 4.5 (2025-09-29) | 13.7 | N/A | Independent | low | N/A | epoch.ai |
| Gemini 2.5 Flash Preview 04-17 | 12.1 | N/A | Independent | low | N/A | epoch.ai |
| Claude Opus 4.1 (2025-08-05) | 11.5 | N/A | Independent | low | N/A | epoch.ai |
| Gemini 2.5 Flash Preview 05-20 | 11.0 | N/A | Independent | low | N/A | epoch.ai |
| Claude Opus 4 (2025-05-14) | 10.7 | N/A | Independent | low | N/A | epoch.ai |
| Gemini 3.1 Flash Lite | 8.6 | N/A | Independent | low | N/A | epoch.ai |
| GLM-4.5 | 8.3 | N/A | Independent | low | N/A | epoch.ai |
| GLM-4.5-Air | 8.1 | N/A | Independent | low | N/A | epoch.ai |
| o1-pro-2025-03-19 | 8.1 | N/A | Independent | low | N/A | epoch.ai |
| Claude 3.7 Sonnet Thinking (2025-02-19) | 8.0 | N/A | Independent | low | N/A | epoch.ai |
| OpenAI/o1-2024-12-17-high | 8.0 | N/A | Independent | low | N/A | epoch.ai |
| Claude Sonnet 4 (2025-05-14) | 7.8 | N/A | Independent | low | N/A | epoch.ai |
| Gemini 2.0 Flash Thinking Exp 01-21 | 6.6 | N/A | Independent | low | N/A | epoch.ai |
| Llama-4-Maverick-17B-128E-Instruct | 5.7 | N/A | Independent | low | N/A | epoch.ai |
| GPT-4.5 Preview (2025-02-27) | 5.4 | N/A | Independent | low | N/A | epoch.ai |
| GPT-4.1 | 5.4 | N/A | Independent | low | N/A | epoch.ai |
| gemini-1.5-pro-002 | 4.6 | N/A | Independent | low | N/A | epoch.ai |
| Mistral Medium 2505 | 4.5 | N/A | Independent | low | N/A | epoch.ai |
| amazon.nova-pro-v1:0 | 4.4 | N/A | Independent | low | N/A | epoch.ai |
| Claude 3.5 Sonnet | 4.1 | N/A | Independent | low | N/A | epoch.ai |
| amazon.nova-lite-v1:0 | 3.6 | N/A | Independent | low | N/A | epoch.ai |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Overview: Humanity's Last Exam (HLE) consists of 2,500 high-difficulty questions curated and reviewed by experts from various academic fields. The benchmark targets the frontier of current model performance.
Scoring & Significance: Models are evaluated using Pass@1 accuracy on these expert-validated items. HLE matters because it provides a stringent test for identifying whether AI systems can handle complex, interdisciplinary knowledge at an elite level.