LetTokenBurn
Sign in
Verified100%Harbor nativeKnowledge & Reasoning

GPQA Diamond

Graduate-Level Google-Proof Q&A

GPQA Diamond is a graduate-level multiple-choice benchmark containing 198 questions in biology, physics, and chemistry designed to test models without reliance on search engines.

PaperRepoDatasetLeaderboard

How to run

ltb · gpqa-diamond
$ npm i -g lettokenburn-cli
$ ltb bench run gpqa-diamond/gpqa-diamond@1 --benchmark gpqa-diamond -a claude-code -m anthropic/claude-opus-4-8
$ ltb bench push

Harbor-native: you can also call Harbor directly.

harbor
$ harbor run -d gpqa-diamond/gpqa-diamond@1 -a claude-code -m anthropic/claude-opus-4-8

Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs

Scores

244 sourced entries
HarnessSourceConfidenceLink
GPT-6 Astra95.8N/AIndependenthighAug 30, 2026epoch.ai
Claude Sonnet 5.595.6N/AIndependenthighSep 29, 2026epoch.ai
GPT-6.1 Sol95.4N/AIndependenthighSep 29, 2026epoch.ai
Gemini 3.8 Flash95.4N/AIndependenthighSep 2, 2026epoch.ai
Gemini 3.7 Flash94.8N/AIndependenthighAug 14, 2026epoch.ai
GPT-5.4 Pro94.6N/AIndependenthighMar 20, 2026epoch.ai
Gemini 3.1 Pro94.4N/AIndependenthighAug 6, 2026epoch.ai
GPT-6 Sol94.3N/AIndependenthighSep 22, 2026epoch.ai
Gemini 3.6 Flash94.1N/AIndependenthighAug 2, 2026epoch.ai
Grok-4.694.0N/AIndependenthighAug 12, 2026epoch.ai
GPT-5.5 Pre Release94.0N/AIndependenthighApr 24, 2026epoch.ai
GPT-5.5 Pro Pre Release93.9N/AIndependenthighApr 24, 2026epoch.ai
Claude Opus 593.9N/AIndependenthighJul 24, 2026epoch.ai
Claude Opus 4.893.6N/ASelf-reportedlowJun 1, 2026artificialanalysis.ai
GPT-5.6 Sol93.5N/AIndependenthighJul 9, 2026epoch.ai
Grok 4.593.4N/AIndependenthighJul 8, 2026epoch.ai
GPT-5.6-terra93.3N/AIndependenthighJul 9, 2026epoch.ai
GPT-5.493.3N/AIndependenthighMar 6, 2026epoch.ai
Kimi K393.1N/AIndependenthighJul 16, 2026epoch.ai
Gemini-3.5-flash92.8N/AIndependenthighMay 22, 2026epoch.ai
GPT-5.492.8N/ASelf-reportedlowJun 1, 2026artificialanalysis.ai
Grok-4.792.7N/AIndependenthighSep 22, 2026epoch.ai
Qwen3.8 Max92.7N/AIndependenthighAug 4, 2026epoch.ai
Gemini 3 Pro92.6N/AIndependenthighNov 19, 2025epoch.ai
Qwen3.8 Max 090292.3N/AIndependenthighSep 2, 2026epoch.ai
Gemini 3.1 Pro91.9N/ASelf-reportedlowJun 1, 2026artificialanalysis.ai
GLM-5.291.9N/AIndependenthighJun 24, 2026epoch.ai
DeepSeek V4 Pro 081391.7N/AIndependenthighAug 18, 2026epoch.ai
GPT-5.6 Luna91.6N/AIndependenthighJul 9, 2026epoch.ai
GPT-5.2-chat91.4N/AIndependenthighDec 13, 2025epoch.ai
Claude Opus 4.691.3N/ASelf-reportedlowJun 1, 2026artificialanalysis.ai
DeepSeek V4 Flash 073191.0N/AIndependenthighAug 2, 2026epoch.ai
Claude Opus 4.891.0N/AIndependenthighJun 7, 2026epoch.ai
GLM-5.390.9N/AIndependenthighAug 24, 2026epoch.ai
MiniMax-M390.9N/AIndependenthighAug 10, 2026epoch.ai
Qwen3.7 Max90.9N/AIndependenthighAug 7, 2026epoch.ai
DeepSeek V4 Pro90.9N/AIndependenthighAug 6, 2026epoch.ai
Kimi K2.690.8N/AIndependenthighMay 1, 2026epoch.ai
GPT-5.590.7N/AIndependenthighMay 5, 2026epoch.ai
Claude Opus 5.590.6N/AIndependenthighSep 22, 2026epoch.ai
Claude Sonnet 590.5N/AIndependenthighJul 1, 2026epoch.ai
claude-opus-4-6-32K90.5N/AIndependenthighFeb 6, 2026epoch.ai
GPT-6 Luna90.5N/AIndependenthighSep 22, 2026epoch.ai
GLM-5.3 Flash90.2N/AIndependenthighAug 26, 2026epoch.ai
Claude Opus 4.790.2N/AIndependenthighApr 17, 2026epoch.ai
Claude Sonnet 4.689.9N/ASelf-reportedlowJun 1, 2026artificialanalysis.ai
GLM-5.189.9N/AIndependenthighAug 10, 2026epoch.ai
Muse Spark89.8N/AIndependenthighApr 8, 2026epoch.ai
Gemini 3 Flash Preview89.4N/AIndependenthighAug 6, 2026epoch.ai
Grok 4.20 0309 Reasoning89.3N/AIndependenthighJul 13, 2026epoch.ai
Top 50 of 244 by score

Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.

Reproduced by the community

All runs
Loading community runs…

About

Overview: GPQA (Graduate-Level Google-Proof Q&A) consists of 198 challenging multiple-choice questions spanning advanced topics in biology, physics, and chemistry.

Scoring & Significance: The benchmark uses Pass@1 scoring via exact-match accuracy. It is significant because its items are constructed to be difficult for models that rely on web search (Google-proof), requiring genuine reasoning over complex scientific knowledge rather than retrieval.