AlgoREval
Harbor: pendingA benchmark of 599 problems spanning 77 algorithms, 14 domains, 7 languages, and 4 graph representations to evaluate parametric code retrieval in LLMs.
Auto86%Coding
- Samples
- 599
- Released
- 2026
- Scores
- 14
GSM8K
Grade School Math 8K
GSM8K: 小学应用题（多步推理）
How to run
ltb · gsm8k
$ npm i -g lettokenburn-cli$ ltb bench pull gsm8k --n 50$ ltb bench run ./gsm8k -a claude-code -m anthropic/claude-opus-4-8$ ltb bench push
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
180 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Qwen2-Math-72B-Instruct | 96.7 | N/A | Independent | medium | Jul 15, 2024 | arxiv.org |
| DeepSeek Coder V2 | 94.5 | N/A | Independent | medium | N/A | arxiv.org |
| Qwen2.5-Coder-14B | 94.2 | N/A | Self-reported | medium | N/A | arxiv.org |
| Qwen2.5-Coder-32B-Instruct | 93.0 | N/A | Self-reported | medium | N/A | arxiv.org |
| GPT-4 | 92.0 | N/A | Independent | medium | N/A | arxiv.org |
| GPT-4 0314 | 92.0 | N/A | Self-reported | medium | N/A | arxiv.org |
| GPT-4o-mini | 91.3 | N/A | Independent | medium | N/A | arxiv.org |
| GPT-4-0613 | 90.0 | N/A | Independent | medium | N/A | arxiv.org |
| Phi-3.5-MoE-instruct | 88.7 | N/A | Self-reported | medium | N/A | arxiv.org |
| DeepSeekMATH-RL-7B | 88.2 | N/A | Independent | medium | Feb 5, 2024 | arxiv.org |
| DeepSeek-Coder-V2-Lite-Base | 87.6 | N/A | Independent | medium | N/A | arxiv.org |
| GPT-4 | 87.1 | N/A | Independent | medium | Mar 22, 2023 | arxiv.org |
| Claude Instant 1.2 | 86.7 | N/A | Self-reported | medium | N/A | anthropic.com |
| Qwen2.5-Coder-7B | 86.7 | N/A | Self-reported | medium | N/A | arxiv.org |
| Phi-3.5-mini-instruct | 86.2 | N/A | Self-reported | medium | N/A | arxiv.org |
| Gemma 2 9B IT | 84.9 | N/A | Independent | medium | N/A | arxiv.org |
| Mistral-Nemo-Instruct-2407 | 84.2 | N/A | Independent | medium | N/A | arxiv.org |
| WizardMath-7B-V1.1 | 83.2 | N/A | Independent | medium | Aug 18, 2023 | arxiv.org |
| gemini-1.5-flash-001 | 82.4 | N/A | Independent | medium | N/A | arxiv.org |
| Llama-3.1-8B-Instruct | 82.4 | N/A | Independent | medium | N/A | arxiv.org |
| MetaMath 70B | 82.3 | N/A | Independent | medium | Sep 21, 2023 | arxiv.org |
| WizardMath-70B-V1.0 | 81.6 | N/A | Independent | medium | Aug 18, 2023 | arxiv.org |
| Claude Instant 1.1 | 80.9 | N/A | Self-reported | medium | N/A | anthropic.com |
| PaLM 2 | 80.7 | N/A | Independent | medium | May 17, 2023 | arxiv.org |
| Qwen2.5-Coder-3B | 80.7 | N/A | Self-reported | medium | N/A | arxiv.org |
| Minerva 540B | 78.5 | N/A | Independent | medium | Jun 29, 2022 | arxiv.org |
| text-davinci-003 | 78.2 | N/A | Independent | medium | N/A | arxiv.org |
| Qwen2idae-16x14B | 77.8 | N/A | Independent | medium | Jan 5, 2024 | arxiv.org |
| MetaMath-Mistral-7B | 77.7 | N/A | Independent | medium | Sep 21, 2023 | arxiv.org |
| Yi 34B | 76.0 | N/A | Independent | medium | N/A | arxiv.org |
| PaLM 540B maj1@40 | 74.4 | N/A | Independent | medium | Mar 21, 2022 | arxiv.org |
| Mixtral-8x7B-Instruct-v0.1 | 74.4 | N/A | Self-reported | medium | N/A | arxiv.org |
| DBRX Instruct 132B | 72.3 | N/A | Independent | medium | N/A | paperswithcode.com |
| MetaMath 13B | 71.0 | N/A | Independent | medium | Sep 21, 2023 | arxiv.org |
| LLaMA 65B-maj1@k | 69.7 | N/A | Independent | medium | Feb 27, 2023 | arxiv.org |
| Llama 2 70B | 69.6 | N/A | Independent | medium | N/A | arxiv.org |
| StableBeluga2 | 69.6 | N/A | Independent | medium | N/A | arxiv.org |
| Minerva 540B | 68.5 | N/A | Independent | medium | N/A | arxiv.org |
| MetaMath 7B | 66.4 | N/A | Independent | medium | Sep 21, 2023 | arxiv.org |
| Qwen2.5-Coder-1.5B | 65.8 | N/A | Self-reported | medium | N/A | arxiv.org |
| WizardMath-13B-V1.0 | 63.9 | N/A | Independent | medium | Aug 18, 2023 | arxiv.org |
| Palmyra X | 63.3 | N/A | Independent | medium | N/A | crfm.stanford.edu |
| InternLM 20B | 62.9 | N/A | Independent | medium | N/A | arxiv.org |
| Qwen-14B | 61.3 | N/A | Independent | medium | N/A | arxiv.org |
| Qwen-14B | 61.3 | N/A | Self-reported | medium | N/A | arxiv.org |
| 8-shot Minerva | 58.8 | N/A | Independent | medium | N/A | arxiv.org |
| GPT-3.5 Turbo 0613 | 57.8 | N/A | Independent | medium | N/A | arxiv.org |
| StarCoder2-15B | 57.7 | N/A | Independent | medium | N/A | arxiv.org |
| GPT-3.5 Turbo | 57.1 | N/A | Independent | medium | Mar 15, 2023 | arxiv.org |
| code-davinci-002 | 56.8 | N/A | Independent | medium | N/A | crfm.stanford.edu |
Top 50 of 180 by score
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsLoading community runs…
About
GSM8K (Grade School Math 8K) — 小学应用题（多步推理）. Scoring: 准确率. Difficulty: 中（饱和）. Imported from benchmark_research.md (2026-06).