Auto71%Harbor adapterMath

AIME24

Name: AIME24
License: https://spdx.org/licenses/Apache-2.0.html

AIME24 — AI evaluation benchmark listed by pwc.

How to run

ltb · aime24

$ npm i -g lettokenburn-cli
$ ltb bench pull aime24 --n 50
$ ltb bench run ./aime24 -a claude-code -m anthropic/claude-opus-4-8
$ ltb bench push

Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs

5 sourced entries

		Harness	Source	Confidence		Link
DeepSeek R1	79.8	N/A	Independent	medium	Jan 22, 2025	arxiv.org
Openai-o1-preview	74.4	N/A	Independent	medium	N/A	paperswithcode.com
Openai-o1-mini	70.0	N/A	Independent	medium	N/A	paperswithcode.com
Qwen2.5-72B-Instruct	23.3	N/A	Independent	medium	Dec 19, 2024	arxiv.org
Claude3.5-Sonnet	16.0	N/A	Independent	medium	N/A	paperswithcode.com

Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.

AIME24 — AI evaluation benchmark listed by pwc.