LetTokenBurnAI2 Reasoning Challenge
ARC-Eval is a dataset of 7,787 multiple-choice science questions designed to evaluate reasoning capabilities in AI systems.
$ npm i -g lettokenburn-cli$ ltb bench pull arc-eval --n 50$ ltb bench run ./arc-eval -a claude-code -m anthropic/claude-opus-4-8$ ltb bench push
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
No sourced scores recorded yet.
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
The AI2 Reasoning Challenge consists of over 7,000 grade-school level scientific problems. It tests logical and common-sense reasoning rather than rote knowledge. Performance is measured using exact-match accuracy on the provided answer choices.