LetTokenBurn
Sign in
Verified86%Harbor adapterKnowledge & Reasoning

MMLU-SR

Title: Reasoning or Simply Next Token Prediction?

PaperDataset

How to run

ltb · mmlu-sr
$ npm i -g lettokenburn-cli
$ ltb bench pull mmlu-sr --n 50
$ ltb bench run ./mmlu-sr -a claude-code -m anthropic/claude-opus-4-8
$ ltb bench push

Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs

Scores

0 sourced entries

No sourced scores recorded yet.

Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.

Reproduced by the community

All runs
Loading community runs…

About

Title: Reasoning or Simply Next Token Prediction? A Benchmark for Stress-Testing Large Language Models

Available in EleutherAI lm-evaluation-harness as task family mmlusr (171 tasks).