DICE-Bench
A multi-variant math tutoring benchmark with skill-preserving problem variants for session-level evaluation of LLM tutors.
How to run
$ npm i -g lettokenburn-cli$ ltb bench info dice-bench
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
2 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Qwen2.5-7B-Instruct | 86.5 | dice-cost | Independent | medium | Oct 4, 2026 | arxiv.org |
| SocraticLM | 81.2 | dice-cost | Independent | medium | Oct 4, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsAbout
DICE-Bench consists of math problems with skill-preserving variants designed to evaluate tutor performance across entire simulated sessions rather than single turns. It assesses the ability of LLM tutors to intervene selectively and guide students effectively.
The benchmark evaluates intervention necessity and capability, focusing on reducing over-intervention while maintaining solution success. Key findings show DICE reduces over-intervention to near zero and solves problems in 3-4 fewer turns than baselines.