LetTokenBurnTasteVal: Measuring the Experimental Research Taste of AI Systems Against Human Experts
Evaluates experimental research taste via 8 open-ended AI R&D tasks, measuring compute efficiency relative to human experts using a fixed Coder agent.
$ npm i -g lettokenburn-cli$ ltb bench info tasteval
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Claude Opus 5.5 | 2.3 | N/A | Independent | medium | Oct 5, 2026 | arxiv.org |
| Claude Opus 5 | 2.1 | N/A | Independent | medium | Oct 5, 2026 | arxiv.org |
| Fable 5.1 | 1.7 | N/A | Independent | medium | Oct 5, 2026 | arxiv.org |
| GPT-6 Astra | 1.6 | N/A | Independent | medium | Oct 5, 2026 | arxiv.org |
| Claude Opus 4.8 | 0.8 | N/A | Independent | medium | Oct 5, 2026 | arxiv.org |
| GLM-5.3 | 0.6 | N/A | Independent | medium | Oct 5, 2026 | arxiv.org |
| GPT-5.6 Sol | 0.6 | N/A | Independent | medium | Oct 5, 2026 | arxiv.org |
| Claude Opus 4.7 | 0.5 | N/A | Independent | medium | Oct 5, 2026 | arxiv.org |
| GPT-5.5 | 0.5 | N/A | Independent | medium | Oct 5, 2026 | arxiv.org |
| Kimi K3 | 0.5 | N/A | Independent | medium | Oct 5, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
TasteVal comprises 8 novel, open-ended tasks covering AI R&D areas like pre-training and alignment. It isolates research taste from coding by having the evaluated model (Researcher) design experiments while a fixed Coder agent implements them.
Scoring uses a compute multiplier metric, comparing GPU hours needed to match expert performance. The best model, Opus 5.5, achieved a 2.3x multiplier over the expert baseline, indicating superior experimental efficiency.