LetTokenBurnDataSense-Bench: The First Step Toward an AI Scientist
Evaluates AI agents' ability to select and rank training data subsets for fine-tuning LLMs, measuring post-training performance and ranking accuracy on terminal and tool-use tasks.
$ npm i -g lettokenburn-cli$ ltb bench info datasense-bench
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| DeepSeek V4 Pro | 36.5 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| GPT-6 Astra | 36.5 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| Kimi K3 | 36.3 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| Claude Opus 5 | 35.9 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| Fable 5.1 | 35.2 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| Gemini-3.5-flash | 35.1 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Agents inspect candidate data pools, execute analysis code, and run forward passes to select and rank disjoint subsets for fine-tuning a base model. They cannot train models or access evaluation tasks during selection.
Scoring measures post-training performance of the top-ranked subset and ranking accuracy against ground truth. Experiments show limited gains over random selection and inconsistent ranking abilities across tasks like TBLite and BFCL.