LetTokenBurnD2K-Bench: Can LLM Agents Turn Expert Designs into Efficient GPU Kernels?
A diagnostic benchmark of 26 tasks and 85 workloads measuring how effectively LLM agents translate expert design guidance into efficient GPU kernels.
$ npm i -g lettokenburn-cli$ ltb bench info d2k-bench
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| GPT-6 Astra | 3.3 | N/A | Independent | medium | Oct 2, 2026 | arxiv.org |
| Claude Opus 4.8 | 2.3 | N/A | Independent | medium | Oct 2, 2026 | arxiv.org |
| GPT-5.6 Sol | 2.0 | N/A | Independent | medium | Oct 2, 2026 | arxiv.org |
| Kimi K3 | 1.4 | N/A | Independent | medium | Oct 2, 2026 | arxiv.org |
| GLM-5.3 | 1.3 | N/A | Independent | medium | Oct 2, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
D2K-Bench comprises 26 GPU kernel optimization tasks with 85 workloads, featuring expert design guidance at algorithmic, dataflow, and low-level optimization levels. It uses pairwise runs with and without guidance to assess design discovery and implementation.
Scoring involves correctness rates, a Performance Score, and geometric mean speedup on NVIDIA B200 GPUs. Guidance improved correctness from 93.1% to 98.5% and increased the Performance Score from 1.46 to 1.95 across five models.