LetTokenBurnMetaKernelBench: Measuring GPU Kernel Knowledge Transfer Beyond Code
Measures cross-DSL kernel knowledge transfer using 74 paired CuTe/TIRx problems, evaluating skill-conditioned vs solo attempts on correctness and runtime.
$ npm i -g lettokenburn-cli$ ltb bench info metakernelbench
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Gemini 3.7 Flash | 59.5 | cute dsl | Independent | medium | Oct 4, 2026 | arxiv.org |
| GPT-5.6 Sol | 56.8 | cute dsl | Independent | medium | Oct 4, 2026 | arxiv.org |
| GLM-5.3 | 51.4 | cute dsl | Independent | medium | Oct 4, 2026 | arxiv.org |
| GLM-5.3 Flash | 51.4 | tirx | Independent | medium | Oct 4, 2026 | arxiv.org |
| GLM-5.3 Flash | 36.5 | cute dsl | Independent | medium | Oct 4, 2026 | arxiv.org |
| Qwen3.8 Flash | 35.1 | cute dsl | Independent | medium | Oct 4, 2026 | arxiv.org |
| DeepSeek V4 Pro | 25.7 | cute dsl | Independent | medium | Oct 4, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
MetaKernelBench comprises 74 fused subgraph problems, each posed as paired variants in CuTe DSL and TIRx. Agents distill optimization findings into natural-language skills from a source DSL attempt, which are transferred to a fresh target DSL attempt.
Scoring compares skill-conditioned attempts against solo baselines on correctness and end-to-end runtime. Results show paired lift ranges from -19% to +29%, with outcomes dependent on the source attempt's relative standing rather than success alone.