LetTokenBurnA benchmark for evaluating multi-turn image editing models on instruction following, content consistency, visual stability, and context use across up to 8 turns.
$ npm i -g lettokenburn-cli$ ltb bench info mtedit-bench
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Nano Banana Pro | 7.4 | N/A | Independent | medium | Oct 16, 2026 | arxiv.org |
| Nano Banana 2 | 7.2 | N/A | Independent | medium | Oct 16, 2026 | arxiv.org |
| Seedream 5.0 Lite | 6.9 | N/A | Independent | medium | Oct 16, 2026 | arxiv.org |
| Qwen-Image-Edit-2511 | 6.7 | N/A | Independent | medium | Oct 16, 2026 | arxiv.org |
| GPT-Image-2 | 6.6 | N/A | Independent | medium | Oct 16, 2026 | arxiv.org |
| FLUX.2-klein-9B | 6.5 | N/A | Independent | medium | Oct 16, 2026 | arxiv.org |
| SenseNova-U1 | 5.9 | N/A | Independent | medium | Oct 16, 2026 | arxiv.org |
| Qwen-Image-Edit-Causal | 5.8 | N/A | Independent | medium | Oct 16, 2026 | arxiv.org |
| FLUX.1-Kontext-dev | 5.0 | N/A | Independent | medium | Oct 16, 2026 | arxiv.org |
| Emu3.5 | 5.0 | N/A | Independent | medium | Oct 16, 2026 | arxiv.org |
| BAGEL | 4.6 | N/A | Independent | medium | Oct 16, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
MTEdit-Bench evaluates step-wise and context-aware multi-turn image editing. It assesses models' ability to follow instructions, maintain consistency, resist drift, and use history.
It complements the MTEdit dataset (500K trajectories). Evaluation shows Qwen-Image-MTEdit achieves top performance among open-source models on these metrics.