VeriEdit-Bench
A benchmark for fine-grained image editing with 1,740 cases from SVGs, charts, web interfaces, and slides, using deterministic four-axis evaluation.
How to run
$ npm i -g lettokenburn-cli$ ltb bench info veriedit-bench
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
11 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| GPT-Image-2 | 61.5 | N/A | Independent | medium | Oct 3, 2026 | arxiv.org |
| FLUX.2-dev | 6.4 | N/A | Independent | medium | Oct 3, 2026 | arxiv.org |
| Qwen-Image-Edit-2511 | 5.5 | N/A | Independent | medium | Oct 3, 2026 | arxiv.org |
| FLUX.2-klein | 3.3 | N/A | Independent | medium | Oct 3, 2026 | arxiv.org |
| FireRed-Edit | 3.0 | N/A | Independent | medium | Oct 3, 2026 | arxiv.org |
| Qwen-Image-Edit | 2.3 | N/A | Independent | medium | Oct 3, 2026 | arxiv.org |
| BAGEL | 0.9 | N/A | Independent | medium | Oct 3, 2026 | arxiv.org |
| Step1X-Edit | 0.9 | N/A | Independent | medium | Oct 3, 2026 | arxiv.org |
| InstructPix2Pix | 0.6 | N/A | Independent | medium | Oct 3, 2026 | arxiv.org |
| OmniGen2 | 0.5 | N/A | Independent | medium | Oct 3, 2026 | arxiv.org |
| FLUX.1-Kontext | 0.2 | N/A | Independent | medium | Oct 3, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsAbout
VeriEdit-Bench comprises 1,740 editing cases derived from 153 structured assets (SVGs, charts, web UIs, slides) via controlled source-code edits. It provides exact target images, pixel-level masks, and edit specifications.
Scoring uses a deterministic four-axis framework: edit fidelity, preservation, localization, and magnitude. Evaluation of eleven models reveals low performance (best composite 22/100), highlighting specific failure modes in localization and magnitude control.