ScribbleEdit
ScribbleEdit: A Benchmark for Scribble-Only Image Editing
Evaluates image editing models on scribble-only inputs using 250 manually labeled evaluation pairs and a classification-based intention alignment metric.
How to run
$ npm i -g lettokenburn-cli$ ltb bench info scribbleedit
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
7 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Nano Banana | 0.3 | N/A | Independent | medium | Oct 7, 2026 | arxiv.org |
| GPT | 0.3 | N/A | Independent | medium | Oct 7, 2026 | arxiv.org |
| BAGEL | 0.3 | N/A | Independent | medium | Oct 7, 2026 | arxiv.org |
| FLUX | 0.2 | N/A | Independent | medium | Oct 7, 2026 | arxiv.org |
| OmniGen2 | 0.2 | N/A | Independent | medium | Oct 7, 2026 | arxiv.org |
| Qwen Max (2025-01-25) | 0.2 | N/A | Independent | medium | Oct 7, 2026 | arxiv.org |
| Step1X | 0.2 | N/A | Independent | medium | Oct 7, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsAbout
ScribbleEdit contains 23,772 synthetic training pairs and 250 manually labeled evaluation pairs for scribble-based image editing. It uses an automated pipeline for data construction.
Scoring combines visual similarity with a classification-based metric for intention alignment. Existing VLM/LLM models fail to accurately capture scribble intentions, while the proposed SSE baseline outperforms them.