LetTokenBurnNegT2IBench: When Negation Changes the Picture. A Polarity Benchmark for Text-to-Image Models
A benchmark of 4,800 prompts evaluating text-to-image models' ability to satisfy negated constraints (attributes and relations) while maintaining object presence.
$ npm i -g lettokenburn-cli$ ltb bench info negt2ibench
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Qwen3-VL-32B | 76.5 | N/A | Independent | medium | Oct 2, 2026 | arxiv.org |
| Qwen3-VL-8B-Instruct | 74.0 | N/A | Independent | medium | Oct 2, 2026 | arxiv.org |
| InternVL3.5-38B | 73.9 | N/A | Independent | medium | Oct 2, 2026 | arxiv.org |
| InternVL3-8B | 69.5 | N/A | Independent | medium | Oct 2, 2026 | arxiv.org |
| InternVL3.5-8B-Instruct | 69.2 | N/A | Independent | medium | Oct 2, 2026 | arxiv.org |
| Qwen2.5-VL-7B | 58.4 | N/A | Independent | medium | Oct 2, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
NegT2IBench comprises 4,800 prompts across 79 object classes, covering color, material, and spatial relations. It systematically varies polarity by mixing positive and negated statements to isolate negation effects from prompt complexity.
Scoring uses detector-based methods for reproducibility, validated against human annotations. Results show most models perform significantly worse on negated statements, with 41.5% of failures involving rendering forbidden content.