VTR-Bench
VTR-Bench: A Systematic Benchmark for Evaluating Visual Text Rendering in Video Generation
A benchmark with 300 prompts across 5 scenarios evaluating visual text rendering fidelity and scene adherence in generated videos.
How to run
$ npm i -g lettokenburn-cli$ ltb bench info vtr-bench
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
11 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Wan3.0 | 0.2 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| MiniMax-H3 | 0.5 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| Seedance2.5 | 0.6 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| HappyHorse1.1 | 0.9 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| Lingbot-Video-MOE | 0.9 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| ViduQ3 | 0.9 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| HunyuanVideo-1.5 | 1.0 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| Kling v3.0 | 1.0 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| Lingbot-Video-Dense | 1.0 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| LTX-2.3 | 1.0 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| Wan2.2-5B | 1.0 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsAbout
VTR-Bench comprises 300 carefully constructed prompts spanning advertising, science, UIs, culture, and daily life. Each prompt includes reference strings and carrier annotations for text targets within coherent video scenarios.
It uses an automated pipeline assessing text fidelity via Word Error Rate (WER) and scene requirements via a Video Score. Experiments on 11 models show significant text rendering challenges, with the best model achieving a WER of 0.250.