FoveDoc-Bench
A benchmark with 2346 questions over 1173 born-digital documents, featuring traceable evidence pages and blocks to measure the retrieval-reading gap in document VLMs.
How to run
$ npm i -g lettokenburn-cli$ ltb bench info fovedoc-bench
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
1 sourced entry| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Qwen3-VL-2B | 0.2 | N/A | Independent | medium | Oct 2, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsAbout
FoveDoc-Bench comprises 2346 questions derived from 1173 held-out born-digital documents, each annotated with specific evidence pages, blocks, and excerpts. It supports tasks like V-NIAH and V-MQAR to isolate reading failures from retrieval errors.
Scoring uses strict accuracy graded by an LLM judge. Key findings show that adding extracted text to retrieved images improves accuracy by 13-16 points, highlighting a significant gap where models find but fail to read evidence.