LetTokenBurn
Sign in
Auto71%Harbor: pendingMultimodalLong Context

VL-DocIR

VL-DocIR: A Benchmark for Vision-Based Long Document Information Retrieval

A page-level benchmark for evidence-complete visual long-document retrieval with 29,641 documents, 388,548 page images, and 271,760 queries across 23 domains.

Paper

How to run

ltb · vl-docir
$ npm i -g lettokenburn-cli
$ ltb bench info vl-docir

External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.

Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs

Scores

3 sourced entries
HarnessSourceConfidenceLink
ColNomic70.3N/AIndependentmediumOct 12, 2026arxiv.org
ColQwen368.5N/AIndependentmediumOct 12, 2026arxiv.org
ColQwen2.565.0N/AIndependentmediumOct 12, 2026arxiv.org

Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.

Reproduced by the community

All runs
Loading community runs…

About

VL-DocIR comprises rendered page images from Wikipedia, arXiv, PubMed, and SEC filings, paired with natural-language queries requiring single-page, multi-page, or cross-document evidence. It evaluates retrievers on evidence completeness, measuring if all ground-truth pages are retrieved within the top-k results.

Evaluation uses nDCG and Recall metrics under structure-balanced Macro evaluation. Results show cross-document retrieval is significantly harder than single-page tasks, with document-context scoring improving performance for within-document multi-page queries.