DataVista
Harbor: pendingA benchmark with 961 real-world data videos and 6,775 multiple-choice questions evaluating MLLMs on data perception, temporal reasoning, and narrative understanding.
- Samples
- 6,775
- Released
- 2026
- Scores
- 13
4MT-VLM
4MT-VLM: How Coarse Is a VLMs Cognitive Map?
A diagnostic benchmark for embodied AI evaluating allocentric spatial memory in vision-language models using procedurally generated landscapes with stripped appearance cues across graded viewpoint shifts.
How to run
ltb · 4mt-vlm
$ npm i -g lettokenburn-cli$ ltb bench info 4mt-vlm
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
13 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Qwen2.5-VL-32B | 75.0 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| InternVL3.5-80B-A3B | 65.0 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| GPT-5.6 Luna | 45.0 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| Gemini 3.8 Flash | 44.0 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| Qwen2.5-VL-7B | 36.0 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| InternVL3.5-38B | 34.0 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| InternVL3.5-14B | 31.0 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| Qwen2.5-VL-72B-Instruct | 31.0 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| Qwen2.5-VL-30B-A3B | 29.0 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| InternVL3.5-8B-Instruct | 28.0 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| InternVL3.5-1B | 23.0 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| InternVL3.5-2B | 21.0 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| InternVL3.5-4B | 17.0 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsLoading community runs…
About
Content & Construction: The dataset consists of 500 trials over 100 procedurally generated landscapes, each rendered across five stimulus modes that remove appearance cues (shape/color) while holding layout fixed. It systematically tests models on identifying places from new viewpoints and distinguishing layouts based on physical distance in meters.
Scoring & Findings: Performance is measured using four-alternative forced choice (4AFC) accuracy, with chance at 25%. Results show that while frontier models can identify scenes without rotation, they fail to maintain stable spatial understanding once the viewpoint changes or distractors are too close, dropping below chance levels.