Video-MME
Harbor adapterVideo-MME stands for Video Multi-Modal Evaluation.
- Samples
- 2,700
- Released
- 2024
- Scores
- 51
MindCube
MindCube: Spatial Mental Modeling from Limited Views MindCube is a novel benchmark designed to evaluate how well Vision Language Models (VLMs) can form robust spatial mental models from limited visual views.
How to run
ltb · mindcube
$ npm i -g lettokenburn-cli$ ltb bench info mindcube
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
5 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Gemma 3 12B IT | 46.7 | N/A | Independent | medium | N/A | mind-cube.github.io |
| mPLUG-Owl3-7B-241101 | 44.9 | N/A | Independent | medium | N/A | mind-cube.github.io |
| Claude Sonnet 4 (2025-05-14) | 44.8 | N/A | Independent | medium | N/A | mind-cube.github.io |
| LLaVA-Video-7B-Qwen2 | 42.0 | N/A | Independent | medium | N/A | mind-cube.github.io |
| LongVA-7B | 29.5 | N/A | Independent | medium | N/A | mind-cube.github.io |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsLoading community runs…
About
MindCube: Spatial Mental Modeling from Limited Views
MindCube is a novel benchmark designed to evaluate how well Vision Language Models (VLMs) can form robust spatial mental models from limited visual views. It comprises 21,154 questions across 3,268 images, assessing capabilities such as cognitive mapping (representing positions), perspective-taking (orientations), and mental simulation (dynamics for "what-if" movements). The dataset aims to expose critical gaps in existing VLMs'…