BrainTRACE
Harbor adapterA report-grounded benchmark with 7,273 VQA instances from 1,778 longitudinal patients evaluating vision-language models on brain MRI evidence tracing.
- Samples
- 7,273
- Released
- 2026
- Scores
- 13
SpatialChain
SpatialChain: A Benchmark for Auditing Spatial Reasoning Faithfulness in VLMs
A dataset of 28,350 training and 899 test examples pairing spatial GQA questions with scene-graph-grounded reasoning chains to audit VLM faithfulness.
How to run
ltb · spatialchain
$ npm i -g lettokenburn-cli$ ltb bench info spatialchain
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
9 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Qwen3-VL-8B FT | 85.0 | N/A | Independent | medium | Oct 5, 2026 | arxiv.org |
| Qwen3-VL-4B FT | 82.2 | N/A | Independent | medium | Oct 5, 2026 | arxiv.org |
| GLM-4.1-9B | 81.3 | N/A | Independent | medium | Oct 5, 2026 | arxiv.org |
| InternVL3.5-4B | 81.3 | N/A | Independent | medium | Oct 5, 2026 | arxiv.org |
| Claude Sonnet 4.6 | 79.9 | N/A | Independent | medium | Oct 5, 2026 | arxiv.org |
| InternVL3.5-8B-Instruct | 79.3 | N/A | Independent | medium | Oct 5, 2026 | arxiv.org |
| Qwen3-VL-8B-Instruct | 78.9 | N/A | Independent | medium | Oct 5, 2026 | arxiv.org |
| Qwen3-VL-4B | 78.4 | N/A | Independent | medium | Oct 5, 2026 | arxiv.org |
| GPT-5.4 | 72.2 | N/A | Independent | medium | Oct 5, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsLoading community runs…
About
SpatialChain pairs spatially-oriented GQA questions with reasoning chains derived from structured scene graphs. It includes 28,350 training and 899 test examples, filtered for spatial relevance and verified against symbolic ground truth.
Evaluation uses a two-axis protocol: objective chain-overlap metrics and a scene-graph-aware LLM judge scoring faithfulness and completeness. This reveals shortcut rates >39% in high-accuracy models and validates judge agreement with human annotators.