ARC-AGI-2
Harbor nativeARC-AGI-2: 更难的抽象推理
Verified100%Abstract Reasoning
- Samples
- 167
- Released
- 2025
- Scores
- 118
BrainTRACE
BrainTRACE: Tracing Longitudinal, Multimodal, and Volumetric Evidence in Brain MRI Clinical Reasoning
A report-grounded benchmark with 7,273 VQA instances from 1,778 longitudinal patients evaluating vision-language models on brain MRI evidence tracing.
How to run
ltb · braintrace
$ npm i -g lettokenburn-cli$ ltb bench pull braintrace --n 50$ ltb bench run ./braintrace -a claude-code -m anthropic/claude-opus-4-8$ ltb bench push
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
13 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| GPT-5 | 41.9 | N/A | Independent | medium | Oct 5, 2026 | arxiv.org |
| GPT-5.4 | 40.4 | N/A | Independent | medium | Oct 5, 2026 | arxiv.org |
| GPT-5-mini | 38.2 | N/A | Independent | medium | Oct 5, 2026 | arxiv.org |
| qwen3.5-27B | 37.3 | N/A | Independent | medium | Oct 5, 2026 | arxiv.org |
| Gemini 2.5 Pro | 36.5 | N/A | Independent | medium | Oct 5, 2026 | arxiv.org |
| InternVL3-8B | 35.8 | N/A | Independent | medium | Oct 5, 2026 | arxiv.org |
| InternVL3-78B | 35.5 | N/A | Independent | medium | Oct 5, 2026 | arxiv.org |
| InternVL3-14B | 35.1 | N/A | Independent | medium | Oct 5, 2026 | arxiv.org |
| Qwen3-VL-30B | 35.1 | N/A | Independent | medium | Oct 5, 2026 | arxiv.org |
| Qwen3-VL-8B-Instruct | 33.7 | N/A | Independent | medium | Oct 5, 2026 | arxiv.org |
| Qwen2.5-VL-7B | 33.2 | N/A | Independent | medium | Oct 5, 2026 | arxiv.org |
| Qwen3-VL-4B | 32.7 | N/A | Independent | medium | Oct 5, 2026 | arxiv.org |
| Claude Opus 4.6 | 29.8 | N/A | Independent | medium | Oct 5, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsLoading community runs…
About
BrainTRACE contains 7,273 scored VQA instances derived from 1,778 longitudinal patients and ~29k 3D MRI volumes. It evaluates five levels of clinical reasoning across four evidence demands: longitudinal comparison, multi-sequence integration, volumetric spatial evidence, and report-grounded references.
The benchmark uses exact closed-form scoring for specific questions and factual audit scoring for open-ended answers. Evaluation of 20 VLM configurations reveals a significant gap in composing grounded longitudinal interpretations, with the best case success rate at only 1.4%.