ARC-AGI-2
Harbor nativeARC-AGI-2: 更难的抽象推理
Verified100%Abstract Reasoning
- Samples
- 167
- Released
- 2025
- Scores
- 118
SpaceCast-Bench
SpaceCast-Bench: Evaluating Predictive Spatial Reasoning in Vision-Language Models
A benchmark with 3,862 questions across 182 scenes and 16 task types evaluating predictive spatial reasoning in vision-language models via an observe-transform-infer framework.
How to run
ltb · spacecast-bench
$ npm i -g lettokenburn-cli$ ltb bench info spacecast-bench
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
13 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Gemini-3.5-flash | 58.0 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| Gemini 3 Flash Preview | 56.7 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| Qwen3.7 Plus | 55.3 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| Qwen3.6 Flash | 47.7 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| GPT-5.6 Luna | 46.6 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| Doubao 2.0 Pro | 45.6 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| qwen3.5-27B | 39.3 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| Claude Sonnet 5 | 38.3 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| GPT-5.4 | 37.7 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| Claude Sonnet 4.6 | 36.0 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| qwen3.5-9B | 34.6 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| Qwen3-VL-4B | 34.0 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| Qwen3.5 4B | 33.7 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsLoading community runs…
About
SpaceCast-Bench evaluates predictive spatial reasoning using 3,862 questions from 182 real-world scenes. It employs a geometry-grounded pipeline to generate depth-verified view chains and programmatic labels for unobserved outcomes after controlled 3D transformations.
Scoring uses accuracy, revealing a significant gap where the strongest model achieves 58.0% against 87.2% human performance. The benchmark diagnoses bottlenecks in spatial state updating and shows that fine-tuning improves both in-domain and out-of-domain performance.