ARC-AGI-2
Harbor nativeARC-AGI-2: 更难的抽象推理
Verified100%Abstract Reasoning
- Samples
- 167
- Released
- 2025
- Scores
- 120
Game2Real-Bench
Game2Real-Bench: Do Toy Games Reveal and Transfer to Real-World Visual Reasoning?
A 1,400-sample benchmark linking toy games and real-world tasks across four visual reasoning capabilities to evaluate VLM performance correspondence and transfer.
How to run
ltb · game2real-bench
$ npm i -g lettokenburn-cli$ ltb bench info game2real-bench
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
13 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| GPT-5.6 Sol | 69.8 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
| Gemini 3.1 Pro | 59.6 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
| Claude Opus 4.8 | 53.7 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
| Qwen3.8 27B | 52.6 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
| Qwen3.5 35B A3B | 45.0 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
| qwen3.5-27B | 44.9 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
| Gemma4 31B | 38.5 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
| qwen3.5-9B | 35.7 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
| Step3-VL-10B | 24.2 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
| Gemma4-12B-it | 23.4 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
| GLM-4.6V-Flash | 22.2 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
| Qwen3-VL-30B-A3B | 21.8 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
| InternVL3.5-30B-A3B | 21.0 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsLoading community runs…
About
Combines 1,000 procedurally generated game samples and 400 curated real-world samples aligned by shared reasoning requirements like spatial operations and constraints.
Evaluated on 25 VLMs using accuracy, revealing strong cross-setting correlation (0.935) and selective one-shot transfer benefits for constraint reasoning.