ARC-AGI-2
Harbor nativeARC-AGI-2: 更难的抽象推理
Verified100%Abstract Reasoning
- Samples
- 167
- Released
- 2025
- Scores
- 118
Learn2Play Bench
Learn2Play Bench: How Well Do LLM Agents Learn from Experience in Unfamiliar Environments?
A benchmark of newly designed text-based games with hidden rules to evaluate how well LLM agents learn from experience and adapt to unfamiliar environments through interaction.
How to run
ltb · learn2play-bench
$ npm i -g lettokenburn-cli$ ltb bench info learn2play-bench
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
13 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Claude Opus 5 | 73.0 | claude code | Independent | medium | Oct 6, 2026 | arxiv.org |
| Kimi K3 | 59.8 | evotest | Independent | medium | Oct 6, 2026 | arxiv.org |
| GPT-5.6 Sol | 59.5 | codex | Independent | medium | Oct 6, 2026 | arxiv.org |
| Kimi K3 | 56.6 | reflexion | Independent | medium | Oct 6, 2026 | arxiv.org |
| Qwen3.8 Max | 56.4 | opencode | Independent | medium | Oct 6, 2026 | arxiv.org |
| Kimi K3 | 56.2 | opencode | Independent | medium | Oct 6, 2026 | arxiv.org |
| Claude Opus 5 | 54.7 | evotest | Independent | medium | Oct 6, 2026 | arxiv.org |
| Claude Opus 5 | 50.3 | opencode | Independent | medium | Oct 6, 2026 | arxiv.org |
| Kimi K3 | 47.7 | awm | Independent | medium | Oct 6, 2026 | arxiv.org |
| Kimi K3 | 47.0 | reasoningbank | Independent | medium | Oct 6, 2026 | arxiv.org |
| Claude Opus 5 | 46.2 | memory | Independent | medium | Oct 6, 2026 | arxiv.org |
| GPT-5.6 Sol | 46.1 | opencode | Independent | medium | Oct 6, 2026 | arxiv.org |
| Kimi K3 | 46.0 | memory | Independent | medium | Oct 6, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsLoading community runs…
About
Learn2Play Bench features novel text-based games with counterintuitive hidden rules, requiring agents to discover mechanics through interaction rather than pretrained knowledge. It includes deterministic feedback, automatic scoring, and re-shuffled instances to test knowledge transfer.
Scoring is automated based on game performance. Key findings indicate that retaining raw interaction history aids learning more than summarization, humans outperform top agents, and agent harnesses significantly impact both performance and inference cost.