Terminal-Bench 2.0
Harbor nativeTerminal-Bench 2.0: 命令行/DevOps操作
Verified86%Coding
- Samples
- 89
- Released
- 2025
- Scores
- 12
AM-Bench
AM-Bench: Understanding Agents under Task Underspecification across Models and Harnesses
A benchmark of 48 tasks with 7 variants each (336 total) evaluating agent performance on ambiguous or missing goals, context, and constraints across models and harnesses.
How to run
ltb · am-bench
$ npm i -g lettokenburn-cli$ ltb bench info am-bench
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
6 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| DS-Flash | 83.3 | codex | Independent | medium | Oct 13, 2026 | arxiv.org |
| DS-Flash | 81.3 | nanobot | Independent | medium | Oct 13, 2026 | arxiv.org |
| GLM-5.3 | 61.1 | nullclaw | Independent | medium | Oct 13, 2026 | arxiv.org |
| DS-Pro | 46.5 | nullclaw | Independent | medium | Oct 13, 2026 | arxiv.org |
| GPT-Sol | 38.2 | nullclaw | Independent | medium | Oct 13, 2026 | arxiv.org |
| GPT-Luna | 36.1 | nullclaw | Independent | medium | Oct 13, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsLoading community runs…
About
AM-Bench adapts 48 tasks from Terminal-Bench 2.0 and Harness-Bench, creating six underspecified variants per task alongside a fully specified reference. It evaluates five models and four harnesses in executable tool environments.
Scoring separates Completion and Compliance, with Success requiring both. Key findings show missing information lowers success more than ambiguity, and model-harness pairings significantly impact rankings under underspecification.