UndoBench
UndoBench: Separating Task Competence from Recovery Capability in Tool-Using AI Agents
A benchmark with 36 base workflows and 36 fault scenarios across 8 enterprise domains, evaluating agent recovery capabilities via counterfactual paired trials.
How to run
$ npm i -g lettokenburn-cli$ ltb bench info undobench
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
2 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| GLM-5.2 MaaS | 21.9 | N/A | Independent | medium | Oct 4, 2026 | arxiv.org |
| Gemini 3.8 Flash | 21.1 | N/A | Independent | medium | Oct 4, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsAbout
UndoBench evaluates tool-using AI agents across 36 workflows and 36 fault scenarios in 8 enterprise domains. It uses counterfactual paired trials to decouple nominal task competence from recovery capability under distributed system faults like lost acknowledgments.
Scoring relies on Conditional Recovery Success Rate (CRSR) and wire-level effect monitors. Results show nominal competence at 83.54% but recovery success drops to 46.72%, with naive retries causing duplicate effects in over half of trials.