LetTokenBurnStoreBench: A Live-Commerce Environment for Evaluating and Training Autonomous Operator Agents
A live-commerce environment evaluating autonomous agents on long-horizon operational tasks (30-365 days) managing an online store under uncertainty.
$ npm i -g lettokenburn-cli$ ltb bench info storebench
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| DeepSeek V4 Pro | 0.7 | N/A | Independent | medium | Oct 7, 2026 | arxiv.org |
| Claude Opus 4.8 | 0.7 | N/A | Independent | medium | Oct 7, 2026 | arxiv.org |
| Claude Fable 5 | 0.6 | N/A | Independent | medium | Oct 7, 2026 | arxiv.org |
| Qwen3.8 Max | 0.6 | N/A | Independent | medium | Oct 7, 2026 | arxiv.org |
| Gemini 3.8 Flash | 0.6 | N/A | Independent | medium | Oct 7, 2026 | arxiv.org |
| GPT-5.6 Sol | 0.5 | N/A | Independent | medium | Oct 7, 2026 | arxiv.org |
| Muse Spark 1.2 | 0.4 | N/A | Independent | medium | Oct 7, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
StoreBench places agents in a production-grade Medusa backend to manage an apparel store, handling demand, suppliers, and shocks via 29 tools. It features calibrated pass thresholds against scripted policies and hardened rewards to prevent hacking.
Evaluation covers 11 scenarios over simulated years. Frontier LLMs lag behind human experts and scripted baselines, with the best model passing 49% of tasks compared to a heuristic's 97%, indicating an unsaturated benchmark.