LetTokenBurnCUA-SpeedRun: Standardized Benchmarking of the Speed of Computer-Use Agents
A standardized platform evaluating computer-use agents (CUAs) on speed and cost efficiency using uniform VM infrastructure and representative task subsets across four benchmarks.
$ npm i -g lettokenburn-cli$ ltb bench info cua-speedrun
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Gemini 3 Flash Preview | 79.6 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| Kimi K3 | 79.6 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| GPT-5.6 Luna | 81.6 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| GPT-5.6 Sol | 81.6 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| Gemini 3.7 Flash | 89.6 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| Claude Opus 5 | 91.6 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| Gemini 3.8 Flash | 91.6 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| GPT-6 Astra | 91.6 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Content & Construction: cua-speedrun standardizes CUA evaluation by decoupling agent, benchmark, environment, and loop components. It utilizes a serverless cloud-based provider with identical virtual machine setups to ensure consistent wall-time measurements. The platform employs an energy minimization framework to select minimal representative task subsets that preserve statistical power while reducing evaluation time significantly (e.g., 84.9% reduction on OSWorld).
Scoring & Key Findings: Metrics include total task time, agent execution time vs environment operation time, and model call costs. The benchmark reveals unintuitive insights: increasing reasoning effort can reduce completion time by avoiding repeated failures, while faster I/O environments can sometimes slow agents down due to lack of training on varying speeds.