AgBench
Agentic AI Benchmarks for Personal AI Devices
A benchmark suite evaluating agentic workloads on personal devices across local-only, hybrid, and cloud architectures using over 162 million data points to measure task success, latency, cost, and privacy.
How to run
$ npm i -g lettokenburn-cli$ ltb bench info agbench
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
1 sourced entry| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| DeepSeek V4 Flash | 0.0 | <null> | Independent | medium | Sep 29, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsAbout
Content & Construction: AgBench is a comprehensive evaluation framework comprising 60 tasks spanning eight categories. It systematically characterizes agent execution on personal AI devices (e.g., RTX 5090 PCs) versus cloud-hosted models under varying concurrency levels.
Scoring & Findings: The benchmark measures task success, completion time, API costs, and data exposure metrics. Key findings indicate that while local-only execution eliminates cloud costs and privacy risks, it often suffers lower task success rates compared to hybrid or cloud-only approaches as concurrency increases.