SkillsBench
89-task SkillsBench benchmark
An evaluation of agent skill retrieval methods across an 89-task benchmark using a marketplace-scale pool to measure pass rates and cost efficiency.
How to run
$ npm i -g lettokenburn-cli$ ltb bench info skillsbench
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
3 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Qwen3-Reranker-0.6B | 47.8 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| Qwen3-Reranker-8B | 47.8 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| Bge Reranker V2 M3 | 40.9 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsAbout
Content & Construction: The benchmark consists of 89 tasks designed to evaluate the ability of agents to retrieve relevant procedural skills from large pools (e.g., 34,000+ skills). It compares deterministic retrieval methods against LLM-mediated loops. Scoring & Findings: Performance is measured by task pass rate and monetary cost per trial. Key findings show that standard IR recipes like BM25 or SkillSeek achieve parity with expensive LLM-mediated refinement in most settings while significantly reducing costs.