AgenticBBO-Bench
Harbor: pendingA cross-domain benchmark evaluating LLM agents on five black-box optimization tasks (synthetic, HPO, DB tuning, chip design, molecular) under a unified finite-budget protocol.
- Samples
- N/A
- Released
- 2026
- Scores
- 7
BioStudyBench
BioStudyBench: Evaluating Agents on Post-Cutoff Biomedical Studies
A benchmark of 25 long-horizon agentic tasks requiring AI agents to reproduce findings from post-cutoff biomedical studies using public data and restricted tools.
How to run
ltb · biostudybench
$ npm i -g lettokenburn-cli$ ltb bench info biostudybench
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
8 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Gemini 3.8 Flash | 94.7 | N/A | Independent | medium | Oct 6, 2026 | arxiv.org |
| GPT-6 Astra | 94.7 | N/A | Independent | medium | Oct 6, 2026 | arxiv.org |
| Claude Opus 5.5 | 93.3 | N/A | Independent | medium | Oct 6, 2026 | arxiv.org |
| GLM-5.3 | 81.3 | N/A | Independent | medium | Oct 6, 2026 | arxiv.org |
| DeepSeek V4.1 Flash | 76.0 | N/A | Independent | medium | Oct 6, 2026 | arxiv.org |
| GPT-6 Luna | 74.7 | N/A | Independent | medium | Oct 6, 2026 | arxiv.org |
| MiMo V2.6 Pro | 66.7 | N/A | Independent | medium | Oct 6, 2026 | arxiv.org |
| Qwen3.8 27B | 56.0 | N/A | Independent | medium | Oct 6, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsLoading community runs…
About
BioStudyBench comprises 25 tasks derived from biomedical papers published after model knowledge cutoffs. Agents must locate public data, perform analysis, and report findings without direct access to the source paper's conclusions.
Scoring uses a rule-based comparison against reference confidence intervals or significance directions. Results show data access improves pass rates by ~47%, with top closed-weight models achieving 94.7% accuracy.