AgenticBBO-Bench
Harbor: pendingA cross-domain benchmark evaluating LLM agents on five black-box optimization tasks (synthetic, HPO, DB tuning, chip design, molecular) under a unified finite-budget protocol.
- Samples
- N/A
- Released
- 2026
- Scores
- 7
DyadMem
DyadMem: A Long-Term Memory Benchmark of How Agents Work with Users
A dual-domain benchmark evaluating long-term agent memory via 3,065 episodes and 61,210 QA instances across Capture, Update, Recall, and QA stages.
How to run
ltb · dyadmem
$ npm i -g lettokenburn-cli$ ltb bench info dyadmem
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
12 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Kimi K3 | 74.9 | N/A | Independent | medium | Oct 2, 2026 | arxiv.org |
| GLM-5.2 | 70.6 | N/A | Independent | medium | Oct 2, 2026 | arxiv.org |
| GPT-5.6 Sol | 70.3 | N/A | Independent | medium | Oct 2, 2026 | arxiv.org |
| Doubao-Seed-2.1-Pro | 68.0 | N/A | Independent | medium | Oct 2, 2026 | arxiv.org |
| DeepSeek V4 Pro | 66.1 | N/A | Independent | medium | Oct 2, 2026 | arxiv.org |
| MiniMax M2.5 | 64.5 | N/A | Independent | medium | Oct 2, 2026 | arxiv.org |
| GPT-oss 120B | 61.2 | N/A | Independent | medium | Oct 2, 2026 | arxiv.org |
| Qwen3.5 397B A17B | 54.7 | N/A | Independent | medium | Oct 2, 2026 | arxiv.org |
| Qwen3.8 27B | 54.5 | N/A | Independent | medium | Oct 2, 2026 | arxiv.org |
| GPT-4o-mini | 47.9 | N/A | Independent | medium | Oct 2, 2026 | arxiv.org |
| Qwen3 8B | 47.4 | N/A | Independent | medium | Oct 2, 2026 | arxiv.org |
| qwen3.5-27B | 45.9 | N/A | Independent | medium | Oct 2, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsLoading community runs…
About
DyadMem annotates user-side memory and User-conditioned Relational Agent Memory (URAM) across 3,065 multi-session trajectories. It features session-level Capture/Update gold annotations and query-level Recall support.
Scoring uses Gold-Memory and Full-Pipeline QA settings to localize failures. Results show a significant performance drop in Full-Pipeline QA, revealing issues like low Capture recall and unsafe deletions in frontier LLMs.