MS^2
Harbor: pendingMS^2 (Multi-Document Summarization of Medical Studies) is a dataset of over 470k documents and 20k summaries derived from the scientific literature.
Auto71%Knowledge & Reasoning
- Samples
- N/A
- Released
- 2021
- Scores
- 0
MS-Eval
MS-Eval: Benchmarking Sub-Agent Delegation and Orchestration in MCP-Intensive Working Scenarios
MS-Eval evaluates sub-agent delegation and orchestration quality in MCP-intensive working scenarios using a curated benchmark, controlled harness, and the MSQ index.
How to run
ltb · ms-eval
$ npm i -g lettokenburn-cli$ ltb bench info ms-eval
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
8 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Kimi K3 | 52.6 | mad-sae (spec) | Independent | medium | Oct 16, 2026 | arxiv.org |
| GLM-5.3 | 48.0 | mad-sae (spec) | Independent | medium | Oct 16, 2026 | arxiv.org |
| Qwen3.8 Max | 47.6 | mad-sae (spec) | Independent | medium | Oct 16, 2026 | arxiv.org |
| GLM-5.3 Flash | 45.8 | mad-sae (spec) | Independent | medium | Oct 16, 2026 | arxiv.org |
| DeepSeek V4 Flash 0731 | 43.8 | mad-sae (spec) | Independent | medium | Oct 16, 2026 | arxiv.org |
| Claude Sonnet 5 | 42.3 | mad-sae (spec) | Independent | medium | Oct 16, 2026 | arxiv.org |
| MiniMax-M3 | 41.0 | mad-sae (spec) | Independent | medium | Oct 16, 2026 | arxiv.org |
| GPT-5.6 Sol | 38.0 | mad-sae (spec) | Independent | medium | Oct 16, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsLoading community runs…
About
MS-Eval comprises an MCP-intensive benchmark covering eight working scenarios with tasks annotated for optimal delegation priors. It uses a controlled execution harness to standardize tool accessibility for Main-Agent Delegation and Sub-Agent Execution (MAD-SAE) paradigms.
Scoring relies on tailored metrics for delegation appropriateness and orchestration utilities, aggregated into the MAD-SAE Quality (MSQ) index. Results indicate sub-agent specialization is the best practice, with open-source models often outperforming closed-source ones in efficiency and cost.