MindBench
Harbor: pendingMultimodal Large Language Models (MLLM) have made significant progress in the field of document analysis.
- Samples
- N/A
- Released
- 2024
- Scores
- 0
Savanna
Agentic Image Generation in the Wild
A benchmark of 500 prompts across 16 subcategories evaluating agentic image generation on recent entities with temporal novelty, semantic crowding, and spatial entanglement.
How to run
ltb · savanna
$ npm i -g lettokenburn-cli$ ltb bench info savanna
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
12 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Ranger | 58.9 | ranger qwen3-vl-8b | Independent | medium | Oct 16, 2026 | arxiv.org |
| Gen-Searcher | 37.9 | gen-searcher qwen3-vl-8b | Independent | medium | Oct 16, 2026 | arxiv.org |
| SCOPE | 37.3 | scope ∗ gpt-6-astra | Independent | medium | Oct 16, 2026 | arxiv.org |
| GenClaw | 36.5 | genclaw ∗ claude-opus-5 | Independent | medium | Oct 16, 2026 | arxiv.org |
| Nano Banana 2 | 34.9 | N/A | Independent | medium | Oct 16, 2026 | arxiv.org |
| GenClaw | 34.2 | genclaw claude-opus-4.6 | Independent | medium | Oct 16, 2026 | arxiv.org |
| GenEvolve | 34.2 | genevolve qwen3-vl-8b | Independent | medium | Oct 16, 2026 | arxiv.org |
| SCOPE | 30.2 | scope gpt-5.4 | Independent | medium | Oct 16, 2026 | arxiv.org |
| Mind-Brush | 29.5 | mind-brush gpt-5.1 | Independent | medium | Oct 16, 2026 | arxiv.org |
| GPT-Image-2 | 26.4 | N/A | Independent | medium | Oct 16, 2026 | arxiv.org |
| Nano Banana Pro | 26.1 | N/A | Independent | medium | Oct 16, 2026 | arxiv.org |
| Unify-Agent BAGEL-14B | 22.5 | N/A | Independent | medium | Oct 16, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsLoading community runs…
About
Savanna contains 500 natural-language prompts synthesized from recent news headlines, filtered for high retrieval confusion. It targets challenges like temporal novelty and semantic crowding in web-based reference retrieval.
Scoring uses separate search and generation scores to measure reference correctness and image fidelity. Frontier models achieve low search scores (~51.5%), indicating the benchmark remains unsaturated and difficult for current agents.