AgenticBBO-Bench
Harbor: pendingA cross-domain benchmark evaluating LLM agents on five black-box optimization tasks (synthetic, HPO, DB tuning, chip design, molecular) under a unified finite-budget protocol.
- Samples
- N/A
- Released
- 2026
- Scores
- 7
GeoSpatialBench
Mind the Spatial Gap: A Benchmark for Geospatial Reasoning and Constrained Spatial Editing
Evaluates MLLMs on 28 spatial relations via QA and constrained editing using GeoJSON and map images from 6 cities (8,400 instances).
How to run
ltb · geospatialbench
$ npm i -g lettokenburn-cli$ ltb bench info geospatialbench
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
9 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| GPT-6 Astra | 96.4 | N/A | Independent | medium | Oct 9, 2026 | arxiv.org |
| Kimi K3 | 89.5 | N/A | Independent | medium | Oct 9, 2026 | arxiv.org |
| GPT-5.6 Sol | 78.1 | N/A | Independent | medium | Oct 9, 2026 | arxiv.org |
| MiMo V2.6 Pro | 63.0 | N/A | Independent | medium | Oct 9, 2026 | arxiv.org |
| Qwen3.8 27B | 52.5 | N/A | Independent | medium | Oct 9, 2026 | arxiv.org |
| DeepSeek V4.1 Flash | 49.5 | N/A | Independent | medium | Oct 9, 2026 | arxiv.org |
| qwen3.5-9B | 46.5 | N/A | Independent | medium | Oct 9, 2026 | arxiv.org |
| Qwen3-VL-8B-Instruct | 37.2 | N/A | Independent | medium | Oct 9, 2026 | arxiv.org |
| InternVL3-8B | 29.3 | N/A | Independent | medium | Oct 9, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsLoading community runs…
About
GeoSpatialBench comprises 8,400 instances derived from real OpenStreetMap data across six cities, covering 28 topological, directional, metric, and ordering relations in GeoJSON and map image modalities. It combines question-answering with constrained spatial editing to test both recognition and application of spatial constraints.
Scoring uses GIS-derived ground truths and deterministic checks for edited geometry. Results show GeoJSON outperforms map images, with significant performance drops on quantitative metric relations and editing tasks, revealing a gap between passive recognition and active manipulation.