AgentDoxx
Harbor: pendingAn evaluation suite of 822 synthetic interview transcripts grounded in public info to measure LLM re-identification risks via web search.
- Samples
- 822
- Released
- 2026
- Scores
- 9
DUDA-Bench
DUDA-Bench: Benchmarking LLM Agents on Multimodal Data-Driven Urban Diagnosis
A hierarchical benchmark with 108 tasks (86 atomic, 22 workflow) evaluating LLM agents on multimodal urban diagnosis across 12 cities.
How to run
ltb · duda-bench
$ npm i -g lettokenburn-cli$ ltb bench info duda-bench
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
7 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Qwen3.8 Max | 0.3 | react | Independent | medium | Oct 7, 2026 | arxiv.org |
| Gemini 3.7 Flash | 0.2 | react | Independent | medium | Oct 7, 2026 | arxiv.org |
| Claude Sonnet 5 | 0.1 | react | Independent | medium | Oct 7, 2026 | arxiv.org |
| Kimi K3 | 0.1 | react | Independent | medium | Oct 7, 2026 | arxiv.org |
| Qwen3.8 27B | 0.1 | react | Independent | medium | Oct 7, 2026 | arxiv.org |
| GPT-5.6 Sol | 0.1 | react | Independent | medium | Oct 7, 2026 | arxiv.org |
| GPT-5.6-terra | 0.1 | react | Independent | medium | Oct 7, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsLoading community runs…
About
DUDA-Bench formalizes data-driven urban diagnosis into atomic and workflow tasks using multimodal data from 12 cities. It features an interactive agentic environment with domain tools and code execution.
Evaluation combines outcome assessment with stage-aligned checkpoints to measure diagnostic success and intermediate progress. Results reveal a significant gap between atomic competence and end-to-end workflow coordination.