AgentHorizon
Harbor: pendingA benchmark of 1,373 computer-use tasks with paired instructions to evaluate agentic judges on long-horizon trajectory success verification across three OSs.
- Samples
- 1,373
- Released
- 2026
- Scores
- 10
HelpMeOut
HelpMeOut: A Neuropsychology-Grounded Benchmark for Error-Aware Monitoring of Household Task Execution
An embodied AI benchmark with 20,000 paired activity episodes for monitoring structured execution errors in household tasks.
How to run
ltb · helpmeout
$ npm i -g lettokenburn-cli$ ltb bench info helpmeout
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
12 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Kimi K2.6 | 0.7 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
| Claude Sonnet 4.6 | 0.7 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
| DeepSeek V4 Pro | 0.7 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
| GPT-5.5 | 0.6 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
| Grok-4.3 | 0.6 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
| GPT-5.4-mini | 0.6 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
| GLM-5.1 | 0.6 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
| Gemini 3.1 Pro | 0.6 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
| Gemini 3.1 Flash Lite | 0.6 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
| MiniMax-M2.7 | 0.5 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
| Qwen3.6 35B A3B | 0.5 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
| DeepSeek V4 Flash | 0.5 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsLoading community runs…
About
HelpMeOut provides 20,000 paired correct and errorful household task episodes, grounded in neuropsychological error categories like omissions and substitutions. Agents monitor streaming symbolic action trajectories to diagnose errors.
Evaluation uses four metrics measuring discrimination, detection latency, and type diagnosis. Baselines including LLMs and symbolic methods perform poorly, often missing errors or failing to localize them timely.