AgentHorizon
Harbor: pendingA benchmark of 1,373 computer-use tasks with paired instructions to evaluate agentic judges on long-horizon trajectory success verification across three OSs.
- Samples
- 1,373
- Released
- 2026
- Scores
- 10
ReFract
Role-aware Evaluation FRamework for ACtion Trajectories
A benchmark of 150 expert-validated entries measuring Perspective Awareness in LLM agents using Text World Models grounded in industrial maintenance scenarios.
How to run
ltb · refract
$ npm i -g lettokenburn-cli$ ltb bench info refract
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
9 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| GPT-5.6 Luna | 0.6 | function calling | Independent | medium | Oct 2, 2026 | arxiv.org |
| GPT-5.5 | 0.6 | function calling | Independent | medium | Oct 2, 2026 | arxiv.org |
| Qwen3-32B | 0.5 | function calling | Independent | medium | Oct 2, 2026 | arxiv.org |
| Qwen3-80B-A3B | 0.4 | function calling | Independent | medium | Oct 2, 2026 | arxiv.org |
| Qwen3 235B A22B | 0.4 | function calling | Independent | medium | Oct 2, 2026 | arxiv.org |
| google/gemma-4-31B-it | 0.4 | function calling | Independent | medium | Oct 2, 2026 | arxiv.org |
| gpt-oss-20b | 0.3 | function calling | Independent | medium | Oct 2, 2026 | arxiv.org |
| GPT-oss 120B | 0.3 | function calling | Independent | medium | Oct 2, 2026 | arxiv.org |
| google/gemma-4-26B-A4B-it | 0.2 | function calling | Independent | medium | Oct 2, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsLoading community runs…
About
ReFract evaluates if agents adapt actions to user roles (Perspective-Taking and Perspective-Routing) using executable Python-based Text World Models. It comprises 150 entries derived from anonymized domain support conversations.
Scoring assesses trajectory compliance with role boundaries. State-of-the-art models solve at most 69% of tasks, with over 50% of trajectories containing perspective-violating actions, highlighting significant unsolved challenges.