APEX-Harm
Harbor: pendingA benchmark evaluating prompt and script injection safety for long-horizon agents in shared professional workspaces across finance, consulting, and legal tasks.
- Samples
- 180
- Released
- 2026
- Scores
- 10
HeiCo-FOCUS
HeiCo-FOCUS: A Clinically Grounded Dataset for Long-Context Video Understanding
A dataset of 30,000 VQA pairs from 96 hours of surgical video evaluating long-context understanding, object tracking, and temporal reasoning.
How to run
ltb · heico-focus
$ npm i -g lettokenburn-cli$ ltb bench info heico-focus
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
10 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Gemini 3.8 Flash | 51.4 | N/A | Independent | medium | Oct 7, 2026 | arxiv.org |
| GPT-5.6 Sol | 46.8 | N/A | Independent | medium | Oct 7, 2026 | arxiv.org |
| Gemini 3 Flash Preview | 46.2 | N/A | Independent | medium | Oct 7, 2026 | arxiv.org |
| Gemini 3.5 Flash Lite | 44.3 | N/A | Independent | medium | Oct 7, 2026 | arxiv.org |
| Meta Muse Spark 1.3 | 40.5 | N/A | Independent | medium | Oct 7, 2026 | arxiv.org |
| Claude Sonnet 5 | 35.3 | N/A | Independent | medium | Oct 7, 2026 | arxiv.org |
| xAI Grok-4.3 | 33.5 | N/A | Independent | medium | Oct 7, 2026 | arxiv.org |
| DeepSeek V4 Flash Vision | 26.6 | N/A | Independent | medium | Oct 7, 2026 | arxiv.org |
| Xiaomi MiMo-V 2.5 | 23.7 | N/A | Independent | medium | Oct 7, 2026 | arxiv.org |
| Amazon Nova 2 Lite | 18.0 | N/A | Independent | medium | Oct 7, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsLoading community runs…
About
HeiCo-FOCUS comprises 30,000 visual question answering pairs derived from 96 hours of Heidelberg Colorectal surgery videos. It was constructed via a rigorous pipeline involving crowd annotation and 39 surgical experts to ensure clinical relevance.
The benchmark evaluates five capabilities including temporal grounding and procedural understanding. Experiments show models struggle with temporal grounding (19.7% accuracy) while performing better on event understanding (56.5%), indicating the task is far from solved.