LetTokenBurn
Sign in
Auto86%Harbor: pendingMultimodalKnowledge & Reasoning

TRAJECTORY-144

A multimodal benchmark of 144 clinician-informed multiple-choice questions grounded in 264 hours of ICU video and clinical records to evaluate longitudinal patient trajectory reasoning.

PaperRepo

How to run

ltb · trajectory-144
$ npm i -g lettokenburn-cli
$ ltb bench info trajectory-144

External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.

Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs

Scores

9 sourced entries
HarnessSourceConfidenceLink
Gemini-3.5-flash76.9tracerIndependentmediumOct 15, 2026arxiv.org
Gemini 3 Flash Preview70.8tracerIndependentmediumOct 15, 2026arxiv.org
qwen3.5-9B61.6tracerIndependentmediumOct 15, 2026arxiv.org
Gemini 3.5 Flash Lite61.1tracerIndependentmediumOct 15, 2026arxiv.org
Gemini 3.1 Flash Lite61.0tracerIndependentmediumOct 15, 2026arxiv.org
Gemini 2.5 Flash60.3tracerIndependentmediumOct 15, 2026arxiv.org
gpt-oss-20b57.4tracerIndependentmediumOct 15, 2026arxiv.org
Gemini 2.5 Flash Lite49.2tracerIndependentmediumOct 15, 2026arxiv.org
Qwen3-VL-8B-Instruct42.1tracerIndependentmediumOct 15, 2026arxiv.org

Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.

Reproduced by the community

All runs
Loading community runs…

About

TRAJECTORY-144 consists of 144 multiple-choice questions derived from two ICU patient stays, integrating continuous video and electronic health records. It evaluates an agent's ability to reason over long-term behavioral patterns and clinical context.

Scoring uses QA accuracy across five gold-answer placements. TRACER achieves up to 76.9% accuracy with proprietary LLMs, significantly outperforming baselines which score below 50%, demonstrating effective longitudinal reasoning without training on protected health data.