LetTokenBurnThe EgoGears Benchmark
A single- and multi-video benchmark for embodied AI containing 567 single-video and 1487 multi-video questions derived from 126 egocentric recordings across 39 outdoor routes.
$ npm i -g lettokenburn-cli$ ltb bench info egogears
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Gemini 3.8 Flash | 79.8 | N/A | Independent | medium | Sep 29, 2026 | arxiv.org |
| Gemini 3.1 Pro | 67.7 | N/A | Independent | medium | Sep 29, 2026 | arxiv.org |
| google/gemma-4-31B-it | 56.4 | N/A | Independent | medium | Sep 29, 2026 | arxiv.org |
| Gemini 2.5 Pro | 53.7 | N/A | Independent | medium | Sep 29, 2026 | arxiv.org |
| Qwen3-VL-235B-A22B | 49.6 | N/A | Independent | medium | Sep 29, 2026 | arxiv.org |
| qwen3.5-27B | 45.1 | N/A | Independent | medium | Sep 29, 2026 | arxiv.org |
| GLM-4.6V | 37.7 | N/A | Independent | medium | Sep 29, 2026 | arxiv.org |
| Qwen3.5 35B A3B | 37.4 | N/A | Independent | medium | Sep 29, 2026 | arxiv.org |
| Qwen3.5 122B A10B | 36.8 | N/A | Independent | medium | Sep 29, 2026 | arxiv.org |
| Qwen2.5-VL-72B-Instruct | 28.5 | N/A | Independent | medium | Sep 29, 2026 | arxiv.org |
| Qwen3-VL-30B-A3B | 25.8 | N/A | Independent | medium | Sep 29, 2026 | arxiv.org |
| qwen3.5-9B | 21.1 | N/A | Independent | medium | Sep 29, 2026 | arxiv.org |
| Kimi-VL-A3B-2506 | 19.3 | N/A | Independent | medium | Sep 29, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Content & Construction: EgoGears evaluates whether local video understanding transfers across encounters in outdoor settings. It uses human-collected egocentric videos of walking, jogging, and running along repeated traversals to test evidence binding, correspondence, and spatiotemporal composition.
Scoring & Findings: The benchmark assesses 29 single-video and 31 multi-video MLLM configurations across six model families. Results show a significant performance drop (mean decrease of 22.5 percentage points) on multi-video questions compared to single-video ones, highlighting bottlenecks in observation-evidence binding and route-state tracking.