InteractionBench
InteractionBench: A Real-Time Interaction Benchmark for Streaming Video Systems
Evaluates streaming video assistants on speech timing and silence compliance across 1,060 interactions in 812 videos, covering queries, events, and updates.
How to run
$ npm i -g lettokenburn-cli$ ltb bench info interactionbench
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
12 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Claude Sonnet 5 agent | 66.1 | agent | Independent | medium | Oct 5, 2026 | arxiv.org |
| GPT-5.6 Sol agent | 58.8 | agent | Independent | medium | Oct 5, 2026 | arxiv.org |
| Claude Sonnet 5 | 44.6 | N/A | Independent | medium | Oct 5, 2026 | arxiv.org |
| Gemini 2.5 Flash | 35.5 | N/A | Independent | medium | Oct 5, 2026 | arxiv.org |
| Qwen3-VL-8B-Instruct | 34.8 | N/A | Independent | medium | Oct 5, 2026 | arxiv.org |
| LLaVA-OV2 | 34.5 | N/A | Independent | medium | Oct 5, 2026 | arxiv.org |
| MiniCPM-o | 29.0 | N/A | Independent | medium | Oct 5, 2026 | arxiv.org |
| Gemma-3n | 27.8 | N/A | Independent | medium | Oct 5, 2026 | arxiv.org |
| Qwen3-VL-4B | 26.2 | N/A | Independent | medium | Oct 5, 2026 | arxiv.org |
| LLaVA-OV | 25.6 | N/A | Independent | medium | Oct 5, 2026 | arxiv.org |
| Qwen2.5-Omni | 18.7 | N/A | Independent | medium | Oct 5, 2026 | arxiv.org |
| Flash-VStream | 14.3 | N/A | Independent | medium | Oct 5, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsAbout
Contains 812 videos with 1,060 interactions across six tasks (Look, Recall, Time, Alert, Track, Commentate), including negative streams and near-miss suites to test silence.
Scores content accuracy, timing accuracy, and silence compliance into an Overall score. Evaluations reveal that timely speech often compromises silence, and offline scores mispredict online behavior.