STOU
Safety for Whom? Benchmarking Understanding Beyond Refusal in Egocentric Multimodal Assistants
STOU is a benchmark of 2,902 egocentric video-audio and video-text queries evaluating safety understanding and refusal behavior across harm targets (Self, Others, Society) in multimodal assistants.
How to run
$ npm i -g lettokenburn-cli$ ltb bench info stou
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
9 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Muse Spark 1.2 | 84.4 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
| Gemini 3 Flash Preview | 72.1 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
| MiniCPM-o 4.5 | 53.3 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
| google/gemma-4-E4B-it | 50.6 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
| MiniCPM-o-2-6 | 44.9 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
| Qwen2.5-Omni-7B | 43.3 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
| Qwen2.5-Omni-3B | 33.7 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
| Baichuan 1.5 | 27.5 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
| VITA-1.5 | 26.0 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsAbout
STOU comprises 2,902 multimodal instances pairing egocentric videos with audio/text queries, categorized by harm target (Self, The Others, Others, Unspecified) and type (Physical, Psychological, etc.). It evaluates whether models recognize intent, harm presence, and consequences beyond simple refusal.
The benchmark decouples safety understanding from refusal behavior, scoring model performance on recognizing user intent and downstream risks. Results reveal that refusal rates vary by harm target and models often comply despite recognizing harm, highlighting gaps in current safety alignment.