OmniReasoningBench
OmniReasoning: Pushing the Limits of Audio-Visual Joint Reasoning
A benchmark comprising 1,150 multiple-choice and open-ended questions requiring indispensable audio-visual joint reasoning across video tasks.
How to run
$ npm i -g lettokenburn-cli$ ltb bench info omnireasoningbench
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
1 sourced entry| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Qwen3-Omni-30B-A3B-Thinking | 87.5 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsAbout
Content & Construction: The benchmark consists of two settings: 'reasoning over video' (connecting observations within a video) and 'reasoning beyond video' (applying derived information to new scenarios). Questions are constructed so that neither audio nor visual inputs alone suffice, necessitating cross-modal integration. A data engine named OmniQA automatically generates these evidence-grounded QA pairs along with timestamped clue chains.
Scoring & Key Findings: The primary metric is accuracy on multiple-choice and open-ended questions. Models achieving high scores must effectively integrate complementary audio-visual clues; ablation studies show that models relying solely on single modalities fail significantly, validating the benchmark's ability to test genuine joint reasoning capabilities.