LetTokenBurnThe ImplicitQA dataset was introduced in the paper ImplicitQA: Going beyond frames towards Implicit Video Reasoning.
$ npm i -g lettokenburn-cli$ ltb bench info implicitqa
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| GPT-4.1 | 54.3 | N/A | Independent | medium | N/A | paperswithcode.com |
| Qwen2-VL-7B | 44.9 | N/A | Independent | medium | Sep 18, 2024 | arxiv.org |
| LLaVA-OneVision-7B | 43.4 | N/A | Independent | medium | Aug 6, 2024 | arxiv.org |
| Qwen2.5-VL-7B | 42.8 | N/A | Independent | medium | Feb 19, 2025 | arxiv.org |
| LLaVA-Video 7B | 42.1 | N/A | Independent | medium | Oct 3, 2024 | arxiv.org |
| LLaVA-NeXT-Video-7B | 33.9 | N/A | Independent | medium | N/A | paperswithcode.com |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
The ImplicitQA dataset was introduced in the paper ImplicitQA: Going beyond frames towards Implicit Video Reasoning.
Project page: https://swetha5.github.io/ImplicitQA/
ImplicitQA is a novel benchmark specifically designed to test models on implicit reasoning in Video Question Answering (VideoQA). Unlike existing VideoQA benchmarks that primarily focus on questions answerable through explicit visual content (actions, objects, events directly observable within individual frames or short clips), ImplicitQA addresses the need for models to infer motives, causality, and relationships <u>across discontinuous frames</u>. This mirrors human-like understanding of creative and cinematic videos, which often employ storytelling techniques that deliberately omit certain depictions.
The dataset comprises 1,000 meticulously annotated QA pairs derived from over 320 high-quality creative video clips. These QA pairs are systematically categorized into key reasoning dimensions, including:
The annotations are deliberately challenging, crafted to ensure high quality and to highlight the difficulty of implicit reasoning for current VideoQA models.