Auto71%Harbor: pendingMultimodalKnowledge & Reasoning

TVQA

The TVQA dataset is a large-scale video dataset for video question answering.

How to run

ltb · tvqa
$ npm i -g lettokenburn-cli
$ ltb bench info tvqa

External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.

Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs

Scores

1 sourced entry
HarnessSourceConfidenceLink
LLaMA-VQA82.2N/AIndependentmediumOct 24, 2023arxiv.org

Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.

Reproduced by the community

All runs
Loading community runs…

About

The TVQA dataset is a large-scale video dataset for video question answering. It is based on 6 popular TV shows (Friends, The Big Bang Theory, How I Met Your Mother, House M.D., Grey's Anatomy, Castle). It includes 152,545 QA pairs from 21,793 TV show clips. The QA pairs are split into the ratio of 8:1:1 for training, validation, and test sets. The TVQA dataset provides the sequence of video frames extracted at 3 FPS, the corresponding subtitles with the video clips, and the query consisting of a question and four answer candidates. Among the four answer candidates, there is only one correct answer.

Source: Two-stream Spatiotemporal Feature for Video QA Task Image Source: https://arxiv.org/abs/1809.01696