OK-VQA
Outside Knowledge Visual Question Answering
Outside Knowledge Visual Question Answering (OK-VQA) includes more than 14,000 questions that require external knowledge to answer.
How to run
$ npm i -g lettokenburn-cli$ ltb bench info ok-vqa
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
1 sourced entry| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| PaLM-E-562B | 66.1 | N/A | Independent | medium | Mar 6, 2023 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsAbout
Outside Knowledge Visual Question Answering (OK-VQA) includes more than 14,000 questions that require external knowledge to answer.
Source: OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledge Image Source: https://okvqa.allenai.org/