ARC-Challenge
Harbor: pendingARC-Challenge — AI evaluation benchmark listed by opencompass.
Incomplete14%Knowledge & Reasoning
- Samples
- N/A
- Released
- N/A
- Scores
- 26
commonsense_qa
A Question Answering Challenge Targeting Commonsense Knowledge
CommonsenseQA is a multiple-choice question answering dataset that requires different types of commonsense knowledge to predict the correct answers.
How to run
ltb · commonsense-qa
$ npm i -g lettokenburn-cli$ ltb bench pull commonsense-qa --n 50$ ltb bench run ./commonsense-qa -a claude-code -m anthropic/claude-opus-4-8$ ltb bench push
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
5 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| OPT 66B | 66.4 | N/A | Independent | medium | Mar 30, 2023 | arxiv.org |
| BLOOM 176B | 64.2 | N/A | Independent | medium | Mar 30, 2023 | arxiv.org |
| GPT-NeoX 20B | 60.4 | N/A | Independent | medium | Mar 30, 2023 | arxiv.org |
| UL2 20B | 51.4 | N/A | Independent | medium | May 10, 2022 | arxiv.org |
| UL2 20B | 34.2 | N/A | Independent | medium | May 10, 2022 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsLoading community runs…
About
CommonsenseQA is a multiple-choice question answering dataset that requires different types of commonsense knowledge to predict the correct answers. It contains 12,102 questions with one correct answer and four distractor answers.
Available in EleutherAI lm-evaluation-harness as task family commonsense_qa (1 tasks).