ANLI
Harbor adapterAdversarial NLI (ANLI) is a dataset collected via an iterative, adversarial human-and-model-in-the-loop procedure.
Verified100%Knowledge & Reasoning
- Samples
- 169,265
- Released
- 2019
- Scores
- 12
PIQA
Physical Commonsense Reasoning Test
Physical Interaction: Question Answering (PIQA) is a physical commonsense reasoning and a corresponding benchmark dataset.
How to run
ltb · piqa
$ npm i -g lettokenburn-cli$ ltb bench pull piqa --n 50$ ltb bench run ./piqa -a claude-code -m anthropic/claude-opus-4-8$ ltb bench push
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
112 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| UNICORN | 90.1 | N/A | Self-reported | medium | N/A | ojs.aaai.org |
| GPT-4o-mini | 88.7 | N/A | Independent | medium | N/A | arxiv.org |
| Phi-3.5-MoE-instruct | 88.6 | N/A | Self-reported | medium | N/A | arxiv.org |
| gemini-1.5-flash-002 | 87.5 | N/A | Independent | medium | N/A | arxiv.org |
| Llama-3.1-405B-Instruct | 85.9 | N/A | Independent | medium | N/A | arxiv.org |
| PaLM 2-L | 85.0 | N/A | Independent | medium | May 17, 2023 | arxiv.org |
| Falcon-180B | 84.9 | N/A | Self-reported | medium | N/A | arxiv.org |
| DeepSeek-V3 | 84.7 | N/A | Self-reported | medium | N/A | arxiv.org |
| Inflection-1 | 84.2 | N/A | Independent | medium | N/A | arxiv.org |
| DeepSeek-V2 | 83.9 | N/A | Self-reported | medium | N/A | arxiv.org |
| Gemma 2 9B IT | 83.7 | N/A | Independent | medium | N/A | arxiv.org |
| Mixtral 8x7B | 83.6 | N/A | Independent | medium | Jan 8, 2024 | arxiv.org |
| Mixtral-8x7B-Instruct-v0.1 | 83.6 | N/A | Self-reported | medium | N/A | arxiv.org |
| Mistral-Nemo-Instruct-2407 | 83.5 | N/A | Independent | medium | N/A | arxiv.org |
| StableBeluga2 | 83.3 | N/A | Independent | medium | N/A | arxiv.org |
| PaLM 2-M | 83.2 | N/A | Independent | medium | May 17, 2023 | arxiv.org |
| Megatron-Turing NLG 530B | 83.2 | N/A | Self-reported | medium | N/A | arxiv.org |
| Mistral 7B | 83.0 | N/A | Independent | medium | Oct 10, 2023 | arxiv.org |
| Falcon 40B | 83.0 | N/A | Self-reported | medium | N/A | arxiv.org |
| Mistral-7B-v0.1 | 83.0 | N/A | Independent | medium | N/A | arxiv.org |
| Llama 2 70B | 82.8 | N/A | Independent | medium | Jul 18, 2023 | arxiv.org |
| LLaMA 65B | 82.8 | N/A | Independent | medium | Feb 27, 2023 | arxiv.org |
| Llama 2 70B | 82.8 | N/A | Self-reported | medium | N/A | arxiv.org |
| Llama 2 70B | 82.8 | N/A | Independent | medium | N/A | arxiv.org |
| LLaMA 65B | 82.8 | N/A | Self-reported | medium | N/A | arxiv.org |
| LLaMA 65B | 82.8 | N/A | Independent | medium | N/A | arxiv.org |
| Qwen2.5-72B-Instruct | 82.6 | N/A | Independent | medium | N/A | arxiv.org |
| Falcon 40B | 82.4 | N/A | Independent | medium | N/A | arxiv.org |
| Nemotron-4 15B | 82.4 | N/A | Self-reported | medium | N/A | arxiv.org |
| LLaMA 33B | 82.3 | N/A | Independent | medium | Feb 27, 2023 | arxiv.org |
| LLaMA 33B | 82.3 | N/A | Self-reported | medium | N/A | arxiv.org |
| LLaMA 33B | 82.3 | N/A | Independent | medium | N/A | arxiv.org |
| PaLM 540B | 82.3 | N/A | Independent | medium | N/A | arxiv.org |
| text-davinci-001 | 82.3 | N/A | Independent | medium | N/A | arxiv.org |
| Mistral 7B | 82.2 | N/A | Independent | medium | Jan 8, 2024 | arxiv.org |
| PaLM 2-S | 82.2 | N/A | Independent | medium | May 17, 2023 | arxiv.org |
| Mistral-7B-Instruct-v0.2 | 82.2 | N/A | Independent | medium | N/A | arxiv.org |
| Megatron-Turing NLG 530B | 82.0 | N/A | Independent | medium | N/A | arxiv.org |
| LLaMA 2 34B | 81.9 | N/A | Independent | medium | Jul 18, 2023 | arxiv.org |
| LLaMA 2 34B | 81.9 | N/A | Self-reported | medium | N/A | arxiv.org |
| LLaMA 2 34B | 81.9 | N/A | Independent | medium | N/A | arxiv.org |
| MPT 30B | 81.9 | N/A | Independent | medium | N/A | arxiv.org |
| Chinchilla-70B | 81.8 | N/A | Independent | medium | Mar 29, 2022 | arxiv.org |
| Gopher-280B | 81.8 | N/A | Independent | medium | Dec 8, 2021 | arxiv.org |
| Chinchilla | 81.8 | N/A | Independent | medium | N/A | arxiv.org |
| GLaM 64B/64E | 81.8 | N/A | Self-reported | medium | N/A | arxiv.org |
| Gopher | 81.8 | N/A | Self-reported | medium | N/A | arxiv.org |
| Gopher | 81.8 | N/A | Independent | medium | N/A | arxiv.org |
| PaLM-cont 62B | 81.4 | N/A | Independent | medium | N/A | arxiv.org |
| Gemma 7B | 81.2 | N/A | Independent | medium | N/A | arxiv.org |
Top 50 of 112 by score
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsLoading community runs…
About
Physical Interaction: Question Answering (PIQA) is a physical commonsense reasoning and a corresponding benchmark dataset. PIQA was designed to investigate the physical knowledge of existing models. To what extent are current approaches actually learning about the world?
Available in EleutherAI lm-evaluation-harness as task family piqa (1 tasks).