ANLI
Harbor adapterAdversarial NLI (ANLI) is a dataset collected via an iterative, adversarial human-and-model-in-the-loop procedure.
Verified100%Knowledge & Reasoning
- Samples
- 169,265
- Released
- 2019
- Scores
- 12
BBH
BIG-Bench Hard
BIG-Bench Hard evaluates advanced reasoning capabilities across 23 challenging tasks using over 6,500 English samples.
How to run
ltb · bbh
$ npm i -g lettokenburn-cli$ ltb bench pull bbh --n 50$ ltb bench run ./bbh -a claude-code -m anthropic/claude-opus-4-8$ ltb bench push
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
59 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| gemini-1.5-pro-001 | 89.2 | N/A | Self-reported | medium | N/A | storage.googleapis.com |
| DeepSeek-V3 | 87.5 | N/A | Self-reported | medium | N/A | arxiv.org |
| Gemini 1.5 Pro 001 Feb24 | 84.0 | N/A | Self-reported | medium | N/A | storage.googleapis.com |
| Llama-3.1-405B-Instruct | 82.9 | N/A | Independent | medium | N/A | arxiv.org |
| Phi-3-medium-128k-instruct | 81.4 | N/A | Self-reported | medium | N/A | arxiv.org |
| Qwen2.5-72B-Instruct | 79.8 | N/A | Independent | medium | N/A | arxiv.org |
| Phi-3-small-8k-instruct | 79.1 | N/A | Self-reported | medium | N/A | arxiv.org |
| DeepSeek-V2 | 78.8 | N/A | Self-reported | medium | N/A | arxiv.org |
| GPT-4-0613 | 75.1 | N/A | Independent | medium | N/A | arxiv.org |
| Phi-3-mini-4k-instruct | 71.7 | N/A | Self-reported | medium | N/A | arxiv.org |
| Yi 34B | 71.7 | N/A | Independent | medium | N/A | arxiv.org |
| StableBeluga2 | 69.3 | N/A | Independent | medium | N/A | arxiv.org |
| Llama 2 70B | 64.9 | N/A | Independent | medium | N/A | arxiv.org |
| GPT-3.5 Turbo 0613 | 61.6 | N/A | Independent | medium | N/A | arxiv.org |
| Phi-2 | 59.4 | N/A | Self-reported | medium | N/A | arxiv.org |
| Nemotron-4 15B | 58.7 | N/A | Self-reported | medium | N/A | arxiv.org |
| LLaMA 65B | 58.4 | N/A | Independent | medium | N/A | arxiv.org |
| LLaMA 2 13B | 58.2 | N/A | Independent | medium | N/A | arxiv.org |
| Mistral-7B-v0.1 | 56.1 | N/A | Independent | medium | N/A | arxiv.org |
| Gemma 7B | 55.1 | N/A | Independent | medium | N/A | arxiv.org |
| Gemma 7B | 55.1 | N/A | Self-reported | medium | N/A | arxiv.org |
| Qwen-14B | 55.0 | N/A | Independent | medium | N/A | arxiv.org |
| Qwen-14B | 53.4 | N/A | Self-reported | medium | N/A | arxiv.org |
| InternLM 20B | 52.5 | N/A | Independent | medium | N/A | arxiv.org |
| Llama 2 70B | 51.2 | N/A | Self-reported | medium | N/A | arxiv.org |
| LLaMA 33B | 50.0 | N/A | Independent | medium | N/A | arxiv.org |
| Baichuan-2-13B-Base | 49.0 | N/A | Independent | medium | N/A | arxiv.org |
| Baichuan-2-13B-Base | 48.8 | N/A | Self-reported | medium | N/A | arxiv.org |
| Baichuan2-13B-Chat | 47.2 | N/A | Independent | medium | N/A | arxiv.org |
| Yi-6B | 47.2 | N/A | Independent | medium | N/A | arxiv.org |
| Qwen 7B | 45.0 | N/A | Self-reported | medium | N/A | arxiv.org |
| LLaMA 2 34B | 44.1 | N/A | Self-reported | medium | N/A | arxiv.org |
| LLaMA 2 34B | 44.1 | N/A | Independent | medium | N/A | arxiv.org |
| LLaMA 65B | 43.5 | N/A | Self-reported | medium | N/A | arxiv.org |
| Vicuna 13B V1.1 | 43.0 | N/A | Independent | medium | N/A | arxiv.org |
| Baichuan-13B-Base | 43.0 | N/A | Self-reported | medium | N/A | arxiv.org |
| Baichuan-2-7B-Base | 41.6 | N/A | Independent | medium | N/A | arxiv.org |
| Baichuan-2-7B-Base | 41.6 | N/A | Self-reported | medium | N/A | arxiv.org |
| LLaMA 33B | 39.8 | N/A | Self-reported | medium | N/A | arxiv.org |
| LLaMA 2 13B | 39.4 | N/A | Self-reported | medium | N/A | arxiv.org |
| LLaMA 2 7B | 39.2 | N/A | Independent | medium | N/A | arxiv.org |
| XVERSE-13B | 38.1 | N/A | Independent | medium | N/A | arxiv.org |
| MPT 30B | 38.0 | N/A | Independent | medium | N/A | arxiv.org |
| LLaMA 13B | 37.9 | N/A | Independent | medium | N/A | arxiv.org |
| Falcon 40B | 37.1 | N/A | Independent | medium | N/A | arxiv.org |
| InternLM 7B | 37.0 | N/A | Independent | medium | N/A | arxiv.org |
| LLaMA 13B | 37.0 | N/A | Self-reported | medium | N/A | arxiv.org |
| MPT 7B | 35.6 | N/A | Independent | medium | N/A | arxiv.org |
| Gemma-2B-IT | 35.2 | N/A | Self-reported | medium | N/A | arxiv.org |
| INTELLECT-1-Instruct | 34.9 | N/A | Self-reported | medium | N/A | arxiv.org |
Top 50 of 59 by score
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsLoading community runs…
About
This benchmark comprises the most difficult subset of BIG-Bench, featuring 23 diverse tasks requiring complex reasoning. It is scored via exact-match accuracy on a dataset of approximately 6,511 examples and serves as a rigorous testbed for evaluating model robustness beyond standard benchmarks.