BBH
Harbor adapterBIG-Bench Hard evaluates advanced reasoning capabilities across 23 challenging tasks using over 6,500 English samples.
Verified100%Knowledge & Reasoning
- Samples
- 6,511
- Released
- 2022
- Scores
- 59
ANLI
Adversarial NLI
Adversarial NLI (ANLI) is a dataset collected via an iterative, adversarial human-and-model-in-the-loop procedure.
How to run
ltb · anli
$ npm i -g lettokenburn-cli$ ltb bench pull anli --n 50$ ltb bench run ./anli -a claude-code -m anthropic/claude-opus-4-8$ ltb bench push
License is link-only: ltb bench pull fetches rows from the original host for local evaluation only and does not write a solution/ directory. Do not redistribute the generated tasks.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
12 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| UNICORN | 87.3 | N/A | Self-reported | medium | N/A | ojs.aaai.org |
| GPT-3.5-Turbo-1106 | 58.1 | N/A | Independent | medium | N/A | arxiv.org |
| Phi-3-small-8k-instruct | 58.1 | N/A | Self-reported | medium | N/A | arxiv.org |
| Meta-Llama-3-8B-Instruct | 57.3 | N/A | Independent | medium | N/A | arxiv.org |
| Phi-3-medium-128k-instruct | 55.8 | N/A | Self-reported | medium | N/A | arxiv.org |
| Mixtral-8x7B-Instruct-v0.1 | 55.2 | N/A | Independent | medium | N/A | arxiv.org |
| Phi-3-mini-4k-instruct | 52.8 | N/A | Self-reported | medium | N/A | arxiv.org |
| Gemma 7B | 48.7 | N/A | Independent | medium | N/A | arxiv.org |
| Mistral-7B-v0.1 | 47.1 | N/A | Independent | medium | N/A | arxiv.org |
| Phi-2 | 42.5 | N/A | Self-reported | medium | N/A | arxiv.org |
| Megatron-Turing NLG 530B | 39.7 | N/A | Self-reported | medium | N/A | arxiv.org |
| text-davinci-001 | 35.4 | N/A | Independent | medium | N/A | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsLoading community runs…
About
Adversarial NLI (ANLI) is a dataset collected via an iterative, adversarial human-and-model-in-the-loop procedure. It consists of three rounds that progressively increase in difficulty and complexity, and each question-answer includes annotator- provided explanations.
Available in EleutherAI lm-evaluation-harness as task family anli (3 tasks).