HANS
Heuristic Analysis for NLI Systems
The HANS (Heuristic Analysis for NLI Systems) dataset which contains many examples where the heuristics fail.
How to run
ltb · hans
$ npm i -g lettokenburn-cli$ ltb bench info hans
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
8 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Qwen3-VL | 0.7 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| Paddle-VL1.5 | 0.7 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| InternVL3.5 | 0.7 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| Deepseek-OCR2 | 0.6 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| MinerU2.7.6-VLM | 0.6 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| PaddleOCR PP-V3 | 0.6 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| MinerU2.7.6-pipeline | 0.4 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| GOT | 0.4 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsLoading community runs…
About
The HANS (Heuristic Analysis for NLI Systems) dataset which contains many examples where the heuristics fail.
Source: Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference