LetTokenBurn
Sign in
Auto86%Harbor: pendingKnowledge & ReasoningSafety & Alignment

PARCEL

ParentheticAl Reasoning and Claim EvaLuation

A claim-level benchmark for legal hallucination detection with 3,396 parenthetical-style claims labeled as Supported, Refuted, or Not Found from NY Court of Appeals decisions.

PaperRepo

How to run

ltb · parcel
$ npm i -g lettokenburn-cli
$ ltb bench info parcel

External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.

Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs

Scores

7 sourced entries
HarnessSourceConfidenceLink
Gemini 3 Flash Preview97.0N/AIndependentmediumOct 7, 2026arxiv.org
Kimi K2 Thinking Turbo95.0N/AIndependentmediumOct 7, 2026arxiv.org
GLM-4.794.0N/AIndependentmediumOct 7, 2026arxiv.org
Qwen3-235B-A22B-Thinking-250793.0N/AIndependentmediumOct 7, 2026arxiv.org
Gemini 2.5 Flash90.0N/AIndependentmediumOct 7, 2026arxiv.org
GPT-oss 120B90.0N/AIndependentmediumOct 7, 2026arxiv.org
Llama-3.3-70B-Instruct81.0N/AIndependentmediumOct 7, 2026arxiv.org

Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.

Reproduced by the community

All runs
Loading community runs…

About

PARCEL consists of 3,396 synthetic claims derived from New York State Court of Appeals decisions (2024-2025), labeled via Natural Language Inference taxonomy. It tests whether LLMs can detect unsupported or fabricated citations in legal RAG contexts.

Evaluated in a zero-shot setting, top models reached 0.97 accuracy but struggled with missing support and plausible fabricated citations, revealing weaknesses in claim-level groundedness verification.