BrickBench
Harbor: pendingA benchmark of 300 text prompts for agentic LEGO-set design, evaluating physical validity, semantic alignment, and design quality across three complexity settings.
- Samples
- 300
- Released
- 2026
- Scores
- 11
InsClaimBench
InsClaimBench: Benchmarking Insurance Claim Adjudication Across the Decision Chain
An end-to-end benchmark with 3,780 insurance cases evaluating LLMs on atomic rules, adjudication modules, and payout decisions across auto, property, and health domains.
How to run
ltb · insclaimbench
$ npm i -g lettokenburn-cli$ ltb bench info insclaimbench
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
6 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Kimi K2.6 | 80.2 | N/A | Independent | medium | Oct 7, 2026 | arxiv.org |
| GPT-5.6 Sol | 80.0 | N/A | Independent | medium | Oct 7, 2026 | arxiv.org |
| Gemini 3.8 Flash | 79.3 | N/A | Independent | medium | Oct 7, 2026 | arxiv.org |
| DeepSeek V4.1 Flash | 77.5 | N/A | Independent | medium | Oct 7, 2026 | arxiv.org |
| Qwen3.8 Flash | 76.6 | N/A | Independent | medium | Oct 7, 2026 | arxiv.org |
| qwen3.5-27B | 74.2 | N/A | Independent | medium | Oct 7, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsLoading community runs…
About
InsClaimBench contains 3,780 cases grounded in real materials and structured rules, featuring controlled factual variants to test judgment propagation. It evaluates models from atomic rule judgments to final payout amounts.
Scoring uses accuracy metrics for decisions and amounts. Results show high atomic accuracy but significant drops in joint decision-amount accuracy and rule-vector exact match, highlighting propagation failures.