LetTokenBurnHakemBench: A Turkish Benchmark of Typed Decisions
A Turkish benchmark with 2,346 items and 4,275 questions measuring typed decisions (choice, yes/no, score) across seven tracks including legal, education, and safety.
$ npm i -g lettokenburn-cli$ ltb bench info hakembench
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Gemini 3.8 Flash | 0.9 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| GPT-5.6 Sol | 0.8 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| GLM-5.3 | 0.8 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| Jev 1.13 | 0.8 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| Kev 4B | 0.7 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| Kev 9B | 0.7 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| Qwen3.5 4B | 0.7 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| DeepSeek V4 Pro | 0.6 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| Decider 2B | 0.5 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
HakemBench contains 2,346 Turkish items covering fact-checking, education, guardrails, legal routing, moderation, spam, and customer support. Gold labels are primarily AI-generated via blind model passes and panel votes, not human-verified.
Scoring combines macro F1, calibration (Brier score), and selective automation via geometric mean. The top leaderboard score is 0.888, with significant performance gaps observed in guardrail and moderation tracks due to training data leakage.