CritPt
Harbor: pendingCritPt — AI evaluation benchmark listed by opencompass.
Verified71%Knowledge & Reasoning
- Samples
- 70
- Released
- 2025
- Scores
- 141
KiranaBench
KiranaBench: Can a Language Model Run a Neighbourhood Shop?
A multi-agent simulation of an Indian general store evaluating LLMs on long-horizon business management in English, Hindi, and Hinglish.
How to run
ltb · kiranabench
$ npm i -g lettokenburn-cli$ ltb bench info kiranabench
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
4 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Qwen3.8 27B | 13683.0 | N/A | Independent | medium | Oct 16, 2026 | arxiv.org |
| google/gemma-4-31B-it | -3730.0 | N/A | Independent | medium | Oct 16, 2026 | arxiv.org |
| LFM2.5-8B | -126554.0 | N/A | Independent | medium | Oct 16, 2026 | arxiv.org |
| gpt-oss-20b | -193411.0 | N/A | Independent | medium | Oct 16, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsLoading community runs…
About
Simulates a kirana shop with four LLM agents (manager, cashier, helper, bookkeeper) interacting with customers and suppliers. Demand is calibrated to India's Household Consumption Expenditure Survey.
Scores models on net profit over 30 simulated days against a fixed-rule baseline. Current open-weight LLMs perform poorly, earning significantly less than the baseline.