LetTokenBurnPhysicsMate: A Curriculum-Grounded Bengali Benchmark for Secondary Physics QA with Small-Model Adaptation
A benchmark of 1,834 Bengali question-answer pairs derived from Grade 9-10 physics textbooks and a multi-relational knowledge graph.
$ npm i -g lettokenburn-cli$ ltb bench info physicsmate
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Gemini-2.5-FL | 82.9 | llm-judge (gemini-2.5-flash-lite) | Independent | medium | Sep 30, 2026 | arxiv.org |
| GPT-4o-mini | 69.5 | llm-judge (gemini-2.5-flash-lite) | Independent | medium | Sep 30, 2026 | arxiv.org |
| Qwen3 4B | 27.6 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| Qwen3 1.7B | 5.5 | lora adaptation | Independent | medium | Sep 30, 2026 | arxiv.org |
| Qwen-3-0.6B | 0.0 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
PhysicsMate is constructed by ingesting the National Curriculum and Textbook Board (NCTB) Grade 9-10 syllabus via OCR and semantic chunking into 811 units. These are formalized into a knowledge graph with 1,760 nodes across ten ontological types to support category-level evaluation.
The benchmark evaluates small language models adapted via LoRA (Qwen3 at 0.6B-4B parameters). Key findings show significant accuracy improvements (+5.5% to +23.3%) and demonstrate that structured curricular knowledge adapts best, while entity-level knowledge is least benefited.