LetTokenBurnSUITE-102: A 102-condition benchmark for Indian Spoken Language Identification
A 102-condition, 318,861-utterance benchmark with speaker- and corpus-held-out tiers for evaluating robustness of spoken language identification across 23 Indian languages.
$ npm i -g lettokenburn-cli$ ltb bench info suite-102
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Anuvani-LID | 0.9 | N/A | Independent | medium | Oct 14, 2026 | arxiv.org |
| Voxlect Indic | 0.7 | N/A | Independent | medium | Oct 14, 2026 | arxiv.org |
| Gemini-3.5-flash | 0.7 | N/A | Independent | medium | Oct 14, 2026 | arxiv.org |
| MMS-LID-126 | 0.7 | N/A | Independent | medium | Oct 14, 2026 | arxiv.org |
| VL107 ECAPA | 0.6 | N/A | Independent | medium | Oct 14, 2026 | arxiv.org |
| AmberNet | 0.6 | N/A | Independent | medium | Oct 14, 2026 | arxiv.org |
| MMS-LID-256 | 0.5 | N/A | Independent | medium | Oct 14, 2026 | arxiv.org |
| Whisper-v3 | 0.5 | N/A | Independent | medium | Oct 14, 2026 | arxiv.org |
| MMS-LID-2048 | 0.5 | N/A | Independent | medium | Oct 14, 2026 | arxiv.org |
| MMS-LID-4017 | 0.5 | N/A | Independent | medium | Oct 14, 2026 | arxiv.org |
| GPT-Audio | 0.4 | N/A | Independent | medium | Oct 14, 2026 | arxiv.org |
| Sarvam Saaras | 0.3 | N/A | Independent | medium | Oct 14, 2026 | arxiv.org |
| GPT-audio-mini | 0.3 | N/A | Independent | medium | Oct 14, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
SUITE-102 comprises 318,861 utterances across 102 controlled conditions, including speaker- and corpus-held-out test sets. It evaluates systems on short, noisy, and code-mixed audio spanning 22 scheduled Indian languages and Indian-accented English.
Scoring uses mean macro-F1 across 37 core conditions. Anuvani-LID achieves 0.914 macro-F1, outperforming generalist models like Gemini 3.5 Flash, demonstrating the need for dedicated, calibrated LID models in complex linguistic landscapes.