LetTokenBurnDuplexAct: How Interactive Are Frontier Full-Duplex Spoken Language Models?
A benchmark with 4,691 instances evaluating interaction action success in full-duplex spoken language models across real-world and synthetic scenarios.
$ npm i -g lettokenburn-cli$ ltb bench info duplexact
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| GPT Realtime 2.1 | 95.7 | direct replay | Independent | medium | Oct 13, 2026 | arxiv.org |
| Gemini 3.1 Flash Live | 85.4 | direct replay | Independent | medium | Oct 13, 2026 | arxiv.org |
| MiniCPM-o 4.5 | 74.4 | direct replay | Independent | medium | Oct 13, 2026 | arxiv.org |
| Grok Voice | 74.3 | direct replay | Independent | medium | Oct 13, 2026 | arxiv.org |
| Raon-SpeechChat | 70.3 | direct replay | Independent | medium | Oct 13, 2026 | arxiv.org |
| VITA-1.5 | 64.4 | direct replay | Independent | medium | Oct 13, 2026 | arxiv.org |
| Lychee-FD | 53.1 | direct replay | Independent | medium | Oct 13, 2026 | arxiv.org |
| Voila-Autonomous | 45.8 | direct replay | Independent | medium | Oct 13, 2026 | arxiv.org |
| Moshi | 45.6 | direct replay | Independent | medium | Oct 13, 2026 | arxiv.org |
| Mini-Omni2 | 32.7 | direct replay | Independent | medium | Oct 13, 2026 | arxiv.org |
| BayLing-Duplex | 31.2 | direct replay | Independent | medium | Oct 13, 2026 | arxiv.org |
| Freeze-Omni | 23.2 | direct replay | Independent | medium | Oct 13, 2026 | arxiv.org |
| FLM-Audio | 18.3 | direct replay | Independent | medium | Oct 13, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
DuplexAct contains 4,691 test instances spanning casual conversation, meetings, customer service, and interviews, with 54% derived from real-world recordings. It defines a three-level taxonomy of eight interaction actions and 24 conditions.
It evaluates success via immediate execution and subsequent follow-through using action-specific metrics. The benchmark reveals uneven model performance and weak subsequent reasoning capabilities across the action space.