ReTurn
ReTurn: Selective History Use across Multimodal Turns
ReTurn is a 7,000-task omni-modal benchmark evaluating selective history use in multi-turn conversations via visual and audio evidence.
How to run
$ npm i -g lettokenburn-cli$ ltb bench info return
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
12 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| StepAudio 3 | 98.4 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| Audio Flamingo 3 | 97.8 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| Doubao-Seed-2.0-Lite | 97.4 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| Qwen 3.8 Omni Flash | 96.2 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| Qwen3-Omni | 95.0 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| Qwen3.7 Plus | 94.9 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| MiniCPM-o 4.5 | 93.9 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| MiMo V2.5 | 92.6 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| Qwen3-VL | 92.0 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| Nemotron-3-Nano-Omni | 90.1 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| MiniCPM-V-4-5 | 87.5 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| InternVL3.5 | 80.8 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsAbout
ReTurn comprises 7,000 base tasks spanning visual and audio evidence, organized into task-carrying (Reconfirm/Reground) and evidence-carrying (Retrieve/Rebind) history conditions with matched single-turn references.
It evaluates models using open-ended and multiple-choice formats, measuring accuracy drops from direct input to conversational settings to assess history selection and interference handling.