ASPIRE
Acoustic-Semantic Probing for Inconsistency REsolution
An adversarial benchmark of 1,200 synthesized audio-text pairs with semantic conflicts to evaluate Speech Emotion Recognition models' robustness against semantic dominance.
How to run
$ npm i -g lettokenburn-cli$ ltb bench info aspire
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
6 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| WavLM-Large | 86.4 | N/A | Independent | medium | Oct 14, 2026 | arxiv.org |
| emotion2vec+ | 86.4 | N/A | Independent | medium | Oct 14, 2026 | arxiv.org |
| Qwen2-Audio | 85.3 | N/A | Independent | medium | Oct 14, 2026 | arxiv.org |
| SALMONN | 81.2 | N/A | Independent | medium | Oct 14, 2026 | arxiv.org |
| Whisper-large-v3 | 79.3 | N/A | Independent | medium | Oct 14, 2026 | arxiv.org |
| SeamlessM4T | 76.9 | N/A | Independent | medium | Oct 14, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsAbout
ASPIRE contains 1,200 high-quality samples (800 conflict, 400 normal) synthesized via EmotiVoice TTS, covering four structural conflict types where vocal tone contradicts textual semantics. It employs two novel metrics: Semantic Overconfidence Penalty (SOP) to measure text bias, and Latent Decoupling Degree (LDD) to assess feature orthogonality.
The benchmark reveals that existing Audio LLMs fail under polarity conflicts due to extreme semantic overconfidence. ACR-Net, a proposed dual-stream architecture, achieves superior Acoustic Accuracy by explicitly decoupling contradictory representations into orthogonal latent spaces.