LetTokenBurn
Sign in
Auto43%Harbor: pendingSafety & AlignmentMultimodal

SSUI-Dialog

A cross-modal multi-turn dialogue dataset with safety-critical intent ambiguity and dual-intent (safe/unsafe) supervision for evaluating proactive inquiry in MLLMs.

Paper

How to run

ltb · ssui-dialog
$ npm i -g lettokenburn-cli
$ ltb bench info ssui-dialog

External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.

Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs

Scores

3 sourced entries
HarnessSourceConfidenceLink
Claude 3.5 Sonnet75.0N/AIndependentmediumOct 15, 2026arxiv.org
GPT-4o70.4N/AIndependentmediumOct 15, 2026arxiv.org
Gemini 1.5 Pro61.8N/AIndependentmediumOct 15, 2026arxiv.org

Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.

Reproduced by the community

All runs
Loading community runs…

About

SSUI-Dialog is a cross-modal multi-turn dialogue dataset constructed by injecting safety-critical premise ambiguity into queries and simulating dual user replies (safe and unsafe intents). It features human-verified trajectories designed to test models' ability to clarify ambiguous cross-modal inputs.

The dataset is evaluated using the Cross-modal Safety with Helpful Clarification (CSHC) metric, which jointly assesses safety, helpfulness, and clarification economy. SAFE-PRO outperforms baselines on this dataset, demonstrating improved safety scores and reduced over-refusal.