LetTokenBurn
Sign in
Auto71%Harbor: pendingSafety & Alignment

OMIT

OMIT the Action: Measuring Framing-Invariant Omission Bias under Philosophical Disagreement

A benchmark of 218 paired-frame scenarios across 10 conflict types measuring framing-invariant omission bias in LLMs using philosophical disagreement signals.

Paper

How to run

ltb · omit
$ npm i -g lettokenburn-cli
$ ltb bench info omit

External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.

Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs

Scores

13 sourced entries
HarnessSourceConfidenceLink
GPT-4o0.1N/AIndependentmediumOct 6, 2026arxiv.org
Gemini 2.0 Flash Experimental0.1N/AIndependentmediumOct 6, 2026arxiv.org
Llama-3.1-70B-Instruct0.1N/AIndependentmediumOct 6, 2026arxiv.org
Grok-4.30.1N/AIndependentmediumOct 6, 2026arxiv.org
qwen3.5-27B0.2N/AIndependentmediumOct 6, 2026arxiv.org
Claude Sonnet 4.5 (2025-09-29)0.2N/AIndependentmediumOct 6, 2026arxiv.org
DeepSeek Chat0.2N/AIndependentmediumOct 6, 2026arxiv.org
GLM-4.60.3N/AIndependentmediumOct 6, 2026arxiv.org
qwen3.5-9B0.3N/AIndependentmediumOct 6, 2026arxiv.org
Gemma 3 27B0.3N/AIndependentmediumOct 6, 2026arxiv.org
GPT-4o-mini0.5N/AIndependentmediumOct 6, 2026arxiv.org
Llama-3.1-8B-Instruct0.5N/AIndependentmediumOct 6, 2026arxiv.org
Gemma 3 12B IT0.6N/AIndependentmediumOct 6, 2026arxiv.org

Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.

Reproduced by the community

All runs
Loading community runs…

About

OMIT consists of 218 scenarios derived from MoralChoice, reframed into action/omission pairs and labeled with 10 moral conflict types based on a five-perspective philosophical persona panel.

It measures the Omission Bias Rate (OBR), finding bias is pervasive but inversely correlated with model size. Interventions encouraging moral principle consideration reduce bias and increase frame-consistent responses.