LetTokenBurn
Sign in
Auto57%Harbor: pendingSafety & Alignment

VALCON

Value Alignment Benchmark (VAB)

A benchmark suite (VAB-Consistency, VAB-Boundary, VAB-Challenge) measuring value consistency in LLMs across generator and evaluator roles via dual-role variation.

PaperRepo

How to run

ltb · valcon
$ npm i -g lettokenburn-cli
$ ltb bench info valcon

External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.

Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs

Scores

12 sourced entries
HarnessSourceConfidenceLink
GPT-5.199.8N/AIndependentmediumOct 15, 2026arxiv.org
Doubao-Seed-1.6-25061599.2N/AIndependentmediumOct 15, 2026arxiv.org
Qwen3-Next-80B-A3B-Thinking98.9N/AIndependentmediumOct 15, 2026arxiv.org
DeepSeek R198.5N/AIndependentmediumOct 15, 2026arxiv.org
Gemini 2.5 Flash97.4N/AIndependentmediumOct 15, 2026arxiv.org
ERNIE-5.0-Thinking-Preview96.7N/AIndependentmediumOct 15, 2026arxiv.org
GLM-4.696.2N/AIndependentmediumOct 15, 2026arxiv.org
Qwen3 Max (2025-09-23)96.0N/AIndependentmediumOct 15, 2026arxiv.org
Kimi K2 Thinking Turbo95.7N/AIndependentmediumOct 15, 2026arxiv.org
MiniMax-M293.8N/AIndependentmediumOct 15, 2026arxiv.org
Qwen3-32B91.4N/AIndependentmediumOct 15, 2026arxiv.org
Hunyuan-T1-Latest88.8N/AIndependentmediumOct 15, 2026arxiv.org

Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.

Reproduced by the community

All runs
Loading community runs…

About

VALCON constructs evaluation data by pairing an LLM's generative behavior with its evaluative judgments under identical principles. It comprises three subsets: VAB-Consistency, VAB-Boundary, and VAB-Challenge.

Scoring measures the agreement between the generator's intended label and the judge's prediction. Experiments on 28 LLMs reveal pervasive cross-role inconsistency, where compliant models often fail to align judgments with their own generations.