LetTokenBurn
Sign in
Auto71%Harbor: pendingSafety & AlignmentCodingAgentic / Tool Use

CWE-bench

CWE-bench: Refocusing Agentic Security from Exploitation to Defense

A private benchmark of 120 held-out repository audit-and-repair tasks spanning 73 CWE types, eight languages, and all OWASP Top 10 2025 categories.

Paper

How to run

ltb · cwe-bench
$ npm i -g lettokenburn-cli
$ ltb bench info cwe-bench

External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.

Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs

Scores

11 sourced entries
HarnessSourceConfidenceLink
Gemini 4 Argon68.0antigravityIndependentmediumOct 12, 2026arxiv.org
GPT-6 Astra68.0codexIndependentmediumOct 12, 2026arxiv.org
Grok-4.768.0opencodeIndependentmediumOct 12, 2026arxiv.org
Claude Opus 5.567.0claude codeIndependentmediumOct 12, 2026arxiv.org
Claude Fable 5.158.0claude codeIndependentmediumOct 12, 2026arxiv.org
Grok-4.657.0opencodeIndependentmediumOct 12, 2026arxiv.org
DeepSeek V4.1 Flash55.0opencodeIndependentmediumOct 12, 2026arxiv.org
Muse Spark 1.355.0opencodeIndependentmediumOct 12, 2026arxiv.org
HY4 Preview53.0opencodeIndependentmediumOct 12, 2026arxiv.org
GPT-6 Sol52.0codexIndependentmediumOct 12, 2026arxiv.org
Inkling37.0opencodeIndependentmediumOct 12, 2026arxiv.org

Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.

Reproduced by the community

All runs
Loading community runs…

About

CWE-bench consists of 120 tasks on real open-source repositories with injected vulnerabilities. Agents receive only the code and a threat model description, without hints on weakness type or location.

Scoring uses a programmatic verifier checking exploit blocking and functionality preservation, validated by agentic judges. Best agents achieve 68% pass@1, showing significant variation in retry gains and cost across models.