LetTokenBurn
Sign in
Auto71%Harbor: pendingCodingAgentic / Tool Use

WEAVE

WEAVE: Weaving Real-World Code Evolution into Dependency-Structured Tasks for Benchmarking and Training Code Agents

A benchmark of 300 held-out executable tasks spanning six languages, constructed from dependency-structured PR DAGs to evaluate code agents on multi-PR software engineering objectives.

Paper

How to run

ltb · weave
$ npm i -g lettokenburn-cli
$ ltb bench info weave

External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.

Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs

Scores

14 sourced entries
HarnessSourceConfidenceLink
GLM-5.243.0clineIndependentmediumOct 15, 2026arxiv.org
Claude Opus 4.741.0opencodeIndependentmediumOct 15, 2026arxiv.org
Kimi K2.637.3clineIndependentmediumOct 15, 2026arxiv.org
DeepSeek V4 Flash34.3clineIndependentmediumOct 15, 2026arxiv.org
GLM-5.234.0opencodeIndependentmediumOct 15, 2026arxiv.org
Claude Opus 4.630.3opencodeIndependentmediumOct 15, 2026arxiv.org
Kimi K2.629.3opencodeIndependentmediumOct 15, 2026arxiv.org
GLM-5.128.7opencodeIndependentmediumOct 15, 2026arxiv.org
DeepSeek V4 Flash21.0opencodeIndependentmediumOct 15, 2026arxiv.org
GLM-5.121.0clineIndependentmediumOct 15, 2026arxiv.org
Kimi K2.518.7clineIndependentmediumOct 15, 2026arxiv.org
Kimi K2.514.0opencodeIndependentmediumOct 15, 2026arxiv.org
MiniMax-2.712.3opencodeIndependentmediumOct 15, 2026arxiv.org
MiniMax-2.79.7clineIndependentmediumOct 15, 2026arxiv.org

Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.

Reproduced by the community

All runs
Loading community runs…

About

WEAVE constructs executable tasks by organizing verified pull requests into bounded dependency DAGs using repository-calibrated typed change loci. It includes 3200 training tasks and 300 benchmark tasks across six programming languages.

Evaluation uses Graph Credit, a deterministic metric measuring dependency-consistent partial progress. Experiments show it exposes structurally inconsistent credit obscured by flat scoring and provides effective verifiable rewards for agent training.