LetTokenBurn
Sign in
Auto71%Harbor: pendingCodingAgentic / Tool Use

Web-Bench

We developed Web-Bench as a benchmark for evaluating the performance of LLMs on real-world web projects.

PaperRepoLeaderboardHomepage

How to run

ltb · web-bench-pwc
$ npm i -g lettokenburn-cli
$ ltb bench info web-bench-pwc

External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.

Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs

Scores

33 sourced entries
HarnessSourceConfidenceLink
Claude 3.7 Sonnet Thinking (2025-02-19)25.1N/AIndependentmediumMay 12, 2025arxiv.org
Claude 3.5 Sonnet23.0N/AIndependentmediumMay 12, 2025arxiv.org
GPT-4.121.1N/AIndependentmediumMay 12, 2025arxiv.org
GPT-4.1-mini20.8N/AIndependentmediumMay 12, 2025arxiv.org
Doubao Pro 1.5 Thinking20.1N/AIndependentmediumMay 12, 2025arxiv.org
GPT-4o17.2N/AIndependentmediumMay 12, 2025arxiv.org
DeepSeek V3 032417.1N/AIndependentmediumMay 12, 2025arxiv.org
DeepSeek Coder V216.7N/AIndependentmediumMay 12, 2025arxiv.org
Doubao Pro 1.5 32K16.6N/AIndependentmediumMay 12, 2025arxiv.org
Llama 4 Maverick16.0N/AIndependentmediumMay 12, 2025arxiv.org
Qwen Max (2025-01-25)15.9N/AIndependentmediumMay 12, 2025arxiv.org
Gemini 2.5 Pro15.7N/AIndependentmediumMay 12, 2025arxiv.org
Claude 3.5 Haiku (2024-10-22)15.4N/AIndependentmediumMay 12, 2025arxiv.org
Gemini 2.0 Flash Experimental15.3N/AIndependentmediumMay 12, 2025arxiv.org
Gemini Pro 1.514.8N/AIndependentmediumMay 12, 2025arxiv.org
DeepSeek R114.5N/AIndependentmediumMay 12, 2025arxiv.org
o4-mini13.3N/AIndependentmediumMay 12, 2025arxiv.org
Mistral Large 241113.0N/AIndependentmediumMay 12, 2025arxiv.org
Gemini Flash 1.512.8N/AIndependentmediumMay 12, 2025arxiv.org
Qwen Plus (2025-01-25)11.8N/AIndependentmediumMay 12, 2025arxiv.org
Grok 2 121211.3N/AIndependentmediumMay 12, 2025arxiv.org
Qwen2.5-72B-Instruct10.5N/AIndependentmediumMay 12, 2025arxiv.org
o110.4N/AIndependentmediumMay 12, 2025arxiv.org
Gemma 3 27B9.9N/AIndependentmediumMay 12, 2025arxiv.org
o3-mini9.1N/AIndependentmediumMay 12, 2025arxiv.org
GPT-4o-mini8.5N/AIndependentmediumMay 12, 2025arxiv.org
MiniMax Text8.5N/AIndependentmediumMay 12, 2025arxiv.org
GLM-4-04147.5N/AIndependentmediumMay 12, 2025arxiv.org
GPT-4.1-nano7.1N/AIndependentmediumMay 12, 2025arxiv.org
Llama 3.36.6N/AIndependentmediumMay 12, 2025arxiv.org
Moonshot Kimi Latest5.2N/AIndependentmediumMay 12, 2025arxiv.org
llama-4 Scout5.0N/AIndependentmediumMay 12, 2025arxiv.org
Doubao Pro 1.5 32K Lite3.5N/AIndependentmediumMay 12, 2025arxiv.org

Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.

Reproduced by the community

All runs
Loading community runs…

About

We developed Web-Bench as a benchmark for evaluating the performance of LLMs on real-world web projects.

  1. 50 projects, each consisting of 20 tasks with sequential dependencies. The tasks implement project features in sequence, simulating real-world human development workflows.
  1. When designing Web-Bench, we aim to cover the foundational elements of Web development: Web Standards and Web Frameworks. Given the scale and complexity of these projects, which were designed by engineers with 5-10 years of experience, each presents a significant challenge. On average, a single project takes 4–8 hours for a senior engineer to complete.
  1. On our given benchmark agent (Web-Agent), SOTA (Claude 3.7 Sonnet) achieves only 25.1% Pass@1, significantly lower than SWE-Bench's Verified (65.4%) and Full (33.8%) scores (2025.4).