LetTokenBurn
Sign in
Verified100%Harbor nativeCodingAgentic / Tool Use

SWE-bench Verified

Software Engineering Benchmark

SWE-bench Verified: 500个真实GitHub Bug修复

PaperRepoDatasetLeaderboardHomepage

How to run

ltb · swe-bench-verified
$ npm i -g lettokenburn-cli
$ ltb bench run swe-bench/swe-bench-verified@2 --benchmark swe-bench-verified -a claude-code -m anthropic/claude-opus-4-8
$ ltb bench push

Harbor-native: you can also call Harbor directly.

harbor
$ harbor run -d swe-bench/swe-bench-verified@2 -a claude-code -m anthropic/claude-opus-4-8

Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs

Scores

46 sourced entries
HarnessSourceConfidenceLink
Claude Opus 4.888.6N/ASelf-reportedlowJun 1, 2026swebench.com
Claude Opus 4.783.5N/AIndependenthighApr 20, 2026epoch.ai
Claude Opus 4.680.8N/ASelf-reportedlowJun 1, 2026swebench.com
GPT-5.5 Pre Release80.6N/AIndependenthighApr 24, 2026epoch.ai
MiniMax M2.580.2N/ASelf-reportedlowJun 1, 2026swebench.com
Claude Sonnet 4.679.6N/ASelf-reportedlowJun 1, 2026swebench.com
Gemini-3.5-flash79.3N/AIndependenthighJun 1, 2026epoch.ai
Claude Opus 4.678.7N/AIndependenthighFeb 18, 2026epoch.ai
GLM-5.278.7N/AIndependenthighJun 25, 2026epoch.ai
Gemini 3.1 Pro78.0N/ASelf-reportedlowJun 1, 2026swebench.com
GLM-577.8N/ASelf-reportedlowJun 1, 2026swebench.com
DeepSeek V4 Pro77.6N/AIndependenthighJun 18, 2026epoch.ai
Qwen3.7 Max77.3N/AIndependenthighJun 18, 2026epoch.ai
GPT-5.476.9N/AIndependenthighMar 6, 2026epoch.ai
Kimi K2.576.8N/ASelf-reportedlowJun 1, 2026swebench.com
Qwen3.6 Max Preview76.7N/AIndependenthighMay 28, 2026epoch.ai
Kimi K2.676.7N/AIndependenthighMay 8, 2026epoch.ai
Claude Opus 4.5 (2025-11-01)76.7N/AIndependenthighFeb 5, 2026epoch.ai
Qwen 3.576.4N/ASelf-reportedlowJun 1, 2026swebench.com
Gemini 3.1 Pro Preview Customtools75.6N/AIndependenthighFeb 24, 2026epoch.ai
Gemini 3 Flash Preview75.4N/AIndependenthighFeb 18, 2026epoch.ai
Claude Sonnet 4.675.2N/AIndependenthighFeb 21, 2026epoch.ai
Gemini 3 Pro75.0N/ASelf-reportedlowJun 1, 2026swebench.com
GPT-5.3-codex74.8N/AIndependenthighFeb 25, 2026epoch.ai
GLM-5.174.2N/AIndependenthighMay 15, 2026epoch.ai
Kimi K2.573.8N/AIndependenthighFeb 17, 2026epoch.ai
GPT-5.2-chat73.8N/AIndependenthighFeb 12, 2026epoch.ai
GPT-573.6N/AIndependenthighFeb 6, 2026epoch.ai
MiMo-V2-Flash73.4N/ASelf-reportedlowJun 1, 2026swebench.com
Claude Opus 4.1 (2025-08-05)73.3N/AIndependenthighFeb 11, 2026epoch.ai
Gemini 3 Pro72.9N/AIndependenthighFeb 13, 2026epoch.ai
GLM-572.1N/AIndependenthighFeb 15, 2026epoch.ai
Claude Sonnet 4.5 (2025-09-29)71.3N/AIndependenthighFeb 5, 2026epoch.ai
Claude Opus 4 (2025-05-14)70.7N/AIndependenthighFeb 6, 2026epoch.ai
GPT-5.168.0N/AIndependenthighFeb 18, 2026epoch.ai
DeepSeek V3.267.8N/ASelf-reportedlowJun 1, 2026swebench.com
GPT-5-mini64.7N/AIndependenthighFeb 1, 2026epoch.ai
GPT-oss 120B62.4N/ASelf-reportedlowJun 1, 2026swebench.com
o362.3N/AIndependenthighFeb 12, 2026epoch.ai
Claude 3.7 Sonnet Thinking (2025-02-19)61.0N/AIndependenthighFeb 4, 2026epoch.ai
Qwen3.6 Plus57.9N/AIndependenthighMay 14, 2026epoch.ai
Gemini 2.5 Pro57.6N/AIndependenthighFeb 13, 2026epoch.ai
DeepSeek R149.2N/ASelf-reportedlowJun 1, 2026swebench.com
Grok 349.0N/ASelf-reportedlowJun 1, 2026swebench.com
GPT-4.148.5N/AIndependenthighFeb 8, 2026epoch.ai
GPT-4o31.0N/AIndependenthighFeb 11, 2026epoch.ai

Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.

Reproduced by the community

All runs
Loading community runs…

About

SWE-bench Verified (Software Engineering Benchmark) — 500个真实GitHub Bug修复. Scoring: Resolved%. Difficulty: 极高. Imported from benchmark_research.md (2026-06).