LetTokenBurnBlog: Read the blog GitHub Repository: View the code arXiv Paper: View the paper Overview SWE-PRBench is a benchmark of 350 pull requests with human-annotated ground truth for evaluating whether LLMs can identify the same issues that real human reviewers flag in production code.
$ npm i -g lettokenburn-cli$ ltb bench info swe-prbench
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
No sourced scores recorded yet.
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
SWE-PRBench
Benchmarking AI Code Review Quality Against Human Pull Request Feedback
Blog: Read the blog GitHub Repository: View the code arXiv Paper: View the paper
Overview
SWE-PRBench is a benchmark of 350 pull requests with human-annotated ground truth for evaluating whether LLMs can identify the same issues that real human reviewers flag in production code. Existing benchmarks like SWE-Bench measure whether models can produce correct code. SWE-PRBench…