LetTokenBurnCan Frontier LLMs Earn $1 Million from Real-World Freelance Software Engineering?
A benchmark of freelance software engineering tasks from Upwork, valued at $1 million USD total in realworld payouts.
$ npm i -g lettokenburn-cli$ ltb bench pull swe-lancer
External harness. pip install inspect-evals && inspect eval inspect_evals/swe_lancer --model <provider/model> https://github.com/UKGovernmentBEIS/inspect_evals
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
No sourced scores recorded yet.
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
A benchmark of freelance software engineering tasks from Upwork, valued at $1 million USD total in realworld payouts.