HPCAgent-Bench
HPCAgent-Bench: A Framework for Sound Benchmarking of Code-Optimizing Agents, Harnesses, and Skills
A framework with 689 kernels across three tracks (loop-level, scientific computing, ML) evaluating code-optimizing agents on speedup, correctness, and token cost via hidden fuzzed inputs.
How to run
$ npm i -g lettokenburn-cli$ ltb bench info hpcagent-bench
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
3 sourced entriesEvery number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsAbout
The benchmark comprises 689 kernels in NumPy, derived into C/Fortran references, covering loop-level reasoning, scientific computing proxies, and ML models. It supports CPU/GPU execution via a common ABI.
Scoring uses an external judge to check correctness on hidden fuzzed inputs and measure runtime. Key metrics include intervention efficacy (speedup, success rate) and weighted token cost, preventing agent exploitation of test systems.