Aider Polyglot
Harbor nativeAider Polyglot: 6种语言diff编辑
Verified71%Coding
- Samples
- 225
- Released
- 2024
- Scores
- 61
AlgoREval
AlgoREval: Evaluating LLMs on Algorithmic Code Retrieval
A benchmark of 599 problems spanning 77 algorithms, 14 domains, 7 languages, and 4 graph representations to evaluate parametric code retrieval in LLMs.
How to run
ltb · algoreval
$ npm i -g lettokenburn-cli$ ltb bench info algoreval
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
14 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| gpt-oss-20b | 88.0 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| GPT-4.1-mini | 80.8 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| QwQ-32B | 69.6 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| GPT-4o-mini | 66.3 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| Phi-4 | 59.4 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| Codestral-22B | 52.6 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| DeepSeek-Coder-Instruct 33B | 46.4 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| Yi-Coder-9B | 46.2 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| DeepSeek-R1-Distill-Qwen-14B | 39.6 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| Qwen2.5-7B-Instruct | 37.4 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| Llama-3.1-8B-Instruct | 31.4 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| CodeGemma-7B | 25.9 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| StarCoder2-15B | 13.7 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| CodeLlama-34b-hf | 12.9 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsLoading community runs…
About
AlgoREval consists of 599 zero-shot problems requiring LLMs to reproduce correct implementations of named algorithms from parametric memory across seven programming languages. It assesses retrieval accuracy without external aids, finding that performance varies by language and input representation rather than model size, with prompt augmentation offering limited benefits.