Model
GPT
OpenAI · closed weights
Coding
Coding| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| PolyCodeEval | 71.7 | pass_rate | repocoder | Independent | medium | arxiv.org |
| PolyCodeEval | 68.9 | pass_rate | hcp | Independent | medium | arxiv.org |
| PolyCodeEval | 50.3 | pass_rate | aligncoder | Independent | medium | arxiv.org |
| PolyCodeEval | 48.8 | pass_rate | llm direct | Independent | medium | arxiv.org |
Multimodal
Multimodal| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| ScribbleEdit | 0.3 | intention_accuracy | N/A | Independent | medium | arxiv.org |
Reproduced by the community
Loading community runs…