Model
Gemini 2.0 Flash Experimental
Knowledge & Reasoning
Knowledge & Reasoning| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| LiveBench | 59.3 | global_average | N/A | Independent | medium | livebench.ai |
| MMLU | 79.7 | exact_match | N/A | Independent | medium | crfm.stanford.edu |
| PHRBench | 84.6 | auroc | N/A | Independent | medium | arxiv.org |
Math
Math| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| MATH | 89.7 | accuracy | N/A | Independent | medium | paperswithcode.com |
Coding
Coding| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| Aider Polyglot | 22.2 | percent_correct | N/A | Independent | medium | aider.chat |
| Web-Bench | 15.3 | pass@1 | N/A | Independent | medium | arxiv.org |
Multimodal
Multimodal| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| MM-Vet | 81.2 | gpt-4_score | N/A | Independent | medium | paperswithcode.com |
| MM-Vet v2 | 77.1 | gpt-4_score | N/A | Independent | medium | paperswithcode.com |
Safety & Alignment
Safety & AlignmentReproduced by the community
Loading community runs…