Model
PaLM 2-S
Knowledge & Reasoning
Knowledge & Reasoning| Benchmark | Score | Metric | Harness | Source | Confidence | Link |
|---|---|---|---|---|---|---|
| ARC-Challenge | 59.6 | accuracy | N/A | Independent | medium | arxiv.org |
| ARC-Easy | 85.6 | accuracy | N/A | Independent | medium | arxiv.org |
| BoolQ | 88.1 | accuracy | N/A | Independent | medium | arxiv.org |
| BoolQ | 88.1 | score | N/A | Independent | medium | arxiv.org |
| CommitmentBank | 82.1 | accuracy | N/A | Independent | medium | arxiv.org |
| HellaSwag | 85.6 | accuracy | N/A | Independent | medium | arxiv.org |
| HellaSwag | 82.0 | overall_accuracy | N/A | Self-reported | medium | arxiv.org |
| LAMBADA | 80.7 | accuracy | N/A | Independent | medium | arxiv.org |
| MultiRC | 84.0 | f1 | N/A | Independent | medium | arxiv.org |
| Natural Questions | 25.3 | exact_match | N/A | Independent | medium | arxiv.org |
| OpenBookQA | 57.4 | accuracy | N/A | Independent | medium | arxiv.org |
| OpenBookQA | 56.2 | accuracy | N/A | Self-reported | medium | arxiv.org |
| PIQA | 82.2 | accuracy | N/A | Independent | medium | arxiv.org |
| RTE | 78.7 | accuracy | N/A | Independent | medium | arxiv.org |
| StoryCloze | 85.6 | accuracy | N/A | Independent | medium | arxiv.org |
| Trivia QA | 75.2 | exact_match | N/A | Independent | medium | arxiv.org |
| Trivia QA | 75.2 | exact_match | N/A | Self-reported | medium | arxiv.org |
| WebQuestions | 21.8 | exact_match | N/A | Independent | medium | arxiv.org |
| WiC | 50.6 | accuracy | N/A | Independent | medium | arxiv.org |
| WinoGrande | 77.9 | accuracy | N/A | Independent | medium | arxiv.org |
| WinoGrande | 77.9 | accuracy | N/A | Self-reported | medium | arxiv.org |
| WSC | 84.6 | accuracy | N/A | Independent | medium | arxiv.org |
Coding
CodingMultimodal
MultimodalInstruction Following
Instruction FollowingReproduced by the community
Loading community runs…