Model

PaLM 2-M

Google

Knowledge & Reasoning

Benchmark	Score	Metric	Harness	Source	Confidence	Link
ARC-Challenge	64.9	accuracy	N/A	Independent	medium	arxiv.org
ARC-Easy	88.0	accuracy	N/A	Independent	medium	arxiv.org
BoolQ	88.6	accuracy	N/A	Independent	medium	arxiv.org
BoolQ	88.6	score	N/A	Independent	medium	arxiv.org
CommitmentBank	80.4	accuracy	N/A	Independent	medium	arxiv.org
HellaSwag	86.7	accuracy	N/A	Independent	medium	arxiv.org
HellaSwag	84.0	overall_accuracy	N/A	Self-reported	medium	arxiv.org
LAMBADA	83.7	accuracy	N/A	Independent	medium	arxiv.org
MultiRC	84.1	f1	N/A	Independent	medium	arxiv.org
Natural Questions	32.0	exact_match	N/A	Independent	medium	arxiv.org
OpenBookQA	57.4	accuracy	N/A	Self-reported	medium	arxiv.org
OpenBookQA	56.2	accuracy	N/A	Independent	medium	arxiv.org
PIQA	83.2	accuracy	N/A	Independent	medium	arxiv.org
RTE	81.9	accuracy	N/A	Independent	medium	arxiv.org
StoryCloze	86.7	accuracy	N/A	Independent	medium	arxiv.org
Trivia QA	81.7	exact_match	N/A	Independent	medium	arxiv.org
Trivia QA	81.7	exact_match	N/A	Self-reported	medium	arxiv.org
WebQuestions	26.9	exact_match	N/A	Independent	medium	arxiv.org
WiC	52.0	accuracy	N/A	Independent	medium	arxiv.org
WinoGrande	79.2	accuracy	N/A	Independent	medium	arxiv.org
WinoGrande	79.2	accuracy	N/A	Self-reported	medium	arxiv.org
WSC	88.1	accuracy	N/A	Independent	medium	arxiv.org

Benchmark	Score	Metric	Harness	Source	Confidence	Link
COPA	90.0	accuracy	N/A	Independent	medium	arxiv.org