Model

PaLM 2-L

Google

Knowledge & Reasoning

Benchmark	Score	Metric	Harness	Source	Confidence	Link
ARC-Challenge	69.2	accuracy	N/A	Independent	medium	arxiv.org
ARC-Easy	89.7	accuracy	N/A	Independent	medium	arxiv.org
BoolQ	90.9	accuracy	N/A	Independent	medium	arxiv.org
BoolQ	90.9	score	N/A	Independent	medium	arxiv.org
CommitmentBank	87.5	accuracy	N/A	Independent	medium	arxiv.org
HellaSwag	87.4	accuracy	N/A	Independent	medium	arxiv.org
HellaSwag	86.8	overall_accuracy	N/A	Self-reported	medium	arxiv.org
LAMBADA	86.9	accuracy	N/A	Independent	medium	arxiv.org
MultiRC	88.2	f1	N/A	Independent	medium	arxiv.org
Natural Questions	37.5	exact_match	N/A	Independent	medium	arxiv.org
OpenBookQA	58.5	accuracy	N/A	Independent	medium	arxiv.org
PIQA	85.0	accuracy	N/A	Independent	medium	arxiv.org
RTE	79.3	accuracy	N/A	Independent	medium	arxiv.org
StoryCloze	87.4	accuracy	N/A	Independent	medium	arxiv.org
Trivia QA	86.1	exact_match	N/A	Independent	medium	arxiv.org
Trivia QA	86.1	exact_match	N/A	Self-reported	medium	arxiv.org
WebQuestions	28.2	exact_match	N/A	Independent	medium	arxiv.org
WiC	66.8	accuracy	N/A	Independent	medium	arxiv.org
WinoGrande	83.0	accuracy	N/A	Independent	medium	arxiv.org
WinoGrande	83.0	accuracy	N/A	Self-reported	medium	arxiv.org
WSC	86.9	accuracy	N/A	Independent	medium	arxiv.org

Benchmark	Score	Metric	Harness	Source	Confidence	Link
COPA	96.0	accuracy	N/A	Independent	medium	arxiv.org