Model

PaLM 540B

Google

Knowledge & Reasoning

Benchmark	Score	Metric	Harness	Source	Confidence	Link
BoolQ	88.7	score	N/A	Independent	medium	arxiv.org
HellaSwag	83.8	accuracy	N/A	Independent	medium	arxiv.org
HellaSwag	83.8	overall_accuracy	N/A	Self-reported	medium	jmlr.org
HellaSwag	83.6	accuracy	N/A	Independent	medium	arxiv.org
HellaSwag	83.4	accuracy	N/A	Independent	medium	arxiv.org
LAMBADA	89.7	accuracy	N/A	Independent	medium	arxiv.org
LAMBADA	81.8	accuracy	N/A	Independent	medium	arxiv.org
LAMBADA	77.9	accuracy	N/A	Independent	medium	arxiv.org
LAMBADA	77.9	score	N/A	Independent	medium	arxiv.org
MMLU	69.3	exact_match	N/A	Independent	medium	arxiv.org
Natural Questions	39.6	exact_match	N/A	Independent	medium	arxiv.org
Natural Questions	29.3	exact_match	N/A	Independent	medium	arxiv.org
Natural Questions	21.2	exact_match	N/A	Independent	medium	arxiv.org
OpenBookQA	68.0	accuracy	N/A	Self-reported	medium	jmlr.org
OpenBookQA	53.4	accuracy	N/A	Independent	medium	arxiv.org
OpenBookQA	53.4	accuracy	N/A	Independent	medium	arxiv.org
PIQA	82.3	score	N/A	Independent	medium	arxiv.org
RTE	79.6	accuracy	N/A	Independent	medium	arxiv.org
RTE	78.7	accuracy	N/A	Independent	medium	arxiv.org
RTE	72.9	accuracy	N/A	Independent	medium	arxiv.org
StrategyQA	76.4	accuracy	N/A	Independent	medium	arxiv.org
Trivia QA	81.4	exact_match	N/A	Independent	medium	arxiv.org
Trivia QA	81.4	exact_match	N/A	Independent	medium	arxiv.org
Trivia QA	81.4	exact_match	N/A	Self-reported	medium	jmlr.org
Trivia QA	76.9	exact_match	N/A	Independent	medium	arxiv.org
WebQuestions	43.5	exact_match	N/A	Independent	medium	arxiv.org
WebQuestions	22.6	exact_match	N/A	Independent	medium	arxiv.org
WebQuestions	10.6	exact_match	N/A	Independent	medium	arxiv.org
WinoGrande	85.1	accuracy	N/A	Independent	medium	arxiv.org
WinoGrande	85.1	accuracy	N/A	Self-reported	medium	jmlr.org
WinoGrande	81.1	accuracy	N/A	Independent	medium	arxiv.org
WSC	89.5	accuracy	N/A	Independent	medium	arxiv.org
WSC	89.1	accuracy	N/A	Independent	medium	arxiv.org
WSC	86.3	accuracy	N/A	Independent	medium	arxiv.org

Math

Benchmark	Score	Metric	Harness	Source	Confidence	Link
GSM8K	56.5	accuracy	N/A	Independent	medium	arxiv.org
GSM8K	56.5	exact_match	N/A	Independent	medium	arxiv.org
GSM8K	17.9	accuracy	N/A	Independent	medium	arxiv.org
MATH	8.8	accuracy	N/A	Independent	medium	arxiv.org

Coding

Benchmark	Score	Metric	Harness	Source	Confidence	Link
MBPP	36.8	accuracy	N/A	Independent	medium	arxiv.org

Multilingual

Benchmark	Score	Metric	Harness	Source	Confidence	Link
MGSM	55.0	average_(%)	N/A	Independent	medium	arxiv.org
TyDiQA-GoldP	52.9	exact_match	N/A	Independent	medium	arxiv.org

Knowledge & Reasoning

Math

Coding

Multilingual

Reproduced by the community