Model

Gopher

Vendor N/A

Knowledge & Reasoning

Benchmark	Score	Metric	Harness	Source	Confidence	Link
BoolQ	79.4	score	N/A	Independent	medium	arxiv.org
BoolQ	79.3	accuracy	N/A	Independent	medium	arxiv.org
HellaSwag	79.2	overall_accuracy	N/A	Independent	medium	arxiv.org
LAMBADA	74.5	score	N/A	Self-reported	medium	arxiv.org
LAMBADA	74.5	score	N/A	Independent	medium	arxiv.org
MMLU	60.0	exact_match	N/A	Self-reported	medium	arxiv.org
MMLU	60.0	exact_match	N/A	Independent	medium	arxiv.org
Natural Questions	28.2	exact_match	N/A	Independent	medium	arxiv.org
PIQA	81.8	score	N/A	Self-reported	medium	arxiv.org
PIQA	81.8	score	N/A	Independent	medium	arxiv.org
PubMed Cognitive Control Abstracts	0.6	bpb	N/A	Independent	medium	arxiv.org
siqa	50.6	accuracy	N/A	Independent	medium	arxiv.org
Trivia QA	57.2	exact_match	N/A	Independent	medium	arxiv.org
Trivia QA	57.1	exact_match	N/A	Self-reported	medium	arxiv.org
Ubuntu IRC	1.1	bpb	N/A	Independent	medium	arxiv.org
WinoGrande	70.2	accuracy	N/A	Independent	medium	arxiv.org
WinoGrande	70.1	accuracy	N/A	Self-reported	medium	arxiv.org