Model

Gemini Pro

Google

deepmind.google

Math

Math
BenchmarkScoreMetricHarnessSourceConfidenceLink
MATH32.6accuracyN/AIndependentmediumarxiv.org
MATH-V17.7accuracyN/AIndependentmediumarxiv.org

Multimodal

Multimodal
BenchmarkScoreMetricHarnessSourceConfidenceLink
IllusionVQA51.3accuracyN/AIndependentmediumarxiv.org
NExT-QA28.0wupsN/AIndependentmediumarxiv.org
REBUS13.2accuracyN/AIndependentmediumarxiv.org

Abstract Reasoning

Abstract Reasoning
BenchmarkScoreMetricHarnessSourceConfidenceLink
DrillBench59.7accuracygenerative (hosted)Independentmediumarxiv.org

Reproduced by the community

Loading community runs…