HellaSwag
Harbor: pendingRecent work by Zellers et al.
Verified86%Knowledge & Reasoning
- Samples
- 10,042
- Released
- 2019
- Scores
- 134
MAWPS
MAth Word ProblemS
MAWPS is an online repository of Math Word Problems, to provide a unified testbed to evaluate different algorithms.
How to run
ltb · mawps
$ npm i -g lettokenburn-cli$ ltb bench info mawps
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
6 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| llama 2 | 82.4 | N/A | Independent | medium | Jul 18, 2023 | arxiv.org |
| GPT-3.5 Turbo | 80.3 | N/A | Independent | medium | Jun 24, 2023 | arxiv.org |
| GPT-3 | 19.8 | N/A | Independent | medium | N/A | paperswithcode.com |
| GPT-J | 9.9 | N/A | Independent | medium | Jun 24, 2023 | arxiv.org |
| GPT-3 text-curie-001 | 4.1 | N/A | Independent | medium | Jun 24, 2023 | arxiv.org |
| GPT-3 text-babbage-001 | 2.8 | N/A | Independent | medium | Jun 24, 2023 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsLoading community runs…
About
MAWPS is an online repository of Math Word Problems, to provide a unified testbed to evaluate different algorithms. MAWPS allows for the automatic construction of datasets with particular characteristics, providing tools for tuning the lexical and template overlap of a dataset as well as for filtering ungrammatical problems from web-sourced corpora. The online nature of this repository facilitates easy community contribution. Amassed 3,320 problems, including the full datasets used in several previous works.