LetTokenBurn
Sign in
Auto100%Harbor adapterMathKnowledge & Reasoning

ParaMAWPS

Paraphrased Math Word Problem Solving Repository

This repository contains the code, data, and models of the paper titled "Math Word Problem Solving by Generating Linguistic Variants of Problem Statements" published in the Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop).

PaperRepoDatasetLeaderboardHomepage

How to run

ltb · paramawps
$ npm i -g lettokenburn-cli
$ ltb bench pull paramawps --n 50
$ ltb bench run ./paramawps -a claude-code -m anthropic/claude-opus-4-8
$ ltb bench push

License is link-only: ltb bench pull fetches rows from the original host for local evaluation only and does not write a solution/ directory. Do not redistribute the generated tasks.

Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs

Scores

4 sourced entries
HarnessSourceConfidenceLink
GPT-3.5 Turbo73.0N/AIndependentmediumJun 24, 2023arxiv.org
GPT-J5.9N/AIndependentmediumJun 24, 2023arxiv.org
GPT-3 text-curie-0014.2N/AIndependentmediumJun 24, 2023arxiv.org
GPT-3 text-babbage-0013.2N/AIndependentmediumJun 24, 2023arxiv.org

Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.

Reproduced by the community

All runs
Loading community runs…

About

This repository contains the code, data, and models of the paper titled "Math Word Problem Solving by Generating Linguistic Variants of Problem Statements" published in the *Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop)*.

The work is outlined in a more detailed and expository manner in our Bachelor of Science (B.Sc.) thesis titled "Variational Mathematical Reasoning: Enhancing Math Word Problem Solvers with Linguistic Variants and Disentangled Attention" which can be accessed from the Islamic University of Technology (IUT) Institutional Repository.

![arXiv](https://arxiv.org/abs/2306.13899) ![anthology](https://aclanthology.org/2023.acl-srw.49/) ![GoogleScholar](https://scholar.google.com/citations?view_op=view_citation&hl=en&user=4L_7vaoAAAAJ&citation_for_view=4L_7vaoAAAAJ:d1gkVwhDpl0C) ![ResearchGate](https://www.researchgate.net/publication/372918265_Math_Word_Problem_Solving_by_Generating_Linguistic_Variants_of_Problem_Statements) ![HuggingFace](https://huggingface.co/datasets/Starscream-11813/ParaMAWPS)

![PDF](https://aclanthology.org/2023.acl-srw.49.pdf) ![Slides](https://drive.google.com/file/d/1R-lB53BeaM-7XE0EoBZ5qfy-BSc61gup/view?usp=sharing) ![Video](https://aclanthology.org/2023.acl-srw.49.mp4) ![Poster](https://drive.google.com/file/d/1FfexZxsKqL0mw2cUy7hTr4vhp7WBjhRW/view?usp=sharing) ![Thesis](http://103.82.172.44:8080/xmlui/handle/123456789/2092)

License: Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International

![license](http://creativecommons.org/licenses/by-nc-sa/4.0/)

Dataset

In order to download our dataset PᴀʀᴀMAWPS, please navigate to the ParaMAWPS folder. We use an $80:10:10$ train-validation-test split for our PᴀʀᴀMAWPS dataset. The splits are available in .json format in the aforementioned folder.

Data Format

Each row consists of a Math Word Problem (MWP). The table below describes what each column signifies.

Column Title | Description ------------ | ------------- id | The unique identification number of the sample. Seed problems have id size of $\leq 4$, whereas, variant problems have id size of $> 4$. The last variant of a seed problem (generally with the id "$16000i$", where $i$ is the id of the seed problem) is the inverse variant of the seed problem. original_text | The problem statement of the MWP. The seed problems have the same problem statement as present in the Mᴀᴡᴘs dataset. equation | The equation with a variable \\(x\\) which solves the MWP quantity_tagged_text | The problem statement of the MWP, where each quantity is replaced with a unique tag \\([Q_i]\\) quantity_tagged_equation | The equation with a variable \\(x\\) which solves the MWP, but each quantity is replaced with its unique tag \\([Q_i]\\) in the problem statement have_constant | Whether the use of a constant value is required to solve the MWP <br> For an MWP sample \\(i\\) with have_constant label \\(C_i\\), the boolean label is,<br>$C_i =\begin{cases} \text{FALSE}, & \text{if $i$ requires $0$ constant values}\\\ \text{TRUE}, & \text{if $i$ requires $\geq 1$ constant values}\end{cases}$

Types of Variations

<img src="https://github.com/Starscream-11813/Variational-Mathematical-Reasoning/raw/main/images/ACLMWP_variationtypes.png" alt="drawing" style="width:1000px;"/>

Dataset Statistics

<img src="https://github.com/Starscream-11813/Variational-Mathematical-Reasoning/raw/main/images/ACLMWP_datasetcomparisontable.png" alt="drawing" style="width:500px;"/> <img src="https://github.com/Starscream-11813/Variational-Mathematical-Reasoning/raw/main/images/ACLMWP_datasetcomparisongraph.png" alt="drawing" style="width:500px;"/>

Methodology

<img src="https://github.com/Starscream-11813/Variational-Mathematical-Reasoning/raw/main/images/ACLMWP_architecture2.png" alt="drawing" style