LetTokenBurnThe Abt-Buy dataset for entity resolution derives from the online retailers Abt.com and Buy.com.
$ npm i -g lettokenburn-cli$ ltb bench pull abt-buy --n 50$ ltb bench run ./abt-buy -a claude-code -m anthropic/claude-opus-4-8$ ltb bench push
License is link-only: ltb bench pull fetches rows from the original host for local evaluation only and does not write a solution/ directory. Do not redistribute the generated tasks.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| GPT4 0613 Zeroshot | 95.8 | N/A | Independent | medium | Oct 17, 2023 | arxiv.org |
| GPT-4o mini 2024.07 18 Fine Tuned | 94.1 | N/A | Independent | medium | Sep 12, 2024 | arxiv.org |
| GPT-4o | 92.2 | N/A | Independent | medium | Sep 12, 2024 | arxiv.org |
| GPT-4o-mini | 87.7 | N/A | Independent | medium | Sep 12, 2024 | arxiv.org |
| Meta-Llama-3.1-8B-Instruct_fine_tuned | 87.3 | N/A | Independent | medium | Sep 12, 2024 | arxiv.org |
| Meta-Llama-3.1-70B-Instruct | 79.1 | N/A | Independent | medium | Sep 12, 2024 | arxiv.org |
| Meta-Llama-3.1-8B-Instruct | 56.6 | N/A | Independent | medium | Sep 12, 2024 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
The Abt-Buy dataset for entity resolution derives from the online retailers Abt.com and Buy.com. The dataset contains 1081 entities from abt.com and 1092 entities from buy.com as well as a gold standard (perfect mapping) with 1097 matching record pairs between the two data sources. The common attributes between the two data sources are: product name, product description and product price.
The dataset was initially published in the repository of the Database Group of the University of Leipzig: https://dbs.uni-leipzig.de/research/projects/object_matching/benchmark_datasets_for_entity_resolution
To enable the reproducibility of the results and the comparability of the performance of different matchers on the Abt-Buy matching task, the dataset was split into fixed train, validation and test sets. The fixed splits are provided in the CompERBench repository:
http://data.dws.informatik.uni-mannheim.de/benchmarkmatchingtasks/index.html