Abt-Buy
Harbor adapterThe Abt-Buy dataset for entity resolution derives from the online retailers Abt.com and Buy.com.
Auto71%Coding
- Samples
- 1,916
- Released
- 2010
- Scores
- 7
Amazon-Google
The Amazon-Google dataset for entity resolution derives from the online retailers Amazon.com and the product search service of Google accessible through the Google Base Data API.
How to run
ltb · amazon-google
$ npm i -g lettokenburn-cli$ ltb bench info amazon-google
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
9 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| GPT4 0613 Fewshot 10 | 85.2 | N/A | Independent | medium | Oct 17, 2023 | arxiv.org |
| GPT-4o mini 2024.07 18 Fine Tuned | 80.3 | N/A | Independent | medium | Sep 12, 2024 | arxiv.org |
| Text Davinci 002 Fewshot 10 | 63.5 | N/A | Independent | medium | May 20, 2022 | arxiv.org |
| GPT-4o | 63.5 | N/A | Independent | medium | Sep 12, 2024 | arxiv.org |
| GPT-4o-mini | 59.2 | N/A | Independent | medium | Sep 12, 2024 | arxiv.org |
| Text Davinci 002 Zeroshot | 54.3 | N/A | Independent | medium | May 20, 2022 | arxiv.org |
| Meta-Llama-3.1-70B-Instruct | 51.4 | N/A | Independent | medium | Sep 12, 2024 | arxiv.org |
| Meta-Llama-3.1-8B-Instruct_fine_tuned | 50.0 | N/A | Independent | medium | Sep 12, 2024 | arxiv.org |
| Meta-Llama-3.1-8B-Instruct | 49.2 | N/A | Independent | medium | Sep 12, 2024 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsLoading community runs…
About
The Amazon-Google dataset for entity resolution derives from the online retailers Amazon.com and the product search service of Google accessible through the Google Base Data API. The dataset contains 1363 entities from amazon.com and 3226 google products as well as a gold standard (perfect mapping) with 1300 matching record pairs between the two data sources. The common attributes between the two data sources are: product name, product description, manufacturer and price.
The dataset was initially published in the repository of the Database Group of the University of Leipzig: https://dbs.uni-leipzig.de/research/projects/object_matching/benchmark_datasets_for_entity_resolution
To enable the reproducibility of the results and the comparability of the performance of different matchers on the Amazon-Google matching task, the dataset was split into fixed train, validation and test sets. The fixed splits are provided in the CompERBench repository:
http://data.dws.informatik.uni-mannheim.de/benchmarkmatchingtasks/index.html