CNN/Daily Mail
CNN/Daily Mail is a dataset for text summarization.
How to run
$ npm i -g lettokenburn-cli$ ltb bench info cnn-daily-mail
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
7 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| GLM-XXLarge | 44.7 | N/A | Independent | medium | Mar 18, 2021 | arxiv.org |
| PALM | 44.3 | N/A | Independent | medium | Apr 14, 2020 | arxiv.org |
| ERNIE-GENLARGE | 44.0 | N/A | Independent | medium | Jan 26, 2020 | arxiv.org |
| T5-11B | 43.5 | N/A | Independent | medium | Oct 23, 2019 | arxiv.org |
| ERNIE-GENBASE | 42.3 | N/A | Independent | medium | Jan 26, 2020 | arxiv.org |
| PALM | 41.4 | N/A | Independent | medium | Apr 14, 2020 | arxiv.org |
| GPT-2 | 29.3 | N/A | Independent | medium | Feb 14, 2019 | d4mucfpksywv.cloudfront.net |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsAbout
CNN/Daily Mail is a dataset for text summarization. Human generated abstractive summary bullets were generated from news stories in CNN and Daily Mail websites as questions (with one of the entities hidden), and stories as the corresponding passages from which the system is expected to answer the fill-in the-blank question. The authors released the scripts that crawl, extract and generate pairs of passages and questions from these websites.
In all, the corpus has 286,817 training pairs, 13,368 validation pairs and 11,487 test pairs, as defined by their scripts. The source documents in the training set have 766 words spanning 29.74 sentences on an average while the summaries consist of 53 words and 3.72 sentences.
Source: Abstractive Text Summarization using Sequence-to-sequence RNNs and Beyond