OpenWebText
OpenWebText is an open-source recreation of the WebText corpus.
How to run
ltb · openwebtext
$ npm i -g lettokenburn-cli$ ltb bench info openwebtext
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
7 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| GPT2-GELU | 19.2 | N/A | Independent | medium | Feb 3, 2025 | arxiv.org |
| GPT2-Fourier | 18.7 | N/A | Independent | medium | Feb 3, 2025 | arxiv.org |
| GPT2-Tropical | 18.6 | N/A | Independent | medium | Feb 3, 2025 | arxiv.org |
| GPT2-Hermite | 18.4 | N/A | Independent | medium | Feb 3, 2025 | arxiv.org |
| GPT2-124M | 3.1 | N/A | Independent | medium | Feb 14, 2019 | d4mucfpksywv.cloudfront.net |
| GPT2-81M-LOOP | 3.1 | N/A | Independent | medium | Sep 21, 2024 | arxiv.org |
| GPT2-Hermite | 2.9 | N/A | Independent | medium | Feb 3, 2025 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsLoading community runs…
About
OpenWebText is an open-source recreation of the [WebText](/dataset/webtext) corpus. The text is web content extracted from URLs shared on Reddit with at least three upvotes. (38GB).
Source: RoBERTa: A Robustly Optimized BERT Pretraining Approach