ARC-Challenge
Harbor: pendingARC-Challenge — AI evaluation benchmark listed by opencompass.
Incomplete14%Knowledge & Reasoning
- Samples
- N/A
- Released
- N/A
- Scores
- 26
QuALITY
Question Answering with Long Input Texts, Yes!
QuALITY — AI evaluation benchmark listed by opencompass.
How to run
ltb · quality
$ npm i -g lettokenburn-cli$ ltb bench pull quality
External harness. pip install opencompass && opencompass --models <model_cfg> --datasets QuALITY_gen (OpenCompass dataset config dir 'QuALITY') https://github.com/open-compass/opencompass
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
3 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Claude 1.3 | 84.1 | N/A | Independent | medium | Jul 11, 2023 | www-files.anthropic.com |
| Claude 2 | 83.2 | N/A | Independent | medium | Jul 11, 2023 | www-files.anthropic.com |
| Claude Instant 1.1 | 80.5 | N/A | Independent | medium | Jul 11, 2023 | www-files.anthropic.com |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsLoading community runs…
About
QuALITY (Question Answering with Long Input Texts, Yes!) is a multiple-choice question answering dataset for long document comprehension. The dataset consists of context passages in English that have an average length of about 5,000 tokens, much longer than typical current models can process. Unlike in prior work with passages, the questions are written and validated by contributors who have read the entire passage, rather than relying on summaries or excerpts.