LetTokenBurn
Sign in
Verified100%Harbor adapterKnowledge & Reasoning

RAFT

Realworld Annotated Few-shot Tasks

Real-world Annotated Few-shot Tasks (RAFT)

PaperRepoDatasetLeaderboardHomepage

How to run

ltb · raft
$ npm i -g lettokenburn-cli
$ ltb bench pull raft --n 50
$ ltb bench run ./raft -a claude-code -m anthropic/claude-opus-4-8
$ ltb bench push

Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs

Scores

4 sourced entries
HarnessSourceConfidenceLink
GPT-362.7N/AIndependentmediumSep 28, 2021arxiv.org
GPT-Neo48.1N/AIndependentmediumSep 28, 2021arxiv.org
GPT-245.8N/AIndependentmediumSep 28, 2021arxiv.org
GPT-3 zero-shot29.2N/AIndependentmediumSep 28, 2021arxiv.org

Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.

Reproduced by the community

All runs
Loading community runs…

About

RAFT: A Real-World Few-Shot Text Classification Benchmark https://arxiv.org/abs/2109.14076

Official website for RAFT dataset: https://raft.elicit.org/

Dataset summary: https://huggingface.co/datasets/ought/raft/blob/main/README.md

Prompts are adapted from: https://github.com/oughtinc/raft-baselines/tree/master/example_prompts

Subsets:

  • ade_corpus_v2
  • banking_77
  • neurips_impact_statement_risks
  • one_stop_english
  • overruling
  • semiconductor_org_types
  • systematic_review_inclusion
  • tai_safety_research
  • terms_of_service
  • tweet_eval_hate
  • twitter_complaints

Prompt format

Sentence: <sentence> Label: <label>

Examples from ADE corpus (adverse drug effect):

Sentence: No regional side effects were noted. Label: not ADE-related