LetTokenBurnThis scenario is based on the manually-curated instructions from the self-instruct paper: https://arxiv.org/pdf/2212.10560.pdf Note that we are not using the self-instruct method here, just the manual data.
$ npm i -g lettokenburn-cli$ ltb bench pull self-instruct
External harness. pip install crfm-helm && helm-run --run-entries self-instruct:model=<model> --suite my-suite --max-eval-instances 100 (HELM scenario 'self-instruct'; some scenarios need extra args) https://github.com/stanford-crfm/helm
License is link-only: ltb bench pull fetches rows from the original host for local evaluation only and does not write a solution/ directory. Do not redistribute the generated tasks.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
No sourced scores recorded yet.
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
This scenario is based on the manually-curated instructions from the self-instruct paper:
https://arxiv.org/pdf/2212.10560.pdf
Note that we are not using the self-instruct method here, just the manual data.