LetTokenBurnSUITE: rephrasings This is the robustness companion to apeleg/SUITE, the benchmark introduced in the paper Forget Narrowly, Retain Broadly: Unlearning as an Asymmetric Generalization Problem (see Citation).
$ npm i -g lettokenburn-cli$ ltb bench pull suite-rephrasings --n 50$ ltb bench run ./suite-rephrasings -a claude-code -m anthropic/claude-opus-4-8$ ltb bench push
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
No sourced scores recorded yet.
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
SUITE: rephrasings
This is the robustness companion to apeleg/SUITE, the benchmark introduced in the paper Forget Narrowly, Retain Broadly: Unlearning as an Asymmetric Generalization Problem (see Citation). It holds the SUITE forget-evaluation questions together with the paraphrased rewordings of each one, produced by a held-out generator (Gemini) that is not used to build the training augmentations. It is kept as a separate repo because its schema differs from the core…