TPBench
TPBench: A Turning-Point Benchmark for Dialogue Compression
Evaluates dialogue compression methods on recovering initial goals, current slot values, and both, using MultiWOZ and SGD datasets with deterministic probes.
How to run
$ npm i -g lettokenburn-cli$ ltb bench info tpbench
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
3 sourced entriesEvery number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsAbout
TPBench evaluates compression by testing recovery of initial user goals (P1), revised slot values (P2), and both (P3) using annotations from MultiWOZ and SGD. It employs deterministic probe construction without new crowdsourcing.
Scoring uses reader LLMs to answer probe questions from compressed contexts. Results show that no single compression method excels across all probes, with joint recovery significantly lagging behind full context performance.