SearchDecision-Bench
A benchmark unifying six types of search decisions (routing, rewriting, relevance, sufficiency, navigation, verification) for training and evaluating search agents.
How to run
$ npm i -g lettokenburn-cli$ ltb bench info searchdecision-bench
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
11 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| kev 8B | 90.3 | N/A | Independent | medium | Oct 4, 2026 | arxiv.org |
| Plumb-4B | 89.6 | N/A | Independent | medium | Oct 4, 2026 | arxiv.org |
| Decision 4B v1.2 | 88.3 | N/A | Independent | medium | Oct 4, 2026 | arxiv.org |
| Cygnet | 87.9 | N/A | Independent | medium | Oct 4, 2026 | arxiv.org |
| Malkuth-4B | 86.8 | N/A | Independent | medium | Oct 4, 2026 | arxiv.org |
| Jev 1.13 | 86.6 | N/A | Independent | medium | Oct 4, 2026 | arxiv.org |
| Imajev-4B | 86.2 | N/A | Independent | medium | Oct 4, 2026 | arxiv.org |
| Winnow-12B | 85.7 | N/A | Independent | medium | Oct 4, 2026 | arxiv.org |
| SearchJev -4B | 80.5 | N/A | Independent | medium | Oct 4, 2026 | arxiv.org |
| Spark S1 4B V6 | 78.7 | N/A | Independent | medium | Oct 4, 2026 | arxiv.org |
| Qwen3.5 4B | 78.4 | N/A | Independent | medium | Oct 4, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsAbout
SearchDecision-Bench converts heterogeneous Chinese and English resources into a common state-schema format covering six decision types. It evaluates decision quality, calibration, and latency using held-out splits.
The benchmark measures performance via decision quality and calibration error. SearchJev improves decision quality and reduces calibration error by 41-74% compared to same-size autoregressive models.