CREATE
Harbor: pendingPrevious works of video captioning aim to objectively describe the video's actual content, which lacks subjective and attractive expression, limiting its practical application scenarios.
- Samples
- 155
- Released
- 2022
- Scores
- 11
LinguaGuardBench
A benchmark evaluating step-level agent guard sensitivity to user request language across 5 languages and 6 English controls using 1,220 labelled steps.
How to run
ltb · linguaguardbench
$ npm i -g lettokenburn-cli$ ltb bench info linguaguardbench
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
13 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Gemini 3.8 Flash | 0.0 | N/A | Independent | medium | Oct 15, 2026 | arxiv.org |
| google/gemma-4-E2B-it | 0.4 | N/A | Independent | medium | Oct 15, 2026 | arxiv.org |
| Safiron 8B | 0.4 | N/A | Independent | medium | Oct 15, 2026 | arxiv.org |
| google/gemma-4-26B-A4B-it | 0.5 | N/A | Independent | medium | Oct 15, 2026 | arxiv.org |
| Qwen3.5 4B | 0.5 | N/A | Independent | medium | Oct 15, 2026 | arxiv.org |
| Qwen3.8 27B | 0.7 | N/A | Independent | medium | Oct 15, 2026 | arxiv.org |
| Gemma-4-12B-it | 1.0 | N/A | Independent | medium | Oct 15, 2026 | arxiv.org |
| Nemotron Nano 9B | 1.2 | N/A | Independent | medium | Oct 15, 2026 | arxiv.org |
| qwen3.5-9B | 1.6 | N/A | Independent | medium | Oct 15, 2026 | arxiv.org |
| ShieldAgent 7B | 1.6 | N/A | Independent | medium | Oct 15, 2026 | arxiv.org |
| Claude Sonnet 5 | 1.7 | N/A | Independent | medium | Oct 15, 2026 | arxiv.org |
| AgentDoG 4B | 2.4 | N/A | Independent | medium | Oct 15, 2026 | arxiv.org |
| Llama 4 Maverick | 2.8 | N/A | Independent | medium | Oct 15, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsLoading community runs…
About
Derived from TS-Bench AgentDojo-Traj, it translates only user instructions into German, French, Turkish, Swahili, and Amharic while keeping tool calls and labels identical. It includes English paraphrase and whitespace controls.
It measures verdict flip rates across 19 guards. Findings show small and purpose-built guards are highly sensitive to language, with error rates increasing for rarer languages, while frontier guards remain robust.