SABER
Synthetic Attribute Binding Evaluation for Representations
SABER: Benchmarking Operational Safety of LLM Coding Agents in Stateful Project Workspaces SABER is the code release for the paper.
How to run
$ npm i -g lettokenburn-cli$ ltb bench info saber
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
8 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| PE-Core-bigG-14-448 | 41.5 | N/A | Independent | medium | Oct 15, 2026 | arxiv.org |
| LABCLIP ViT-B/32 | 38.1 | N/A | Independent | medium | Oct 15, 2026 | arxiv.org |
| CLIP ViT-B/32 | 38.0 | N/A | Independent | medium | Oct 15, 2026 | arxiv.org |
| Qwen3-VL-Embedding-8B | 37.8 | N/A | Independent | medium | Oct 15, 2026 | arxiv.org |
| READ-CLIP ViT-B/32 | 37.7 | N/A | Independent | medium | Oct 15, 2026 | arxiv.org |
| LamRA-Ret | 37.3 | N/A | Independent | medium | Oct 15, 2026 | arxiv.org |
| TripletCLIP ViT-B/32 | 35.8 | N/A | Independent | medium | Oct 15, 2026 | arxiv.org |
| Qwen3-VL-Embedding-2B | 34.0 | N/A | Independent | medium | Oct 15, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsAbout
SABER: Benchmarking Operational Safety of LLM Coding Agents in Stateful Project Workspaces
SABER is the code release for the paper. It includes the benchmark tasks, sandbox runtime, judging pipeline, and baseline reproduction utilities used to evaluate operational safety in stateful project workspaces.
What is included
tasks/: benchmark task definitions and metadata run_osbench.py, judge_osbench.py: historical inference and judging entry points sandbox_shell.py…