RT-Safe
RT-Safe: Benchmarking Agent Safety in Real-Time Embodied Environment
A simulated urban benchmark with 36 routes across 5 maps evaluating embodied agent safety under real-time constraints using VLMs.
How to run
$ npm i -g lettokenburn-cli$ ltb bench info rt-safe
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
8 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Sonnet | 2.8 | N/A | Independent | medium | Oct 7, 2026 | arxiv.org |
| Sol | 2.8 | N/A | Independent | medium | Oct 7, 2026 | arxiv.org |
| Astra | 0.0 | N/A | Independent | medium | Oct 7, 2026 | arxiv.org |
| DeepSeek Chat | 0.0 | N/A | Independent | medium | Oct 7, 2026 | arxiv.org |
| Fable | 0.0 | N/A | Independent | medium | Oct 7, 2026 | arxiv.org |
| Gemini | 0.0 | N/A | Independent | medium | Oct 7, 2026 | arxiv.org |
| Grok | 0.0 | N/A | Independent | medium | Oct 7, 2026 | arxiv.org |
| Inkling | 0.0 | N/A | Independent | medium | Oct 7, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsAbout
RT-Safe combines navigation tasks with moving actors, hazards, and traffic rules in Unreal Engine 5, allowing the world to evolve during inference. It records collisions and violations to evaluate safety, revealing that real-time execution drastically increases collision rates compared to static settings.