ExceptionDrive
ExceptionDrive: A Planning-Oriented Counterfactual Corner-Case Benchmark for Autonomous Driving
A counterfactual planning benchmark inserting localized hazards into real nuScenes scenes to evaluate autonomous driving planners on safety-critical corner cases using metrics like Unsafe Rate and Hazard Clearance Compliance.
How to run
$ npm i -g lettokenburn-cli$ ltb bench info exceptiondrive
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
2 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Qwen2.5VL-7B + Reminder | 29.2 | N/A | Independent | medium | Sep 29, 2026 | arxiv.org |
| Qwen2.5VL-7B | 45.5 | N/A | Independent | medium | Sep 29, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsAbout
Content & Construction: ExceptionDrive is a planning-oriented counterfactual benchmark built by inserting localized, multi-view edited hazards (e.g., occluded pedestrians) into real-world nuScenes observations while preserving context. It utilizes VLM-assisted screening to select scenes and includes quality auditing via review assistants.
Scoring & Key Findings: The framework evaluates 21 tasks across six safety families without requiring expert reference trajectories for edited scenes. Primary metrics include Unsafe Rate (UR) and Hazard Clearance Compliance (HCC). Experiments revealed that representative planners frequently intrude into hazard regions or fail to provide sufficient clearance.