BrainTRACE
Harbor adapterA report-grounded benchmark with 7,273 VQA instances from 1,778 longitudinal patients evaluating vision-language models on brain MRI evidence tracing.
- Samples
- 7,273
- Released
- 2026
- Scores
- 13
PhysicsLENS
PhysicsLENS: Diagnosing Physical Property Blindness in Video Generation Models
A robotics-focused benchmark evaluating video generation models on physical plausibility and grounding of seven hidden properties (collision, gravity, momentum, friction, deformation, fluid, causality) using matched scenario pairs with human annotations.
How to run
ltb · physicslens
$ npm i -g lettokenburn-cli$ ltb bench info physicslens
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
6 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Qwen3-VL-32B | 0.8 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| Qwen3-VL-8B-Instruct | 0.7 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| InternVL3-14B | 0.7 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| Qwen2.5-VL-7B | 0.7 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| Gemma 3 12B IT | 0.7 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
| InternVL3-8B | 0.7 | N/A | Independent | medium | Oct 1, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsLoading community runs…
About
Content & Construction: PhysicsLENS is a dataset comprising curated image-text prompts derived from public robot videos. It constructs matched scenarios where the visual context and task are fixed while varying text-specified physical conditions (observable vs. unobservable). The benchmark covers seven domains: collision, gravity, momentum, friction, deformation, fluid, and causality.
Scoring & Findings: Evaluation involves a human track with over 400 annotations rating plausibility and task completion, alongside an automated VLM-based pipeline. Key findings indicate that plausible-looking videos often ignore stated properties (34 of 47 cases), demonstrating a disconnect between visual realism and physical grounding.