CritPt
Harbor: pendingCritPt — AI evaluation benchmark listed by opencompass.
Verified71%Knowledge & Reasoning
- Samples
- 70
- Released
- 2025
- Scores
- 138
WOVEN
WOVEN: Weaving Visual World Modeling into Multimodal LLMs
A benchmark of 36,076 examples for visual transition reasoning across 20 scenes, 5 actions, and 8 reasoning types using video-generated rollouts.
How to run
ltb · woven
$ npm i -g lettokenburn-cli$ ltb bench info woven
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
13 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| GPT-5.4 | 83.8 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| Intern-S1-Pro | 77.6 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| google/gemma-4-31B-it | 76.3 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| Seed-2.0-Mini | 76.0 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| Qwen3.5 397B A17B | 75.0 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| Qwen2.5-VL-72B-Instruct | 74.7 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| Qwen3-VL-32B | 71.4 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| GPT-5.2-chat | 70.7 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| Qwen-VL-Max | 69.6 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| ERNIE-4.5-VL-424B-A47B | 64.6 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| Qwen3-VL-235B-A22B | 62.7 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| Gemma 3 27B | 62.2 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
| ERNIE-4.5-VL-28B-A3B | 53.7 | N/A | Independent | medium | Oct 8, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsLoading community runs…
About
WOVEN organizes transition supervision by scene, action, and reasoning type using realistic rollouts from video-pretrained generative models. It comprises 36,076 examples with error-typed multiple-choice questions.
Scoring uses accuracy against human baselines. Evaluation of 38 MLLMs reveals substantial deficits, with even strong models falling far below the 92.3% human baseline.