EgoTools-Bench
Harbor: pendingA diagnostic benchmark of 1,000 QA pairs across four tracks evaluating tool-use understanding from perception and geometry to procedure and causal reasoning.
- Samples
- 1,000
- Released
- 2026
- Scores
- 9
RepProbe
RepProbe: Can Video Language Models Count Repeated Actions? A Diagnostic Benchmark
A diagnostic benchmark of 42K real and synthetic video clips with exact repetition counts (1-20) to evaluate if Video Language Models can count repeated actions.
How to run
ltb · repprobe
$ npm i -g lettokenburn-cli$ ltb bench info repprobe
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
7 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Cosmos-Reason2-8B | 17.5 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
| Qwen3-VL-8B-Instruct | 12.9 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
| InternVL3.5-8B-Instruct | 7.3 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
| Video-R1-7B | 7.2 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
| VideoChat-R1.5-7B | 7.0 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
| LongVILA-R1-7B | 6.5 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
| LLaVA-OneVision-7B | 4.8 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsLoading community runs…
About
RepProbe contains 32K real clips from RepCount and 10K synthetic clips, controlling for duration and position. It evaluates VLMs on counting action repetitions.
Scoring uses accuracy (specifically OBO). Findings show models fail at high counts due to output readout issues, not representation deficits.