AlgoREval
Harbor: pendingA benchmark of 599 problems spanning 77 algorithms, 14 domains, 7 languages, and 4 graph representations to evaluate parametric code retrieval in LLMs.
Auto86%Coding
- Samples
- 599
- Released
- 2026
- Scores
- 14
VisualNoiseQA
VisualNoiseQA: An Active Visual Reasoning Benchmark
A benchmark of 1,000 instances evaluating text-only LLMs on active reasoning under noisy visual feedback from VLM sensors.
How to run
ltb · visualnoiseqa
$ npm i -g lettokenburn-cli$ ltb bench info visualnoiseqa
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
10 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Qwen3.5 122B A10B | 54.6 | bp | Independent | medium | Oct 6, 2026 | arxiv.org |
| GPT-5-mini | 47.0 | bp | Independent | medium | Oct 6, 2026 | arxiv.org |
| Qwen2.5-VL-7B | 46.9 | bp | Independent | medium | Oct 6, 2026 | arxiv.org |
| InternVL3.5-8B-Instruct | 46.5 | bp | Independent | medium | Oct 6, 2026 | arxiv.org |
| Qwen2.5-72B-Instruct | 42.8 | bp | Independent | medium | Oct 6, 2026 | arxiv.org |
| gpt-oss-20b | 42.2 | bp | Independent | medium | Oct 6, 2026 | arxiv.org |
| Qwen2.5-7B-Instruct | 37.3 | bp | Independent | medium | Oct 6, 2026 | arxiv.org |
| Llama-3.1-8B-Instruct | 36.4 | react | Independent | medium | Oct 6, 2026 | arxiv.org |
| Qwen3-32B | 36.2 | bp | Independent | medium | Oct 6, 2026 | arxiv.org |
| Qwen3 8B | 35.2 | bp | Independent | medium | Oct 6, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsLoading community runs…
About
VisualNoiseQA evaluates text-only LLMs acting as reasoners that query a noisy VLM sensor. It uses self-consistency scores to expose uncertainty, requiring models to decide what to ask and when to stop.
Constructed automatically from diverse VQA sources, it retains questions where sensors are inconsistent but humans can solve them. Results show LLMs lag behind humans but leverage consistency signals to improve robustness.