CritPt
Harbor: pendingCritPt — AI evaluation benchmark listed by opencompass.
Verified71%Knowledge & Reasoning
- Samples
- 70
- Released
- 2025
- Scores
- 138
RateAR
RateAR: A Benchmark for Perceptual Quality Assessment and Autonomous Content Adjustment in Augmented Reality
A benchmark of 321 AR images and 112 videos evaluating VLM alignment with human judgments on placement, scale, and shadows.
How to run
ltb · ratear
$ npm i -g lettokenburn-cli$ ltb bench info ratear
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
13 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Gemini 2.5 Pro | 0.8 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| GPT-o3 | 0.8 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| GPT-5 | 0.8 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| Gemini 2.5 Flash | 0.7 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| Gemini-3.5-flash | 0.7 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| LIQE | 0.7 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| QualiCLIP+ | 0.7 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| HyperIQA | 0.7 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| TOPIQ | 0.7 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| Muse Spark 1.1 | 0.6 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| CLIPIQA+ | 0.6 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| LPIPS-VGG+ | 0.6 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| CNNIQA | 0.5 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsLoading community runs…
About
Content & Construction: RateAR is a curated dataset comprising 321 AR images and 112 AR videos captured from real-world renderings. It covers diverse scenes and environmental conditions to assess three key perceptual factors: object placement plausibility, size appropriateness, and shadow consistency.
Scoring & Findings: The benchmark evaluates eleven commercial Vision-Language Models (VLMs) against human subjective ratings using Spearman's rank-order correlation as the primary metric. Results show strong alignment with human judgments (SRCC up to 0.8695), validating VLM-based quality assessment for AR immersion.