@BENCH
AuthorizationPairs
As Vision-Language Models (VLMs) advance, human-centered Assistive Technologies (ATs) for helping People with Visual Impairments (PVIs) are evolving into generalists, capable of performing multiple tasks simultaneously.
How to run
$ npm i -g lettokenburn-cli$ ltb bench info bench
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
8 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| GPT-OSS Safeguard | 80.0 | N/A | Independent | medium | Oct 12, 2026 | arxiv.org |
| qwen3.5-27B | 80.0 | N/A | Independent | medium | Oct 12, 2026 | arxiv.org |
| Qwen3Guard | 80.0 | N/A | Independent | medium | Oct 12, 2026 | arxiv.org |
| Nemotron | 76.0 | N/A | Independent | medium | Oct 12, 2026 | arxiv.org |
| NeMo self-check | 72.0 | N/A | Independent | medium | Oct 12, 2026 | arxiv.org |
| Granite Guardian | 69.0 | N/A | Independent | medium | Oct 12, 2026 | arxiv.org |
| qwen3.5-9B | 65.0 | N/A | Independent | medium | Oct 12, 2026 | arxiv.org |
| PolicyGuard | 46.0 | N/A | Independent | medium | Oct 12, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsAbout
As Vision-Language Models (VLMs) advance, human-centered Assistive Technologies (ATs) for helping People with Visual Impairments (PVIs) are evolving into generalists, capable of performing multiple tasks simultaneously. However, benchmarking VLMs for ATs remains under-explored. To bridge this gap, we first create a novel AT benchmark (@ Bench). Guided by a pre-design user study with PVIs, our benchmark includes the five most crucial vision-language tasks: Panoptic Segmentation, Depth Estimation, Optical Character Recognition (OCR), Image Captioning, and Visual Question Answering (VQA). Besides, we propose a novel AT model (@MODEL) that addresses all tasks simultaneously and can be expanded to more assistive functions for helping PVIs. Our framework exhibits outstanding performance across tasks by integrating multi-modal information, and it offers PVIs a more comprehensive assistance. Extensive experiments prove the effectiveness and generalizability of our framework.