R Bench
Harbor: pendingR Bench — AI evaluation benchmark listed by opencompass.
- Samples
- 3,518
- Released
- 2026
- Scores
- 0
R-GroundBench
R-GroundBench: A Diagnostic Benchmark for R-Group Grounding in Markush Molecular Editing
A diagnostic benchmark evaluating multimodal AI models on grounding variable placeholders (R-groups) and performing molecular editing within pharmaceutical patent Markush structures.
How to run
ltb · r-groundbench
$ npm i -g lettokenburn-cli$ ltb bench info r-groundbench
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
14 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| GPT-5.5 | 73.5 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| Gemini 2.5 Flash | 73.4 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| Qwen3-VL-32B | 71.9 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| ChemVLM-8B | 71.6 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| DeepSeek V4 Pro | 71.0 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| GPT-4.1 | 70.8 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| Claude Sonnet 4.6 | 70.5 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| GPT-4o | 69.3 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| Qwen3 Max (2025-09-23) | 67.7 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| Llama 4 Maverick | 67.5 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| o3 | 67.5 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| DeepSeek R1 | 66.0 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| Qwen3-VL-8B-Instruct | 64.9 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
| Llama-3.3-70B-Instruct | 61.1 | N/A | Independent | medium | Sep 30, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsLoading community runs…
About
Content & Construction: Built from real patent records sourced from MolParser-7M, the dataset contains aligned molecular images, E-SMILES representations, natural-language instructions, edited structures, and annotations. It filters 46,727 records into 14,394 clean samples across ~56k instruction instances.
Scoring & Findings: The benchmark features a Multiple-Choice (VQA) track with controlled difficulty splits (Easy/Medium/Hard) to diagnose shortcut reasoning and an open-ended Generation track. Results reveal models achieve >90% accuracy on easy tasks but drop significantly on hard grounding, while generation exact match remains below 20%, highlighting gaps in autonomous molecular instantiation.