LetTokenBurnMOF-Verify: A Failure-Aware Agentic Harness for MOF Hypothesis Verification
A diagnostic benchmark with four task families (structural, synthesis, evidence-sufficiency, computational) for evaluating LLM-based MOF hypothesis verification under controlled evidence settings.
$ npm i -g lettokenburn-cli$ ltb bench info mof-verify
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Claude Opus 4.8 | 62.8 | N/A | Independent | medium | Oct 2, 2026 | arxiv.org |
| Claude Opus 5 | 62.8 | N/A | Independent | medium | Oct 2, 2026 | arxiv.org |
| Claude Sonnet 5 | 62.8 | N/A | Independent | medium | Oct 2, 2026 | arxiv.org |
| DeepSeek V4 Pro | 62.8 | N/A | Independent | medium | Oct 2, 2026 | arxiv.org |
| Gemini 3.7 Flash | 62.8 | N/A | Independent | medium | Oct 2, 2026 | arxiv.org |
| GPT-4o | 62.8 | N/A | Independent | medium | Oct 2, 2026 | arxiv.org |
| GPT-5.4 | 62.8 | N/A | Independent | medium | Oct 2, 2026 | arxiv.org |
| GPT-5.6 Sol | 62.8 | N/A | Independent | medium | Oct 2, 2026 | arxiv.org |
| GPT-5.6-terra | 62.8 | N/A | Independent | medium | Oct 2, 2026 | arxiv.org |
| Qwen3.7 Max | 62.8 | N/A | Independent | medium | Oct 2, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
The benchmark comprises four task families (T-MOF-1 to T-MOF-4) covering structural grounding, synthesis verification, evidence sufficiency, and MLIP-based computation. It uses a 3x3 diagnostic matrix (closed-book, retrieval, oracle) to localize failure modes in AI Co-Scientists.
Scoring involves predicting Yes/No/Uncertain or computational outcomes. The associated MOF-Verify harness significantly improves performance over baselines by addressing identified bottlenecks through specialized agentic modules.