ARC-AGI-2
Harbor nativeARC-AGI-2: 更难的抽象推理
Verified100%Abstract Reasoning
- Samples
- 167
- Released
- 2025
- Scores
- 120
NLData2Opt
NLData2Opt: Benchmarking Parameter Derivation and Prediction for Modeling Optimization Problems from Natural Language and Data
A benchmark evaluating LLMs on formulating optimization problems (LP/MILP) from natural language and noisy CSV data, requiring parameter extraction, derivation, and prediction.
How to run
ltb · nldata2opt
$ npm i -g lettokenburn-cli$ ltb bench info nldata2opt
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
10 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| Claude Haiku 4.5 | 0.6 | N/A | Independent | medium | Oct 16, 2026 | arxiv.org |
| Claude Sonnet 4.6 | 0.6 | N/A | Independent | medium | Oct 16, 2026 | arxiv.org |
| Claude Opus 4.6 | 0.6 | N/A | Independent | medium | Oct 16, 2026 | arxiv.org |
| GPT-4o | 0.4 | N/A | Independent | medium | Oct 16, 2026 | arxiv.org |
| Mistral Large 2411 | 0.3 | N/A | Independent | medium | Oct 16, 2026 | arxiv.org |
| Llama-3.1-8B-Instruct | 0.3 | N/A | Independent | medium | Oct 16, 2026 | arxiv.org |
| Llama-3.3-70B-Instruct | 0.3 | N/A | Independent | medium | Oct 16, 2026 | arxiv.org |
| Ministral-3-8B-2512 | 0.1 | N/A | Independent | medium | Oct 16, 2026 | arxiv.org |
| Gemini 2.5 Pro | 0.0 | N/A | Independent | medium | Oct 16, 2026 | arxiv.org |
| GPT-4o-mini | 0.0 | N/A | Independent | medium | Oct 16, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsLoading community runs…
About
NLData2Opt evaluates LLMs on multi-stage reasoning for optimization modeling, separating problem descriptions from noisy CSV data. It requires models to extract relevant parameters, derive missing values, and predict future data.
The benchmark uses AMPL formulation as the output format. Results indicate substantial performance declines compared to existing benchmarks, highlighting gaps in handling real-world data imperfections.