GIFT-Eval-CTX
GIFT-Eval-CTX: A Benchmark For Contextual Forecasting Model Evaluation
A benchmark with 10,000 instances across 20 datasets evaluating contextual forecasting by measuring how models translate structured future effects into accurate numerical time series trajectories.
How to run
$ npm i -g lettokenburn-cli$ ltb bench info gift-eval-ctx
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
7 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| MoiraiAgent | 0.5 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
| GPT-5.6 Sol | 0.6 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
| TimesFM-3 | 0.8 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
| Claude Opus 4.8 | 0.8 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
| Moirai-2 | 0.8 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
| Chronos-2 | 0.8 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
| Toto-2 | 0.8 | N/A | Independent | medium | Oct 13, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsAbout
GIFT-Eval-CTX extends GIFT-Eval with 10,000 instances across 20 domains, pairing time series history with structured 'effects' (Macro/Micro) derived from realized futures via an agentic pipeline. It evaluates the ability to convert verified forward-looking context into numerical forecasts.
Scoring uses relative MASE to compare history-only versus contextual prompting. Frontier LLMs show significant improvement with structured context, while the MoiraiAgent system achieves the best overall performance by combining LLMs with code-execution tools.