LLM4Impact Dataset
LLM4Impact Benchmark for Scientific Impact Prediction
A large-scale benchmark with 2 million papers, leakage-safe heterogeneous ego graphs, and temporal splits for predicting year-level and month-level citation counts.
How to run
$ npm i -g lettokenburn-cli$ ltb bench info llm4impact-dataset
External harness. No Harbor adapter has been generated for this benchmark yet. `ltb bench info` prints the known links.
Runs happen on your machine with your own subscription; only the score summary is pushed. CLI docs
Scores
8 sourced entries| Harness | Source | Confidence | Link | |||
|---|---|---|---|---|---|---|
| NAIP | 0.8 | N/A | Independent | medium | Oct 7, 2026 | arxiv.org |
| H2CGL | 0.8 | N/A | Independent | medium | Oct 7, 2026 | arxiv.org |
| SciBERT | 0.8 | N/A | Independent | medium | Oct 7, 2026 | arxiv.org |
| DPPDCC | 0.8 | N/A | Independent | medium | Oct 7, 2026 | arxiv.org |
| HINTS | 0.9 | N/A | Independent | medium | Oct 7, 2026 | arxiv.org |
| Qwen3.5 4B | 1.6 | zero-shot | Independent | medium | Oct 7, 2026 | arxiv.org |
| Qwen-3-0.6B | 1.9 | zero-shot | Independent | medium | Oct 7, 2026 | arxiv.org |
| qwen3.5-9B | 2.0 | zero-shot | Independent | medium | Oct 7, 2026 | arxiv.org |
Every number links to where it was published. Low-confidence rows are muted and need per-cell verification.
Reproduced by the community
All runsAbout
Constructed from Semantic Scholar (2005-2020), featuring 2 million query papers with point-in-time heterogeneous ego graphs to prevent data leakage. It supports both in-distribution and temporally shifted test sets.
Evaluation focuses on RMSE for year-level and month-level citation prediction. The proposed LLM4Impact method achieves a 10.13% reduction in year RMSE on the in-distribution test set compared to baselines.