DatedGPT: Preventing Lookahead Bias in Large Language Models with Time-Aware Pretraining

Yutong Yan, Raphael Tang, Zhenyu Gao, Wenxi Jiang and Yao Lu

Papers from arXiv.org

Abstract: Large language models pretrained on internet-scale data risk lookahead bias in forecasting tasks, as they may have already seen the true outcome during training. To address this, we present DatedGPT, a family of twelve 1.3B-parameter language models trained from scratch on approximately 100 billion tokens each with strict annual data cutoffs spanning 2013 to 2024, together with DatedInstruct, an instruction dataset grounded in each year's documents to prevent leakage during post-training. The models are competitive with open models of similar scale, and perplexity-based probing confirms that each model's knowledge is bounded by its cutoff year. On stock return prediction over 61,000 firm-day news headlines, DatedGPT-instruct achieves an annualised Sharpe ratio of $3.20$ under the lookahead-bias-free setup. Lookahead-biased models, whose training data covers the outcome period, add a lookahead premium of $26.4$ b.p. per standard deviation, significant at the 1% level. The series thus enables direct analysis of lookahead bias in financial forecasting. We provide an interactive web demo that allows users to query and compare responses from models across different cutoff years, available at www.datedgpt.com.

Date: 2026-03, Revised 2026-07
New Economics Papers: this item is included in nep-ain, nep-big, nep-cmp and nep-for
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (3)

Downloads: (external link)
https://arxiv.org/pdf/2603.11838 Latest version (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:arx:papers:2603.11838

Access Statistics for this paper

More papers in Papers from arXiv.org
Bibliographic data for series maintained by arXiv administrators ().