More Mentions, Less Information? Diagnosing Identity and Alignment in Reddit Embeddings for Volatility Forecasting

Zhiyu Zheng

Papers from arXiv.org

Abstract: Since the 2021 GameStop short squeeze, r/WallStreetBets (WSB) has been associated with episodes of extreme volatility, and its text is increasingly used to forecast volatility. A model using only large language model (LLM) embeddings of WSB posts attains an out-of-sample $R^2$ of 0.13 for log realized variance. That number shows predictive power, but it does not reveal where the information comes from or how much the text adds beyond a firm's own volatility history. We answer these questions with a diagnostic protocol that nests embeddings on a heterogeneous autoregressive (HAR) benchmark, compares them with text-free firm-identity controls, shuffles the day-to-text pairing within firm-years to test day-specific alignment, and checks whether gains persist across periods and coverage groups. We apply the protocol to frozen LLM embeddings in a panel of 202,145 mentioned stock-days, using retrospective rolling out-of-sample tests over 2015-2022. The results indicate that most of the standalone fit reflects firm or firm-year information: on firms seen in training, a text-free firm baseline reaches $R^2 = 0.32$, shuffling preserves 89-92% of the standalone fit, and masking ticker strings reduces standalone $R^2$ to -0.005. Adding embeddings to HAR reduces equal-date MSE by 1.22% (95% CI [0.94, 1.49]). This small but statistically significant gain survives firm controls and ticker masking, but is largely eliminated by alignment shuffling. The gain is concentrated in 2015-2019 and is not distinguishable from zero in 2020-2022. Expanding coverage by 48% through off-hours and weekend posts adds stock-days with no reliable positive increment. More mentions therefore do not imply more incremental forecasting information.

Date: 2026-10
New Economics Papers: this item is included in nep-rmg
References: Add references at CitEc
Citations:

Downloads: (external link)
https://arxiv.org/pdf/2610.25468 Latest version (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:arx:papers:2610.25468

Access Statistics for this paper

More papers in Papers from arXiv.org
Bibliographic data for series maintained by arXiv administrators ().