Market Efficiency in the Age of Big Data

Ian Martin and Stefan Nagel

No 8015, CESifo Working Paper Series from CESifo

Abstract: Modern investors face a high-dimensional prediction problem: thousands of observable variables are potentially relevant for forecasting. We reassess the conventional wisdom on market efficiency in light of this fact. In our model economy, which resembles a typical machine learning setting, N assets have cash flows that are a linear function of J firm characteristics, but with uncertain coefficients. Risk-neutral Bayesian investors impose shrinkage (ridge regression) or sparsity (Lasso) when they estimate the J coefficients of the model and use them to price assets. When J is comparable in size to N, returns appear cross-sectionally predictable using firm characteristics to an econometrician who analyzes data from the economy ex post. A factor zoo emerges even without p-hacking and data-mining. Standard in-sample tests of market efficiency reject the no-predictability null with high probability, despite the fact that investors optimally use the information available to them in real time. In contrast, out-of-sample tests retain their economic meaning.

Keywords: Bayesian learning; high-dimensional prediction problems; return predictability; out-of-sample tests (search for similar items in EconPapers)
JEL-codes: C11 G12 G14 (search for similar items in EconPapers)
Date: 2019
New Economics Papers: this item is included in nep-big
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (4)

Downloads: (external link)
https://www.cesifo.org/DocDL/cesifo1_wp8015.pdf (application/pdf)

Related works:
Journal Article: Market efficiency in the age of big data (2022)
Working Paper: Market efficiency in the age of big data (2022)
Working Paper: Market Efficiency in the Age of Big Data (2019)
Working Paper: Market Efficiency in the Age of Big Data (2019)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:ces:ceswps:_8015

Access Statistics for this paper

More papers in CESifo Working Paper Series from CESifo Contact information at EDIRC.
Bibliographic data for series maintained by Klaus Wohlrabe (wohlrabe@ifo.de).