Market Efficiency in the Age of Big Data

Ian Martin and Stefan Nagel

No 14235, CEPR Discussion Papers from C.E.P.R. Discussion Papers

Abstract: Modern investors face a high-dimensional prediction problem: thousands of observable variables are potentially relevant for forecasting. We reassess the conventional wisdom on market efficiency in light of this fact. In our model economy, which resembles a typical machine learning setting, N assets have cash flows that are a linear function of J firm characteristics, but with uncertain coefficients. Risk-neutral Bayesian investors impose shrinkage (ridge regression) or sparsity (Lasso) when they estimate the J coefficients of the model and use them to price assets. When J is comparable in size to N, returns appear cross-sectionally predictable using firm characteristics to an econometrician who analyzes data from the economy ex post. A factor zoo emerges even without p-hacking and data-mining. Standard in-sample tests of market efficiency reject the no-predictability null with high probability, despite the fact that investors optimally use the information available to them in real time. In contrast, out-of-sample tests retain their economic meaning.

Keywords: Market efficiency; Big data; Machine learning (search for similar items in EconPapers)
JEL-codes: C11 C12 C58 G10 G12 G14 (search for similar items in EconPapers)
Date: 2019-12
New Economics Papers: this item is included in nep-big, nep-cmp, nep-fmk and nep-ore
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (5)

Downloads: (external link)
https://cepr.org/publications/DP14235 (application/pdf)
CEPR Discussion Papers are free to download for our researchers, subscribers and members. If you fall into one of these categories but have trouble downloading our papers, please contact us at subscribers@cepr.org

Related works:
Journal Article: Market efficiency in the age of big data (2022)
Working Paper: Market efficiency in the age of big data (2022)
Working Paper: Market Efficiency in the Age of Big Data (2019)
Working Paper: Market Efficiency in the Age of Big Data (2019)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:cpr:ceprdp:14235

Ordering information: This working paper can be ordered from
https://cepr.org/publications/DP14235

Access Statistics for this paper

More papers in CEPR Discussion Papers from C.E.P.R. Discussion Papers Centre for Economic Policy Research, 33 Great Sutton Street, London EC1V 0DX.
Bibliographic data for series maintained by ().