Deep Regression Ensembles

Antoine Didisheim, Bryan T. Kelly and Semyon Malamud
Additional contact information
Antoine Didisheim: Swiss Finance Institute, UNIL
Bryan T. Kelly: Yale SOM; AQR Capital Management, LLC; National Bureau of Economic Research (NBER)
Semyon Malamud: Ecole Polytechnique Federale de Lausanne; Centre for Economic Policy Research (CEPR); Swiss Finance Institute

No 22-20, Swiss Finance Institute Research Paper Series from Swiss Finance Institute

Abstract: We introduce a methodology for designing and training deep neural networks (DNN) that we call “Deep Regression Ensembles" (DRE). It bridges the gap between DNN and two-layer neural networks trained with random feature regression. Each layer of DRE has two components, randomly drawn input weights and output weights trained myopically (as if the final output layer) using linear ridge regression. Within a layer, each neuron uses a different subset of inputs and a different ridge penalty, constituting an ensemble of random feature ridge regressions. Our experiments show that a single DRE architecture is at par with or exceeds state-of-the-art DNN in many data sets. Yet, because DRE neural weights are either known in closed-form or randomly drawn, its computational cost is orders of magnitude smaller than DNN.

Keywords: Deep learning; Neural network; Random features; Ensembles (search for similar items in EconPapers)
Pages: 29 pages
Date: 2022-03
New Economics Papers: this item is included in nep-ban, nep-big, nep-cmp, nep-cwa, nep-ecm and nep-ore
References: Add references at CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4049493 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:chf:rpseri:rp2220

Access Statistics for this paper

More papers in Swiss Finance Institute Research Paper Series from Swiss Finance Institute Contact information at EDIRC.
Bibliographic data for series maintained by Ridima Mittal ().