A lava attack on the recovery of sums of dense and sparse signals

Victor Chernozhukov, Christian Hansen and Yuan Liao
Additional contact information
Christian Hansen: Institute for Fiscal Studies and Chicago GSB

No CWP05/15, CeMMAP working papers from Centre for Microdata Methods and Practice, Institute for Fiscal Studies

Abstract: Common high-dimensional methods for prediction rely on having either a sparse signal model, a model in which most parameters are zero and there are a small number of non-zero parameters that are large in magnitude, or a dense signal model, a model with no large parameters and very many small non-zero parameters. We consider a generalization of these two basic models, termed here a “sparse+dense” model, in which the signal is given by the sum of a sparse signal and a dense signal. Such a structure poses problems for traditional sparse estimators, such as the lasso, and for traditional dense estimation methods, such as ridge estimation. We propose a new penalization-based method, called lava, which is computationally efficient. With suitable choices of penalty parameters, the proposed method strictly dominates both lasso and ridge. We derive analytic expressions for the finite-sample risk function of the lava estimator in the Gaussian sequence model. We also provide a deviation bound for the prediction risk in the Gaussian regression model with fixed design. In both cases, we provide Stein’s unbiased estimator for lava’s prediction risk. A simulation example compares the performance of lava to lasso, ridge, and elastic net in a regression example using feasible, data-dependent penalty parameters and illustrates lava’s improved performance relative to these benchmarks.

Keywords: High-dimensional models; penalization; shrinkage; non-sparse signal recovery (search for similar items in EconPapers)
Date: 2015-02-13
New Economics Papers: this item is included in nep-ecm
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (4)

Downloads: (external link)
https://www.ifs.org.uk/uploads/cemmap/wps/cwp051515.pdf (application/pdf)
Our link check indicates that this URL is bad, the error code is: 404 Not Found (https://www.ifs.org.uk/uploads/cemmap/wps/cwp051515.pdf [302 Found]--> https://ifs.org.uk/uploads/cemmap/wps/cwp051515.pdf)

Related works:
Working Paper: A lava attack on the recovery of sums of dense and sparse signals (2015)
Working Paper: A lava attack on the recovery of sums of dense and sparse signals (2015)
Working Paper: A lava attack on the recovery of sums of dense and sparse signals (2015)
Working Paper: A lava attack on the recovery of sums of dense and sparse signals (2015)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:ifs:cemmap:05/15

Ordering information: This working paper can be ordered from
The Institute for Fiscal Studies 7 Ridgmount Street LONDON WC1E 7AE

Access Statistics for this paper

More papers in CeMMAP working papers from Centre for Microdata Methods and Practice, Institute for Fiscal Studies The Institute for Fiscal Studies 7 Ridgmount Street LONDON WC1E 7AE. Contact information at EDIRC.
Bibliographic data for series maintained by Emma Hyman ().