A lava attack on the recovery of sums of dense and sparse signals

Victor Chernozhukov, Christian Hansen and Yuan Liao

No 56/15, CeMMAP working papers from Institute for Fiscal Studies

Abstract: Common high-dimensional methods for prediction rely on having either a sparse signal model, a model in which most parameters are zero and there are a small number of non-zero parameters that are large in magnitude, or a dense signal model, a model with no large parameters and very many small non-zero parameters. We consider a generalization of these two basic models, termed here a “sparse + dense” model, in which the signal is given by the sum of a sparse signal and a dense signal. Such a structure poses problems for traditional sparse estimators, such as the lasso, and for traditional dense estimation methods, such as ridge estimation. We propose a new penalization-based method, called lava, which is computationally efficient. With suitable choices of penalty parameters, the proposed method strictly dominates both lasso and ridge. We derive analytic expressions for the finite-sample risk function of the lava estimator in the Gaussian sequence model. We also provide a deviation bound for the prediction risk in the Gaussian regression model with fixed design. In both cases, we provide Stein's unbiased estimator for lava's prediction risk. A simulation example compares the performance of lava to lasso, ridge, and elastic net in a regression example using data-dependent penalty parameters and illustrates lava's improved performance relative to these benchmarks.

Date: 2015-09-22
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.cemmap.ac.uk/wp-content/uploads/2020/08/CWP5615.pdf (application/pdf)

Related works:
Working Paper: A lava attack on the recovery of sums of dense and sparse signals (2015)
Working Paper: A lava attack on the recovery of sums of dense and sparse signals (2015)
Working Paper: A lava attack on the recovery of sums of dense and sparse signals (2015)
Working Paper: A lava attack on the recovery of sums of dense and sparse signals (2015)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:azt:cemmap:56/15

DOI: 10.1920/wp.cem.2015.5615

Access Statistics for this paper

More papers in CeMMAP working papers from Institute for Fiscal Studies Contact information at EDIRC.
Bibliographic data for series maintained by Dermot Watson ().