Monte Carlo Simulation for Lasso-Type Problems by Estimator Augmentation

Qing Zhou

Journal of the American Statistical Association, 2014, vol. 109, issue 508, 1495-1516

Abstract: Regularized linear regression under the ℓ 1 penalty, such as the Lasso, has been shown to be effective in variable selection and sparse modeling. The sampling distribution of an ℓ 1 -penalized estimator is hard to determine as the estimator is defined by an optimization problem that in general can only be solved numerically and many of its components may be exactly zero. Let S be the subgradient of the ℓ 1 norm of the coefficient vector β evaluated at . We find that the joint sampling distribution of and S , together called an augmented estimator, is much more tractable and has a closed-form density under a normal error distribution in both low-dimensional ( p ⩽ n ) and high-dimensional ( p > n ) settings. Given β and the error variance σ-super-2, one may employ standard Monte Carlo methods, such as Markov chain Monte Carlo (MCMC) and importance sampling (IS), to draw samples from the distribution of the augmented estimator and calculate expectations with respect to the sampling distribution of . We develop a few concrete Monte Carlo algorithms and demonstrate with numerical examples that our approach may offer huge advantages and great flexibility in studying sampling distributions in ℓ 1 -penalized linear regression. We also establish nonasymptotic bounds on the difference between the true sampling distribution of and its estimator obtained by plugging in estimated parameters, which justifies the validity of Monte Carlo simulation from an estimated sampling distribution even when p >> n → ∞.

Date: 2014
References: Add references at CitEc
Citations: View citations in EconPapers (3)

Downloads: (external link)
http://hdl.handle.net/10.1080/01621459.2014.946035 (text/html)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:taf:jnlasa:v:109:y:2014:i:508:p:1495-1516

Ordering information: This journal article can be ordered from
http://www.tandfonline.com/pricing/journal/UASA20

DOI: 10.1080/01621459.2014.946035

Access Statistics for this article

Journal of the American Statistical Association is currently edited by Xuming He, Jun Liu, Joseph Ibrahim and Alyson Wilson

More articles in Journal of the American Statistical Association from Taylor & Francis Journals
Bibliographic data for series maintained by Chris Longhurst ().