On the distribution of penalized maximum likelihood estimators: The LASSO, SCAD, and thresholding

Benedikt Pötscher and Hannes Leeb ()

Journal of Multivariate Analysis, 2009, vol. 100, issue 9, 2065-2082

Abstract: We study the distributions of the LASSO, SCAD, and thresholding estimators, in finite samples and in the large-sample limit. The asymptotic distributions are derived for both the case where the estimators are tuned to perform consistent model selection and for the case where the estimators are tuned to perform conservative model selection. Our findings complement those of Knight and Fu [K. Knight, W. Fu, Asymptotics for lasso-type estimators, Annals of Statistics 28 (2000) 1356-1378] and Fan and Li [J. Fan, R. Li, Variable selection via non-concave penalized likelihood and its oracle properties, Journal of the American Statistical Association 96 (2001) 1348-1360]. We show that the distributions are typically highly non-normal regardless of how the estimator is tuned, and that this property persists in large samples. The uniform convergence rate of these estimators is also obtained, and is shown to be slower than n-1/2 in case the estimator is tuned to perform consistent model selection. An impossibility result regarding estimation of the estimators' distribution function is also provided.

Keywords: primary; 62J07; 62J05; 62F11; 62F12; 62E15 Penalized maximum likelihood LASSO SCAD Thresholding Post-model-selection estimator Finite-sample distribution Asymptotic distribution Oracle property Estimation of distribution Uniform consistency (search for similar items in EconPapers)
Date: 2009
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (51)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0047-259X(09)00125-0
Full text for ScienceDirect subscribers only

Related works:
Working Paper: On the distribution of penalized maximum likelihood estimators: The LASSO, SCAD, and thresholding (2007)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:jmvana:v:100:y:2009:i:9:p:2065-2082

Ordering information: This journal article can be ordered from
http://www.elsevier.com/wps/find/supportfaq.cws_home/regional
https://shop.elsevie ... _01_ooc_1&version=01

Access Statistics for this article

Journal of Multivariate Analysis is currently edited by de Leeuw, J.

More articles in Journal of Multivariate Analysis from Elsevier
Bibliographic data for series maintained by Catherine Liu ().