Bayesian empirical likelihood for ridge and lasso regressions

Adel Bedoui and Nicole A. Lazar

Computational Statistics & Data Analysis, 2020, vol. 145, issue C

Abstract: Ridge and lasso regression models, which are also known as regularization methods, are widely used methods in machine learning and inverse problems that introduce additional information to solve ill-posed problems and/or perform feature selection. The ridge and lasso estimates for linear regression parameters can be interpreted as Bayesian posterior estimates when the regression parameters have Normal and independent Laplace (i.e., double-exponential) priors, respectively. A significant challenge in regularization problems is that these approaches assume that data are normally distributed, which makes them not robust to model misspecification. A Bayesian approach for ridge and lasso models based on empirical likelihood is proposed. This method is semiparametric because it combines a nonparametric model and a parametric model. Hence, problems with model misspecification are avoided. Under the Bayesian empirical likelihood approach, the resulting posterior distribution lacks a closed form and has a nonconvex support, which makes the implementation of traditional Markov chain Monte Carlo (MCMC) methods such as Gibbs sampling and Metropolis–Hastings very challenging. To solve the nonconvex optimization and nonconvergence problems, the tailored Metropolis–Hastings approach is implemented. The asymptotic Bayesian credible intervals are derived.

Keywords: Bayesian inference; Empirical likelihood; Ridge regression; Lasso regression; MCMC (search for similar items in EconPapers)
Date: 2020
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (4)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0167947320300086
Full text for ScienceDirect subscribers only.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:csdana:v:145:y:2020:i:c:s0167947320300086

DOI: 10.1016/j.csda.2020.106917

Access Statistics for this article

Computational Statistics & Data Analysis is currently edited by S.P. Azen

More articles in Computational Statistics & Data Analysis from Elsevier
Bibliographic data for series maintained by Catherine Liu ().