On efficient calculations for Bayesian variable selection

Eric Ruggieri and Charles E. Lawrence

Computational Statistics & Data Analysis, 2012, vol. 56, issue 6, 1319-1332

Abstract: We describe an efficient, exact Bayesian algorithm applicable to both variable selection and model averaging problems. A fully Bayesian approach provides a more complete characterization of the posterior ensemble of possible sub-models, but presents a computational challenge as the number of candidate variables increases. While several approximation techniques have been developed to deal with problems that contain a large numbers of candidate variables, including BMA, IBMA, MCMC and Gibbs Sampling approaches, here we focus on improving the time complexity of exact inference using a recursive algorithm (Exact Bayesian Inference in Regression, or EBIR) that uses components of one sub-model to rapidly generate another and prove that its time complexity is O(m2), where m is the number candidate variables. Testing against simulated data shows that EBIR significantly reduces compute time without sacrificing accuracy, while comparisons to the results obtained by MCMC approaches on the Crime and Punishment data set show that model averaging yields improved predictive performance over two model selection approaches. In addition, we show that finite mixtures of centroid solutions provide a means to better characterize the shape of multimodal posterior spaces than any individual model. Finally, we describe how the BIC approximations employed in the BMA and IBMA algorithms can be replaced by an EBIR calculation of equal time complexity and illustrate the departure of the BIC approximation from the exact Bayesian inference of EBIR.

Keywords: Bayesian model averaging; Variable selection; Dynamic programming; Inversion of matrix sums; Regression; Spike and slab (search for similar items in EconPapers)
Date: 2012
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0167947311003574
Full text for ScienceDirect subscribers only.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:csdana:v:56:y:2012:i:6:p:1319-1332

DOI: 10.1016/j.csda.2011.09.026

Access Statistics for this article

Computational Statistics & Data Analysis is currently edited by S.P. Azen

More articles in Computational Statistics & Data Analysis from Elsevier
Bibliographic data for series maintained by Catherine Liu ().