A class of adaptive importance sampling weighted EM algorithms for efficient and robust posterior and predictive simulation

Lennart Hoogerheide, Anne Opschoor and Herman van Dijk

Journal of Econometrics, 2012, vol. 171, issue 2, 101-120

Abstract: A class of adaptive sampling methods is introduced for efficient posterior and predictive simulation. The proposed methods are robust in the sense that they can handle target distributions that exhibit non-elliptical shapes such as multimodality and skewness. The basic method makes use of sequences of importance weighted Expectation Maximization steps in order to efficiently construct a mixture of Student-t densities that approximates accurately the target distribution–typically a posterior distribution, of which we only require a kernel–in the sense that the Kullback–Leibler divergence between target and mixture is minimized. We label this approach Mixture oftby Importance Sampling weighted Expectation Maximization (MitISEM). The constructed mixture is used as a candidate density for quick and reliable application of either Importance Sampling (IS) or the Metropolis–Hastings (MH) method. We also introduce three extensions of the basic MitISEM approach. First, we propose a method for applying MitISEM in a sequential manner, so that the candidate distribution for posterior simulation is cleverly updated when new data become available. Our results show that the computational effort reduces enormously, while the quality of the approximation remains almost unchanged. This sequential approach can be combined with a tempering approach, which facilitates the simulation from densities with multiple modes that are far apart. Second, we introduce a permutation-augmented MitISEM approach. This is useful for importance or Metropolis–Hastings sampling from posterior distributions in mixture models without the requirement of imposing identification restrictions on the model’s mixture regimes’ parameters. Third, we propose a partial MitISEM approach, which aims at approximating the joint distribution by estimating a product of marginal and conditional distributions. This division can substantially reduce the dimension of the approximation problem, which facilitates the application of adaptive importance sampling for posterior simulation in more complex models with larger numbers of parameters. Our results indicate that the proposed methods can substantially reduce the computational burden in econometric models like DCC or mixture GARCH models and a mixture instrumental variables model.

Keywords: Mixture of Student-t distributions; Importance sampling; Kullback–Leibler divergence; Expectation Maximization; Metropolis–Hastings algorithm; Predictive likelihood; DCC GARCH; Mixture GARCH; Instrumental variables (search for similar items in EconPapers)
JEL-codes: C11 C15 C22 C26 C58 (search for similar items in EconPapers)
Date: 2012
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (40)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0304407612001583
Full text for ScienceDirect subscribers only

Related works:
Working Paper: A Class of Adaptive Importance Sampling Weighted EM Algorithms for Efficient and Robust Posterior and Predictive Simulation (2012)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:econom:v:171:y:2012:i:2:p:101-120

DOI: 10.1016/j.jeconom.2012.06.011

Access Statistics for this article

Journal of Econometrics is currently edited by T. Amemiya, A. R. Gallant, J. F. Geweke, C. Hsiao and P. M. Robinson

More articles in Journal of Econometrics from Elsevier
Bibliographic data for series maintained by Catherine Liu ().