Explaining the behavior of joint and marginal Monte Carlo estimators in latent variable models with independence assumptions

Silia Vitoratou, Ioannis Ntzoufras and Irini Moustaki

LSE Research Online Documents on Economics from London School of Economics and Political Science, LSE Library

Abstract: In latent variable models parameter estimation can be implemented by using the joint or the marginal likelihood, based on independence or conditional independence assumptions. The same dilemma occurs within the Bayesian framework with respect to the estimation of the Bayesian marginal (or integrated) likelihood, which is the main tool for model comparison and averaging. In most cases, the Bayesian marginal likelihood is a high dimensional integral that cannot be computed analytically and a plethora of methods based on Monte Carlo integration (MCI) are used for its estimation. In this work, it is shown that the joint MCI approach makes subtle use of the properties of the adopted model, leading to increased error and bias in finite settings. The sources and the components of the error associated with estimators under the two approaches are identified here and provided in exact forms. Additionally, the effect of the sample covariation on the Monte Carlo estimators is examined. In particular, even under independence assumptions the sample covariance will be close to (but not exactly) zero which surprisingly has a severe effect on the estimated values and their variability. To address this problem, an index of the sample's divergence from independence is introduced as a multivariate extension of covariance. The implications addressed here are important in the majority of practical problems appearing in Bayesian inference of multi-parameter models with analogous structures.

Keywords: Bayes factor; marginal likelihood; Monte Carlo integration (search for similar items in EconPapers)
JEL-codes: C1 (search for similar items in EconPapers)
Date: 2016-01-01
New Economics Papers: this item is included in nep-ecm and nep-ore
References: View references in EconPapers View complete reference list from CitEc
Citations:

Published in Statistics and Computing, 1, January, 2016, 26(1), pp. 333-348. ISSN: 0960-3174

Downloads: (external link)
http://eprints.lse.ac.uk/57685/ Open access version. (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:ehl:lserod:57685

Access Statistics for this paper

More papers in LSE Research Online Documents on Economics from London School of Economics and Political Science, LSE Library LSE Library Portugal Street London, WC2A 2HD, U.K.. Contact information at EDIRC.
Bibliographic data for series maintained by LSERO Manager ().