Learning multiple belief propagation fixed points for real time inference

Cyril Furtlehner, Jean-Marc Lasgouttes and Anne Auger

Physica A: Statistical Mechanics and its Applications, 2010, vol. 389, issue 1, 149-163

Abstract: In the context of inference with expectation constraints, we propose an approach based on the “loopy belief propagation” algorithm (lpb), as a surrogate to an exact Markov Random Field (mrf) modelling. A prior information composed of correlations among a large set of N variables, is encoded into a graphical model; this encoding is optimized with respect to an approximate decoding procedure (lbp), which is used to infer hidden variables from an observed subset. We focus on the situation where the underlying data have many different statistical components, representing a variety of independent patterns. Considering a single parameter family of models we show how lpb may be used to encode and decode efficiently such information, without solving the NP-hard inverse problem yielding the optimal mrf. Contrary to usual practice, we work in the non-convex Bethe free energy minimization framework, and manage to associate a belief propagation fixed point to each component of the underlying probabilistic mixture. The mean field limit is considered and yields an exact connection with the Hopfield model at finite temperature and steady state, when the number of mixture components is proportional to the number of variables. In addition, we provide an enhanced learning procedure, based on a straightforward multi-parameter extension of the model in conjunction with an effective continuous optimization procedure. This is performed using the stochastic search heuristic cmaes and yields a significant improvement with respect to the single parameter basic model.

Keywords: Inference; Belief propagation; Bethe free energy; Mean field theory; Markov random field; Hopfield model; Continuous optimization (search for similar items in EconPapers)
Date: 2010
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0378437109007158
Full text for ScienceDirect subscribers only. Journal offers the option of making the article available online on Science direct for a fee of $3,000

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:phsmap:v:389:y:2010:i:1:p:149-163

DOI: 10.1016/j.physa.2009.08.030

Access Statistics for this article

Physica A: Statistical Mechanics and its Applications is currently edited by K. A. Dawson, J. O. Indekeu, H.E. Stanley and C. Tsallis

More articles in Physica A: Statistical Mechanics and its Applications from Elsevier
Bibliographic data for series maintained by Catherine Liu ().