Entropy concentration and the empirical coding game

Peter Grünwald

Statistica Neerlandica, 2008, vol. 62, issue 3, 374-392

Abstract: We give a characterization of maximum entropy/minimum relative entropy inference by providing two ‘strong entropy concentration’ theorems. These theorems unify and generalize Jaynes’‘concentration phenomenon’ and Van Campenhout and Cover's ‘conditional limit theorem’. The theorems characterize exactly in what sense a prior distribution Q conditioned on a given constraint and the distribution minimizing D(P ‖ Q) over all P satisfying the constraint are ‘close’ to each other. We then apply our theorems to establish the relationship between entropy concentration and a game‐theoretic characterization of maximum entropy inference of Topsøe and others.

Date: 2008
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://doi.org/10.1111/j.1467-9574.2008.00401.x

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:bla:stanee:v:62:y:2008:i:3:p:374-392

Ordering information: This journal article can be ordered from
http://www.blackwell ... bs.asp?ref=0039-0402

Access Statistics for this article

Statistica Neerlandica is currently edited by Miroslav Ristic, Marijtje van Duijn and Nan van Geloven

More articles in Statistica Neerlandica from Netherlands Society for Statistics and Operations Research
Bibliographic data for series maintained by Wiley Content Delivery ().