Bayes Estimate and Inference for Entropy and Information Index of Fit

Refik Soyer, Thomas A. Mazzuchi and Ehsan S. Soofi
Additional contact information
Refik Soyer: The George Washington University School of Business
Thomas A. Mazzuchi: George Washington University School of Engineering and Applied Science
Ehsan S. Soofi: University of Wisconsin-Milwaukee

Working Papers from School of Business, The George Washington University

Abstract: Kullback-Leibler information is widely used for developing indices of distributional fit. The most celebrated of such indices is Akaike’s AIC, which is derived as an estimate of the minimum Kullback-Leibler information between the unknown data-generating distribution and a parametric model. In the derivation of AIC, the entropy of the data-generating distribution is bypassed because it is free from the parameters. Consequently, the AIC type measures provide criteria for model comparison purposes only, and do not provide information diagnostic about the model fit. A nonparametric estimate of entropy of the data-generating distribution is needed for assessing the model fit. Several entropy estimates are available and have been used for frequentist inference about information fit indices. A few entropy-based fit indices have been suggested for Bayesian inference. This paper develops a class of entropy estimates and provides a procedure for Bayesian inference on the entropy and a fit index. For the continuous case, we define a quantized entropy that approximates and converges to the entropy integral. The quantized entropy includes some well known measures of sample entropy and the existing Bayes entropy estimates as its special cases. For inference about the fit, we use the candidate model as the expected distribution in the Dirichlet process prior and derive the posterior mean of the quantized entropy as the Bayes estimate. The maximum entropy characterization of the candidate model is then used to derive the prior and posterior distributions for the Kullback-Leibler information index of fit. The consistency of the proposed Bayes estimates for the entropy and for the information index are shown. As by-products, the procedure also produces priors and posteriors for the model parameters and the moments.

Date: 2006-06
New Economics Papers: this item is included in nep-ecm
References: Add references at CitEc
Citations:

Downloads: (external link)
http://www.gwu.edu/%7Ebusiness/research/workingpapers/Mazzuchi-Soofi-Soyer.pdf
Our link check indicates that this URL is bad, the error code is: 404 Not Found (http://www.gwu.edu/%7Ebusiness/research/workingpapers/Mazzuchi-Soofi-Soyer.pdf [301 Moved Permanently]--> https://www2.gwu.edu/~business/research/workingpapers/Mazzuchi-Soofi-Soyer.pdf [302 Found]--> https://business.gwu.edu/research/workingpapers/Mazzuchi-Soofi-Soyer.pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gwu:wpaper:0012

Access Statistics for this paper

More papers in Working Papers from School of Business, The George Washington University Contact information at EDIRC.
Bibliographic data for series maintained by GW School of Business Communications ().