Mixture-based estimation of entropy

Stéphane Robin and Luca Scrucca

Computational Statistics & Data Analysis, 2023, vol. 177, issue C

Abstract: The entropy is a measure of uncertainty that plays a central role in information theory. When the distribution of the data is unknown, an estimate of the entropy needs to be obtained from the data sample itself. A semi-parametric estimate is proposed based on a mixture model approximation of the distribution of interest. A Gaussian mixture model is used to illustrate the accuracy and versatility of the proposal, although the estimate can rely on any type of mixture. Performance of the proposed approach is assessed through a series of simulation studies. Two real-life data examples are also provided to illustrate its use.

Keywords: Entropy estimation; Gaussian mixtures; Mixture models; Mutual information (search for similar items in EconPapers)
Date: 2023
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0167947322001621
Full text for ScienceDirect subscribers only.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:csdana:v:177:y:2023:i:c:s0167947322001621

DOI: 10.1016/j.csda.2022.107582

Access Statistics for this article

Computational Statistics & Data Analysis is currently edited by S.P. Azen

More articles in Computational Statistics & Data Analysis from Elsevier
Bibliographic data for series maintained by Catherine Liu ().