Generalized Mutual Information

Zhiyi Zhang
Additional contact information
Zhiyi Zhang: Department of Mathematics and Statistics, University of North Carolina at Charlotte, Charlotte, NC 28223, USA

Stats, 2020, vol. 3, issue 2, 1-8

Abstract: Mutual information is one of the essential building blocks of information theory. It is however only finitely defined for distributions in a subclass of the general class of all distributions on a joint alphabet. The unboundedness of mutual information prevents its potential utility from being extended to the general class. This is in fact a void in the foundation of information theory that needs to be filled. This article proposes a family of generalized mutual information whose members are indexed by a positive integer n , with the n th member being the mutual information of n th order. The mutual information of the first order coincides with Shannon’s, which may or may not be finite. It is however established (a) that each mutual information of an order greater than 1 is finitely defined for all distributions of two random elements on a joint countable alphabet, and (b) that each and every member of the family enjoys all the utilities of a finite Shannon’s mutual information.

Keywords: mutual information; Shannon’s entropy; conditional distribution of total collision; generalized entropy; generalized mutual information (search for similar items in EconPapers)
JEL-codes: C1 C10 C11 C14 C15 C16 (search for similar items in EconPapers)
Date: 2020
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (2)

Downloads: (external link)
https://www.mdpi.com/2571-905X/3/2/13/pdf (application/pdf)
https://www.mdpi.com/2571-905X/3/2/13/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jstats:v:3:y:2020:i:2:p:13-165:d:369599

Access Statistics for this article

Stats is currently edited by Mrs. Minnie Li

More articles in Stats from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().