Kullback–Leibler Relative Entropy

Ovidiu Calin and Constantin Udrişte
Additional contact information
Ovidiu Calin: Eastern Michigan University, Department of Mathematics
Constantin Udrişte: University Politehnica of Bucharest, Faculty of Applied Sciences Department of Mathematics-Informatics

Chapter Chapter 4 in Geometric Modeling in Probability and Statistics, 2014, pp 111-131 from Springer

Abstract: Abstract Even if the entropy of a finite, discrete density is always positive, in the case of continuous density the entropy is not always positive. This drawback can be corrected by introducing another concept, which measures the relative entropy between two given densities. This chapter studies the Kullback–Leibler relative entropy (known also as the Kullback–Leibler divergence) between two probability densities in both discrete and continuous cases.

Keywords: Gamma Distribution; Triangle Inequality; Fisher Information; Relative Entropy; Entropy Function (search for similar items in EconPapers)
Date: 2014
References: Add references at CitEc
Citations:

There are no downloads for this item, see the EconPapers FAQ for hints about obtaining it.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:sprchp:978-3-319-07779-6_4

Ordering information: This item can be ordered from
http://www.springer.com/9783319077796

DOI: 10.1007/978-3-319-07779-6_4

Access Statistics for this chapter

More chapters in Springer Books from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().