Information measures of kernel estimation

Neshat Beheshti, Jeffrey Racine and Ehsan S. Soofi

Econometric Reviews, 2019, vol. 38, issue 1, 47-68

Abstract: Kernel estimates of entropy and mutual information have been studied extensively in statistics and econometrics. Kullback–Leibler divergence has been used in the kernel estimation literature; yet the information characteristic of kernel estimation remains unexplored. We explore kernel estimation as an information transmission operation where the empirical cumulative distribution function is transformed into a smooth estimate. The smooth kernel estimate is a mixture of kernel functions. The Jensen–Shannon (JS) divergence of the mixture distribution provides the information measure of kernel estimation. This measure admits Kullback–Leibler and mutual information representations and provides a lower bound for the entropy of the kernel estimate of the distribution in terms of the Shannon entropy of the kernel function and the bandwidth. The JS divergence provides guidance for kernel choice based on information-theoretic considerations which helps resolve a conundrum, namely that it is legitimate and desirable to base such choice on considerations other than the mean integrated square error of the kernel smoother. We introduce a generalized polynomial kernel (GPK) family that nests a broad range of popular kernel functions, and explore its properties in terms of Shannon and Rényi entropies. We show that these entropies and variance order the GPK functions similarly. The JS information measures of six kernel functions are compared via simulations from Gaussian, gamma, and Student-t data-generating processes. The proposed framework provides the foundation for further explorations into the information-theoretic nature of kernel smoothing.

Date: 2019
References: Add references at CitEc
Citations:

Downloads: (external link)
http://hdl.handle.net/10.1080/07474938.2016.1222236 (text/html)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:taf:emetrv:v:38:y:2019:i:1:p:47-68

Ordering information: This journal article can be ordered from
http://www.tandfonline.com/pricing/journal/LECR20

DOI: 10.1080/07474938.2016.1222236

Access Statistics for this article

Econometric Reviews is currently edited by Dr. Essie Maasoumi

More articles in Econometric Reviews from Taylor & Francis Journals
Bibliographic data for series maintained by ().