Noise and conditional entropy evolution

Michael C. Mackey and Marta Tyran-Kamińska

Physica A: Statistical Mechanics and its Applications, 2006, vol. 365, issue 2, 360-382

Abstract: We study the convergence properties of the conditional (Kullback–Leibler) entropy in stochastic systems. We have proved general results showing that asymptotic stability is a necessary and sufficient condition for the monotone convergence of the conditional entropy to its maximal value of zero. Additionally we have made specific calculations of the rate of convergence of this entropy to zero in a one-dimensional situation, illustrated by Ornstein–Uhlenbeck and Rayleigh processes, higher dimensional situations, and a two-dimensional Ornstein–Uhlenbeck process with a stochastically perturbed harmonic oscillator and colored noise as examples. We also apply our general results to the problem of conditional entropy convergence in the presence of dichotomous noise. In both the one-dimensional and multidimensional cases we show that the convergence of the conditional entropy to zero is monotone and at least exponential. In the specific cases of the Ornstein–Uhlenbeck and Rayleigh processes, as well as the stochastically perturbed harmonic oscillator and colored noise examples, we obtain exact formulae for the temporal evolution of the conditional entropy starting from a concrete initial distribution. The rather surprising result in this case is that the rate of convergence of the entropy to zero is independent of the noise amplitude.

Keywords: Markov operator; Asymptotic stability; Conditional entropy (search for similar items in EconPapers)
Date: 2006
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0378437105010447
Full text for ScienceDirect subscribers only. Journal offers the option of making the article available online on Science direct for a fee of $3,000

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:phsmap:v:365:y:2006:i:2:p:360-382

DOI: 10.1016/j.physa.2005.10.001

Access Statistics for this article

Physica A: Statistical Mechanics and its Applications is currently edited by K. A. Dawson, J. O. Indekeu, H.E. Stanley and C. Tsallis

More articles in Physica A: Statistical Mechanics and its Applications from Elsevier
Bibliographic data for series maintained by Catherine Liu ().