First asymptotic theorem and related results

Roman V. Belavkin, Panos M. Pardalos, Jose C. Principe and Ruslan L. Stratonovich
Additional contact information
Roman V. Belavkin: Middlesex University, Faculty of Science and Technology
Panos M. Pardalos: University of Florida, Industrial and Systems Engineering
Jose C. Principe: University of Florida, Electrical & Computer Engineering

Chapter Chapter 4 in Theory of Information and its Value, 2020, pp 77-101 from Springer

Abstract: Abstract In the previous chapter, for one particular example (see Sections 3.1 and 3.4 ) we showed that in calculating the maximum entropy (i.e. the capacity of a noiseless channel) the constraint c ( y ) ⩽ a $$c(y) \leqslant a$$ imposed on feasible realizations is equivalent, for a sufficiently long code sequence, to the constraint 𝔼 [ c ( y ) ] ⩽ a $$\mathbb {E}[c(y)] \leqslant a$$ on the mean value 𝔼 [ c ( y ) ] $$\mathbb {E}[c(y)]$$ . In this chapter we prove (Section 4.3) that under certain assumptions such equivalence takes place in the general case; this is the assertion of the first asymptotic theorem. In what follows, we shall also consider the other two asymptotic theorems (Chapters 7 and 11 ), which are the most profound results of information theory. All of them have the following feature in common: ultimately all these theorems state that, for utmost large systems, the difference between the concepts of discreteness and continuity disappears, and that the characteristics of a large collection of discrete objects can be calculated using a continuous functional dependence involving averaged quantities. For the first variational problem, this feature is expressed by the fact that the discrete function H = ln M $$H = \ln M$$ of a, which exists under the constraint c(y) ≤ a, is asymptotically replaced by a continuous function H(a) calculated by solving the first variational problem. As far as the proof is concerned, the first asymptotic theorem turns out to be related to the theorem on canonical distribution stability (Section 4.2), which is very important in statistical thermodynamics and which is actually proved there when the canonical distribution is derived from the microcanonical one. Here we consider it in a more general and abstract form. The relationship between the first asymptotic theorem and the theorem on the canonical distribution once more underlines the intrinsic unity of the mathematical apparatus of information theory and statistical thermodynamics.

Date: 2020
References: Add references at CitEc
Citations:

There are no downloads for this item, see the EconPapers FAQ for hints about obtaining it.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:sprchp:978-3-030-22833-0_4

Ordering information: This item can be ordered from
http://www.springer.com/9783030228330

DOI: 10.1007/978-3-030-22833-0_4

Access Statistics for this chapter

More chapters in Springer Books from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().