Information Theory and Log-Likelihood Models: A Basis for Model Selection and Inference

Kenneth P. Burnham and David R. Anderson
Additional contact information
Kenneth P. Burnham: Colorado State University, Colorado Cooperative Fish and Wildlife Research Unit
David R. Anderson: Colorado State University, Colorado Cooperative Fish and Wildlife Research Unit

Chapter 2 in Model Selection and Inference, 1998, pp 32-74 from Springer

Abstract: Abstract Full reality cannot be included in a model; thus we seek a good model to approximate the effects or factors supported by the empirical data. The selection of an appropriate approximating model is critical to statistical inference from many types of empirical data. This chapter introduces concepts from information theory (see Guiasu 1977), which has been a discipline only since the mid-1940s and covers a variety of theories and methods that are fundamental to many of the sciences (see Cover and Thomas 1991 for an exciting overview; Fig. 2.1 is produced from their book and shows their view of the relationship of information theory to several other fields). In particular, the Kullback—Liebler “distance,” or “information,” between two models (Kullback and Leibler 1951) is introduced, discussed, and linked to Boltzmann’s entropy in this chapter. Akaike (1973) found a simple relationship between the Kullback—Liebler distance and Fisher's maximized log-likelihood function (see deLeeuw 1992 for a brief review). This relationship leads to a simple, effective, and very general methodology for selecting a parsimonious model for the analysis of empirical data.

Keywords: Model Selection; True Model; Candidate Model; Monte Carlo Study; Model Selection Criterion (search for similar items in EconPapers)
Date: 1998
References: Add references at CitEc
Citations:

There are no downloads for this item, see the EconPapers FAQ for hints about obtaining it.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:sprchp:978-1-4757-2917-7_2

Ordering information: This item can be ordered from
http://www.springer.com/9781475729177

DOI: 10.1007/978-1-4757-2917-7_2

Access Statistics for this chapter

More chapters in Springer Books from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().