Information and Entropy in Cluster Analysis

H. H. Bock
Additional contact information
H. H. Bock: Technical University of Aachen, Institute of Statistics

Chapter 3 in Proceedings of the First US/Japan Conference on the Frontiers of Statistical Modeling: An Informational Approach, 1994, pp 115-147 from Springer

Abstract: Abstract Cluster analysis provides methods for subdividing a set of objects into a suitable number of ‘classes’, ‘groups’, or ‘types’ C 1,…,C m such that each class is as homogeneous as possible and different classes are sufficiently separated. This paper shows how entropy and information measures have been or can be used in this framework. We present several probabilistic clustering approaches which are related to, or lead to, information and entropy criteria g(C) for selecting an optimum partition C = (C 1,…,C m ) of n data vectors, for qualitative and for quantitative data, assuming loglinear, logistic, and normal distribution models, together with appropriate iterative clustering algorithms. A new partitioning problem is considered in Section 5 where we look for a dissection (discretization) C of an arbitrary sample space Y (e.g. R p or 0,1 p ) such that the ø—divergence I c (P 0, P 1) between two discretized distributions P o (C i ), P 1(C i ) (i = 1,…, m) will be maximized (e.g., Kullback-Leibler’s discrimination information or the X 2 noncentrality parameter). We conclude with some comments on methods for selecting a suitable number of classes, e.g., by using Akaike’s information criterion AIC and its modifications.

Keywords: Latent Class Analysis; Support Point; Optimum Partition; Loglinear Model; Support Line (search for similar items in EconPapers)
Date: 1994
References: Add references at CitEc
Citations:

There are no downloads for this item, see the EconPapers FAQ for hints about obtaining it.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:sprchp:978-94-011-0800-3_4

Ordering information: This item can be ordered from
http://www.springer.com/9789401108003

DOI: 10.1007/978-94-011-0800-3_4

Access Statistics for this chapter

More chapters in Springer Books from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().