Convergence of the EM algorithm in KL distance for overspecified Gaussian mixtures

Alan Legg (), Artur Pak (), Igor Melnykov (), Arman Bolatov () and Zhenisbek Assylbekov ()
Additional contact information
Alan Legg: Purdue University Fort Wayne
Artur Pak: Nazarbayev University
Igor Melnykov: University of Minnesota Duluth
Arman Bolatov: Mohamed bin Zayed University of Artificial Intelligence
Zhenisbek Assylbekov: Purdue University Fort Wayne

Statistical Papers, 2025, vol. 66, issue 6, No 2, 33 pages

Abstract: Abstract We present a study of the convergence properties of the Expectation-Maximization (EM) algorithm when applied to an overspecified model. In particular, we consider fitting a balanced mixture of two Gaussians to data originating from a single Gaussian. We provide theoretical bounds on the Kullback–Leibler (KL) divergence between the fitted and true distributions. An important feature is concavity and radiality of the expected log-likelihood function on a hypersurface induced by the EM algorithm, which greatly simplifies the analysis. We also show how our result on KL divergence can be used to upperbound the error rate of a mixture discriminant analysis classifier trained by the EM algorithm.

Keywords: Mixture models; Expectation-maximization; KL divergence (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s00362-025-01749-z Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:stpapr:v:66:y:2025:i:6:d:10.1007_s00362-025-01749-z

Ordering information: This journal article can be ordered from
http://www.springer. ... business/journal/362

DOI: 10.1007/s00362-025-01749-z

Access Statistics for this article

Statistical Papers is currently edited by C. Müller, W. Krämer and W.G. Müller

More articles in Statistical Papers from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().