Parameter-expanded ECME algorithms for logistic and penalized logistic regression

Nicholas C. Henderson () and Zhongzhe Ouyang
Additional contact information
Nicholas C. Henderson: University of Michigan
Zhongzhe Ouyang: University of Michigan

Computational Statistics, 2025, vol. 40, issue 7, No 19, 3883-3909

Abstract: Abstract Parameter estimation in logistic regression is a well-studied problem with the Newton–Raphson method being one of the most prominent optimization techniques used in practice. A number of monotone optimization methods including minorization-maximization (MM) algorithms, expectation-maximization (EM) algorithms and related variational Bayes approaches offer useful alternatives guaranteed to increase the logistic regression likelihood at every iteration. In this article, we propose and evaluate an optimization procedure that is based on a straightforward modification of an EM algorithm for logistic regression. Our method can substantially improve the computational efficiency of the EM algorithm while preserving the monotonicity of EM and the simplicity of the EM parameter updates. By introducing an additional latent parameter and selecting this parameter to maximize the penalized observed-data log-likelihood at every iteration, our iterative algorithm can be interpreted as a parameter-expanded expectation-conditional maximization either (ECME) algorithm, and we demonstrate how to use the parameter-expanded ECME with an arbitrary choice of weights and penalty function. In addition, we describe a generalized version of our parameter-expanded ECME algorithm that can be tailored to the challenges encountered in specific high-dimensional problems, and we study several interesting connections between this generalized algorithm and other well-known methods. Performance comparisons between our method, the EM algorithm, Newton–Raphson, and several other optimization methods are presented using an extensive series of simulation studies based upon both real and synthetic datasets.

Keywords: Convergence acceleration; EM algorithm; MM algorithm; Pólya-Gamma augmentation; Parameter expansion (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s00180-025-01619-0 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:compst:v:40:y:2025:i:7:d:10.1007_s00180-025-01619-0

Ordering information: This journal article can be ordered from
http://www.springer.com/statistics/journal/180/PS2

DOI: 10.1007/s00180-025-01619-0

Access Statistics for this article

Computational Statistics is currently edited by Wataru Sakamoto, Ricardo Cao and Jürgen Symanzik

More articles in Computational Statistics from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().