A comparison between penalized logistic regressions and classification tree approaches

Mostafa Behzadi, Rossita Mohamad Yunus, Saharuddin Bin Mohamad and Nor Aishah Hamzah

Communications in Statistics - Theory and Methods, 2025, vol. 54, issue 19, 6231-6248

Abstract: The penalized models have been established to be necessary for classifying the emerging complex, high-dimensional datasets. One of these penalized models is the “modified elastic-net” (M-Enet). The special cases of M-Enet (ridge, bridge, Lasso, and elastic-net) can be produced based on its penalty function. By using the fact that the Fisher-Scoring (FS) and Newton-Raphson (NR) algorithms use different approaches to solve numerical problems but are the same in the canonical link, in order to ease implementation, we developed an FS algorithm for the computation of the parameter estimates of the M-Enet. A simulation study of the microbiome as complex, high-dimensional data shows that each special case of the M-Enet penalty function has superiority in the sense of model accuracy, sensitivity, and feature selection technique. A comparison study is performed between the classification of binary responses using M-Enet as a parametric method and the classification tree via “GUIDE” as a non parametric method. The statistical analysis of the numerical results reveals that the special cases of M-Enet models outperform other M-Enet models and GUIDE in terms of average prediction accuracy and sensitivity, with p-values of 0. 0362 and 0. 0214, respectively. Consequently, these models exhibit superior classification performance compared to other classifier models.

Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://hdl.handle.net/10.1080/03610926.2025.2450779 (text/html)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:taf:lstaxx:v:54:y:2025:i:19:p:6231-6248

Ordering information: This journal article can be ordered from
http://www.tandfonline.com/pricing/journal/lsta20

DOI: 10.1080/03610926.2025.2450779

Access Statistics for this article

Communications in Statistics - Theory and Methods is currently edited by Debbie Iscoe

More articles in Communications in Statistics - Theory and Methods from Taylor & Francis Journals
Bibliographic data for series maintained by Chris Longhurst ().