A Survey of Classic Machine Learning Algorithms: Principles and Applications

Xi Long and Xing Zhao
Additional contact information
Xi Long: School of Statistics and Data Science, Capital University of Economics and Business, Beijing 100071, China
Xing Zhao: School of Urban Economics and Public Administration, Capital University of Economics and Business, Beijing 100071, China

Innovation in Science and Technology, 2026, vol. 5, issue 2, 26-44

Abstract: Classical machine learning algorithms constitute the fundamental cornerstone of modern data science and intelligent system development. While deep learning has achieved transformative breakthroughs across numerous fields in recent years, classical methods remain indispensable in practical scenarios characterized by limited training data, stringent interpretability requirements, or constrained computational resources. Nevertheless, existing studies generally lack a systematic, unified, and beginner-friendly comprehensive survey that integrates theoretical elaboration, multi-dimensional comparative analysis, and actionable algorithm selection guidance. To fill this research gap, this paper presents a thorough investigation of ten representative classical machine learning algorithms: Linear Regression, Logistic Regression, Decision Tree, Random Forest, Support Vector Machine, K-Nearest Neighbors, Naive Bayes, Adaptive Boosting, K-Means, and Principal Component Analysis. Each algorithm is explicated within a consistent structural framework, covering its mathematical formulation, core working mechanism, distinct advantages, inherent limitations, and typical application scenarios. Furthermore, a horizontal comparative analysis is conducted across seven critical dimensions, and a practical algorithm selection framework with targeted recommendations for typical industrial scenarios is proposed. This work constructs a complete and logically coherent knowledge system of classical machine learning, serving as an accessible and pragmatic reference for novice learners and engineering practitioners. It also provides insights into future integration trends between classical algorithms and emerging technologies including large language models, explainable artificial intelligence, edge intelligence, automated machine learning, and federated learning.

Keywords: machine learning; algorithm survey; comparative analysis; algorithm selection; mathematical formulation (search for similar items in EconPapers)
Date: 2026
References: Add references at CitEc
Citations:

Downloads: (external link)
https://www.paradigmpress.org/ist/article/view/2144/1991 (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:bdz:inscte:v:5:y:2026:i:2:p:26-44

DOI: 10.63593/IST.2788-7030.2026.06.003

Access Statistics for this article

More articles in Innovation in Science and Technology from Paradigm Academic Press
Bibliographic data for series maintained by Editorial Office ().