Understanding Machine Learning Principles: Learning, Inference, Generalization, and Computational Learning Theory

Ke-Lin Du, Rengong Zhang, Bingchun Jiang (), Jie Zeng and Jiabin Lu
Additional contact information
Ke-Lin Du: School of Mechanical and Electrical Engineering, Guangdong University of Science and Technology, Dongguan 523668, China
Rengong Zhang: Zhejiang Yugong Information Technology Co., Ltd., Changhe Road 475, Hangzhou 310002, China
Bingchun Jiang: School of Mechanical and Electrical Engineering, Guangdong University of Science and Technology, Dongguan 523668, China
Jie Zeng: Shenzhen Feng Xing Tai Bao Technology Co., Ltd., Shenzhen 518063, China
Jiabin Lu: Faculty of Electromechanical Engineering, Guangdong University of Technology, Guangzhou 510006, China

Mathematics, 2025, vol. 13, issue 3, 1-56

Abstract: Machine learning has become indispensable across various domains, yet understanding its theoretical underpinnings remains challenging for many practitioners and researchers. Despite the availability of numerous resources, there is a need for a cohesive tutorial that integrates foundational principles with state-of-the-art theories. This paper addresses the fundamental concepts and theories of machine learning, with an emphasis on neural networks, serving as both a foundational exploration and a tutorial. It begins by introducing essential concepts in machine learning, including various learning and inference methods, followed by criterion functions, robust learning, discussions on learning and generalization, model selection, bias–variance trade-off, and the role of neural networks as universal approximators. Subsequently, the paper delves into computational learning theory, with probably approximately correct (PAC) learning theory forming its cornerstone. Key concepts such as the VC-dimension, Rademacher complexity, and empirical risk minimization principle are introduced as tools for establishing generalization error bounds in trained models. The fundamental theorem of learning theory establishes the relationship between PAC learnability, Vapnik–Chervonenkis (VC)-dimension, and the empirical risk minimization principle. Additionally, the paper discusses the no-free-lunch theorem, another pivotal result in computational learning theory. By laying a rigorous theoretical foundation, this paper provides a comprehensive tutorial for understanding the principles underpinning machine learning.

Keywords: PAC learning theory; empirical risk minimization; computational learning theory; model selection; universal approximation; Turing-complete (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2025
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2227-7390/13/3/451/pdf (application/pdf)
https://www.mdpi.com/2227-7390/13/3/451/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:13:y:2025:i:3:p:451-:d:1579575

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().