Perceptron: Learning, Generalization, Model Selection, Fault Tolerance, and Role in the Deep Learning Era

Ke-Lin Du (), Chi-Sing Leung, Wai Ho Mow and M. N. S. Swamy
Additional contact information
Ke-Lin Du: Department of Electrical and Computer Engineering, Concordia University, Montreal, QC H3G 1M8, Canada
Chi-Sing Leung: Department of Electrical Engineering, City University of Hong Kong, Hong Kong, China
Wai Ho Mow: Department of Electronic and Computer Engineering, The Hong Kong University of Science and Technology, Hong Kong, China
M. N. S. Swamy: Department of Electrical and Computer Engineering, Concordia University, Montreal, QC H3G 1M8, Canada

Mathematics, 2022, vol. 10, issue 24, 1-46

Abstract: The single-layer perceptron, introduced by Rosenblatt in 1958, is one of the earliest and simplest neural network models. However, it is incapable of classifying linearly inseparable patterns. A new era of neural network research started in 1986, when the backpropagation (BP) algorithm was rediscovered for training the multilayer perceptron (MLP) model. An MLP with a large number of hidden nodes can function as a universal approximator. To date, the MLP model is the most fundamental and important neural network model. It is also the most investigated neural network model. Even in this AI or deep learning era, the MLP is still among the few most investigated and used neural network models. Numerous new results have been obtained in the past three decades. This survey paper gives a comprehensive and state-of-the-art introduction to the perceptron model, with emphasis on learning, generalization, model selection and fault tolerance. The role of the perceptron model in the deep learning era is also described. This paper provides a concluding survey of perceptron learning, and it covers all the major achievements in the past seven decades. It also serves a tutorial for perceptron learning.

Keywords: multilayer perceptron; perceptron; backpropagation; stochastic gradient descent; second-order learning; model selection; robust learning; deep learning (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2022
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (3)

Downloads: (external link)
https://www.mdpi.com/2227-7390/10/24/4730/pdf (application/pdf)
https://www.mdpi.com/2227-7390/10/24/4730/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:10:y:2022:i:24:p:4730-:d:1001814

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().