Neural Networks

Vladimir Shikhman () and David Müller ()
Additional contact information
Vladimir Shikhman: Chemnitz University of Technology
David Müller: Chemnitz University of Technology

Chapter 8 in Mathematical Foundations of Big Data Analytics, 2021, pp 149-169 from Springer

Abstract: Abstract In biology, a neural network is a circuit composed of a group of chemically connected or functionally associated neurons. The connections of neurons are modeled by means of weights. A positive weight reflects an excitatory connection, while negative values mean inhibitory connections. All inputs are modified by weights and summed up. This aggregation corresponds to taking linear combinations. Finally, an activation function controls the amplitude of the output. Although each of them being relatively simple, the neurons can build networks with surprisingly high processing power. This gave rise since 1970s to the development of neural networks for solving artificial intelligence problems. Remarkable progress in this direction have been achieved particularly in the last decade. As example, we just mention the neural network Leela Chess Zero that managed to win in May 2019 the Top Chess Engine Championship, defeating the conventional chess engine Stockfish in the final. In this chapter, we get to know how the neuron’s functioning can be mathematically modeled. This is done on example of classification neurons in terms of the generalized linear regression. The generalization refers here to the use of activation functions. First, we focus on the sigmoid activation function and the corresponding logistic regression. The training of weights will be based on the minimization of the average cross-entropy arising from the maximum likelihood estimation. We minimize the average cross-entropy by means of the stochastic gradient descent. Second, the threshold activation function is considered. The corresponding neuron is referred to as perceptron. For the latter, the Rosenblatt learning is shown to provide a correct linear classifier in finitely many iteration steps. After mentioning the XOR problem, which cannot be handled by perceptrons with one layer, we introduce multilayer perceptrons. We point out their importance by stating the universal approximation theorem.

Date: 2021
References: Add references at CitEc
Citations:

There are no downloads for this item, see the EconPapers FAQ for hints about obtaining it.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:sprchp:978-3-662-62521-7_8

Ordering information: This item can be ordered from
http://www.springer.com/9783662625217

DOI: 10.1007/978-3-662-62521-7_8

Access Statistics for this chapter

More chapters in Springer Books from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().