A Sequential Training Strategy for Locally Recurrent Neural Networks

Jie Zhang and A. Julian Morris
Additional contact information
Jie Zhang: University of Newcastle, Centre for Process Analysis, Chemometrics and Control Dept of Chemical & Process Engineering
A. Julian Morris: University of Newcastle, Centre for Process Analysis, Chemometrics and Control Dept of Chemical & Process Engineering

Chapter 10 in Intelligent Hybrid Systems, 1997, pp 231-252 from Springer

Abstract: Abstract In locally recurrent neural networks, the output of a dynamic neuron is only fed back to itself. This particular structure makes it possible to train the network sequentially. A sequential orthogonal training method is developed in this chapter to train locally recurrent neural networks. The networks considered here contain a single-hidden-layer and dynamic neurons are located in the hidden layer. During network training, the first hidden neuron is used to model the relationship between inputs and outputs whereas other hidden neurons are added sequentially to model the relationship between inputs and model residuals. When adding a hidden neuron, its contribution is due to that part of its output vector which is orthogonal to the space spanned by the output vectors of the previous hidden neurons. The Gram-Schmidt orthogonalisation technique is used at each training step to form a set of orthogonal bases for the space spanned by the hidden neuron outputs. The optimum hidden layer weights can be obtained through gradient based optimisation method while the output layer weights can be found using least squares regression. Hidden neurons are added sequentially and the training procedure terminates when the model error is lower than a predefined level. Using this training method, the necessary number of hidden neurons can be found and, hence, avoiding the problem of over fitting. Neurons with mixed types of activation functions and dynamic orders can be incorporated into a single network. Mixed node networks can offer improved performance in terms of representation capabilities and network size parsimony. The excellent performance of the proposed technique is demonstrated by application examples.

Keywords: Hide Neuron; Model Predictive Control; Recurrent Neural Network; Distillation Column; Continuous Stir Tank Reactor (search for similar items in EconPapers)
Date: 1997
References: Add references at CitEc
Citations:

There are no downloads for this item, see the EconPapers FAQ for hints about obtaining it.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:sprchp:978-1-4615-6191-0_10

Ordering information: This item can be ordered from
http://www.springer.com/9781461561910

DOI: 10.1007/978-1-4615-6191-0_10

Access Statistics for this chapter

More chapters in Springer Books from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().