On the Generalization Problem

L. C. W. Dixon
Additional contact information
L. C. W. Dixon: University of Hertfordshire

Journal of Optimization Theory and Applications, 2001, vol. 111, issue 3, No 2, 489-500

Abstract: Abstract The generalization problem considered in this paper assumes that a limited amount of input and output data from a system is available, and that from this information an estimate of the output produced by another input is required. The ideas arose in the study of neural networks, but apply equally to any approximation approach. The main result is that the type of neural network to be used for generalization should be determined by the prior knowledge about the nature of the output from the system. Without such information, either of two networks matching the training data is equally likely to be the better at estimating the output generated by the same system at a new input. Therefore, the search for an optimum generalization network for use on all problems is inappropriate. For both (0, 1) and accurate real outputs, it is shown that simple approximations exist that fit the data, so these will be equally likely to generalize better than more sophisticated networks, unless prior knowledge is available that excludes them. For noisy real outputs, it is shown that the standard least squares approach forces the neural network to approximate an incorrect process; an alternative approach is outlined, which again is much easier to learn and use.

Keywords: Generalization; neural networks; approximation; learning (search for similar items in EconPapers)
Date: 2001
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1023/A:1012693710097 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:joptap:v:111:y:2001:i:3:d:10.1023_a:1012693710097

Ordering information: This journal article can be ordered from
http://www.springer. ... cs/journal/10957/PS2

DOI: 10.1023/A:1012693710097

Access Statistics for this article

Journal of Optimization Theory and Applications is currently edited by Franco Giannessi and David G. Hull

More articles in Journal of Optimization Theory and Applications from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().