Testing for Neglected Nonlinearity Using Regularized Artificial Neural Networks

Tae Hwy Lee, Zhou Xi () and Ru Zhang ()
Additional contact information
Zhou Xi: University of California, Riverside
Ru Zhang: University of California, Riverside

No 201422, Working Papers from University of California at Riverside, Department of Economics

Abstract: The artificial neural network (ANN) test of Lee et al. (Journal of Econometrics 56, 269–290, 1993) uses the ability of the ANN activation functions in the hidden layer to detect neglected functionalmisspecification. As the estimation of the ANN model is often quite difficult, LWG suggested activate the ANN hidden units based on randomly drawn activation parameters. To be robust to the random activations, a large number of activations is desirable. This leads to a situation for which regularization of the dimensionality is needed by techniques such as principal component analysis (PCA), Lasso, Pretest, partial least squares (PLS), among others. However, some regularization methods can lead to selection bias in testing if the dimensionality reduction is conducted by supervising the relationship between the ANN hidden layer activations of inputs and the output variable. This paper demonstrates that while these supervised regularization methods such as Lasso, Pretest, PLS, may be useful for forecasting, they may not be used for testing because the supervised regularizationwould create the post-sample inference or post-selection inference (PoSI) problem. Our Monte Carlo simulation shows that the PoSI problem is especially severe with PLS and Pretest while it seems relatively mild or even negligible with Lasso. This paper also demonstrates that the use of unsupervised regularization does not lead to the PoSI problem. Lee et al. (Journal of Econometrics 56, 269–290, 1993) suggested a regularization by principal components, which is a unsupervised regularization.While the supervised regularizations may be useful in forecasting, regularization should not be supervised in inference.

Keywords: Randomized; ANN; activations; •; Dimension; reduction; •; Supervised; regularization; •; Unsupervised; regularization; •; PCA; •; Lasso; •; PLS; •; Pretest; •; PoSI; problem (search for similar items in EconPapers)
JEL-codes: C12 C45 (search for similar items in EconPapers)
Pages: 25 Pages
Date: 2013-09, Revised 2012-04
New Economics Papers: this item is included in nep-cmp, nep-ecm and nep-ore
References: View references in EconPapers View complete reference list from CitEc
Citations:

Published in Recent Advances in Estimating Nonlinear Models: With Applications in Economics and Finance, Chapter 3, pages 33-57, 2013. Jun Ma and Mark E. Wohar (ed.), Springer Publishers. September 2013.

Downloads: (external link)
https://economics.ucr.edu/repec/ucr/wpaper/201422.pdf First version, 2013 (application/pdf)
https://economics.ucr.edu/repec/ucr/wpaper/201422R.pdf Revised version, 2012 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:ucr:wpaper:201422

Access Statistics for this paper

More papers in Working Papers from University of California at Riverside, Department of Economics Contact information at EDIRC.
Bibliographic data for series maintained by Kelvin Mac ().