Robust kernel-based regression with bounded influence for outliers

Sangheum Hwang, Dohyun Kim, Myong K Jeong and Bong-Jin Yum
Additional contact information
Sangheum Hwang: Korea Advanced Institute of Science and Technology, Daejeon, Korea
Dohyun Kim: Myongji University, Yongin, Korea
Myong K Jeong: RUTCOR (Rutgers Center for Operations Research), The State University of New Jersey, Piscataway, USA
Bong-Jin Yum: Korea Advanced Institute of Science and Technology, Daejeon, Korea

Journal of the Operational Research Society, 2015, vol. 66, issue 8, 1385-1398

Abstract: The kernel-based regression (KBR) method, such as support vector machine for regression (SVR) is a well-established methodology for estimating the nonlinear functional relationship between the response variable and predictor variables. KBR methods can be very sensitive to influential observations that in turn have a noticeable impact on the model coefficients. The robustness of KBR methods has recently been the subject of wide-scale investigations with the aim of obtaining a regression estimator insensitive to outlying observations. However, existing robust KBR (RKBR) methods only consider Y-space outliers and, consequently, are sensitive to X-space outliers. As a result, even a single anomalous outlying observation in X-space may greatly affect the estimator. In order to resolve this issue, we propose a new RKBR method that gives reliable result even if a training data set is contaminated with both Y-space and X-space outliers. The proposed method utilizes a weighting scheme based on the hat matrix that resembles the generalized M-estimator (GM-estimator) of conventional robust linear analysis. The diagonal elements of hat matrix in kernel-induced feature space are used as leverage measures to downweight the effects of potential X-space outliers. We show that the kernelized hat diagonal elements can be obtained via eigen decomposition of the kernel matrix. The regularized version of kernelized hat diagonal elements is also proposed to deal with the case of the kernel matrix having full rank where the kernelized hat diagonal elements are not suitable for leverage. We have shown that two kernelized leverage measures, namely, the kernel hat diagonal element and the regularized one, are related to statistical distance measures in the feature space. We also develop an efficiently kernelized training algorithm for the parameter estimation based on iteratively reweighted least squares (IRLS) method. The experimental results from simulated examples and real data sets demonstrate the robustness of our proposed method compared with conventional approaches.

Date: 2015
References: Add references at CitEc
Citations: View citations in EconPapers (2)

Downloads: (external link)
http://www.palgrave-journals.com/jors/journal/v66/n8/pdf/jors201442a.pdf Link to full text PDF (application/pdf)
http://www.palgrave-journals.com/jors/journal/v66/n8/full/jors201442a.html Link to full text HTML (text/html)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:pal:jorsoc:v:66:y:2015:i:8:p:1385-1398

Ordering information: This journal article can be ordered from
http://www.springer. ... search/journal/41274

Access Statistics for this article

Journal of the Operational Research Society is currently edited by Tom Archibald and Jonathan Crook

More articles in Journal of the Operational Research Society from Palgrave Macmillan, The OR Society
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().