Reproducing Kernel Hilbert Spaces for Penalized Regression: A Tutorial

Alvaro Nosedal-Sanchez, Curtis B. Storlie, Thomas C.M. Lee and Ronald Christensen

The American Statistician, 2012, vol. 66, issue 1, 50-60

Abstract: Penalized regression procedures have become very popular ways to estimate complicated functions. The smoothing spline, for example, is the solution of a minimization problem in a functional space. If such a minimization problem is posed on a reproducing kernel Hilbert space (RKHS), the solution is guaranteed to exist, is unique, and has a very simple form. There are excellent books and articles about RKHS and their applications in statistics; however, this existing literature is very dense. This article provides a friendly reference for a reader approaching this subject for the first time. It begins with a simple problem, a system of linear equations, and then gives an intuitive motivation for reproducing kernels. Armed with the intuition gained from our first examples, we take the reader from vector spaces to Banach spaces and to RKHS. Finally, we present some statistical estimation problems that can be solved using the mathematical machinery discussed. After reading this tutorial, the reader will be ready to study more advanced texts and articles about the subject, such as those by Wahba or Gu. Online supplements are available for this article.

Date: 2012
References: View complete reference list from CitEc
Citations: View citations in EconPapers (2)

Downloads: (external link)
http://hdl.handle.net/10.1080/00031305.2012.678196 (text/html)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:taf:amstat:v:66:y:2012:i:1:p:50-60

Ordering information: This journal article can be ordered from
http://www.tandfonline.com/pricing/journal/UTAS20

DOI: 10.1080/00031305.2012.678196

Access Statistics for this article

The American Statistician is currently edited by Eric Sampson

More articles in The American Statistician from Taylor & Francis Journals
Bibliographic data for series maintained by Chris Longhurst ().