Regression with I-priors

Wicher Bergsma

LSE Research Online Documents on Economics from London School of Economics and Political Science, LSE Library

Abstract: The problem of estimating a parametric or nonparametric regression function in a model with normal errors is considered. For this purpose, a novel objective prior for the regression function is proposed, defined as the distribution maximizing entropy subject to a suitable constraint based on the Fisher information on the regression function. The prior is named I-prior. For the present model, it is Gaussian with covariance kernel proportional to the Fisher information, and mean chosen a priori (e.g., 0). The I-prior has the intuitively appealing property that the more information is available about a linear functional of the regression function, the larger its prior variance, and, broadly speaking, the less influential the prior is on the posterior. Unlike the Jeffreys prior, it can be used in high dimensional settings. The I-prior methodology can be used as a principled alternative to Tikhonov regularization, which suffers from well-known theoretical problems which are briefly reviewed. The regression function is assumed to lie in a reproducing kernel Hilbert space (RKHS) over a low or high dimensional covariate space, giving a high degree of generality. Analysis of some real data sets and a small-scale simulation study show competitive performance of the I-prior methodology, which is implemented in the R-package iprior.

Keywords: reproducing kernel; RKHS; fisher information; maximum entropy; objective prior; g-prior; empirical Bates; regression; nonparametric regression; functional data analysis; classification; Tikhonov regularization (search for similar items in EconPapers)
JEL-codes: C1 (search for similar items in EconPapers)
Pages: 23 pages
Date: 2020-04-01
New Economics Papers: this item is included in nep-ecm and nep-ore
References: View references in EconPapers View complete reference list from CitEc
Citations:

Published in Econometrics and Statistics, 1, April, 2020, 14, pp. 89 - 111. ISSN: 2452-3062

Downloads: (external link)
http://eprints.lse.ac.uk/102136/ Open access version. (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:ehl:lserod:102136

Access Statistics for this paper

More papers in LSE Research Online Documents on Economics from London School of Economics and Political Science, LSE Library LSE Library Portugal Street London, WC2A 2HD, U.K.. Contact information at EDIRC.
Bibliographic data for series maintained by LSERO Manager ().