Data-Driven Model Evaluation: A Test for Revealed Performance

Jeffrey Racine and Christopher Parmeter

Department of Economics Working Papers from McMaster University

Abstract: When comparing two competing approximate models using a particular loss function, the one having smallest `expected true error' for that loss function is expected to lie closest to the underlying data generating process (DGP) given this loss function and is therefore to be preferred. In this chapter we consider a data-driven method for testing whether or not two competing approximate models are equivalent in terms of their expected true error (i.e., their expected performance on unseen data drawn from the same DGP). The proposed test is quite flexible with regards to the types of models that can be compared (i.e., nested versus non-nested, parametric versus nonparametric) and is applicable in cross-sectional and time-series settings. Moreover, in time-series settings our method overcomes two of the drawbacks associated with dominant approaches, namely, their reliance on only one split of the data and the need to have a sufficiently large `hold-out' sample for these tests to possess adequate power.

Keywords: approximate; misspecified; model selection; predictive accuracy; data mining (search for similar items in EconPapers)
Pages: 35 pages
Date: 2012-10
New Economics Papers: this item is included in nep-for
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (13)

Downloads: (external link)
http://socserv.mcmaster.ca/econ/rsrch/papers/archive/2012-13.pdf (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:mcm:deptwp:2012-13

Access Statistics for this paper

More papers in Department of Economics Working Papers from McMaster University Contact information at EDIRC.
Bibliographic data for series maintained by ().