K-Fold Cross-Validation is Superior to Split Sample Validation for Risk Adjustment Models

Randall Ellis and Pooja Mookim

No wp2013-026, Boston University - Department of Economics - Working Papers Series from Boston University - Department of Economics

Abstract: This paper examines cross-validation techniques, with a particular focus on assessing thepredictive validity of risk adjustment models as commonly estimated. We validate that K-Fold cross-validation is more efficient than a 50-50 split sample and illustrate that overfitting with rich risk adjustment models remains meaningful even in samples of a million observations. A new estimation algorithm is described that efficiently calculates K-Fold cross-validated R-squared and other measures of goodness of fit using only three (XXX verify) passes through the data, and hence can be applied easily on sample sizes in the millions without sorting or relying on repeated split-sample techniques. Analysis of K-fold cross-validation results using a large claims dataset is used to calculate the standard deviation and bias of fitted R-squares for different models and sample sizes, which have a larger bias in moderately large sample sizes than most researchers would realize. Programs that implement the algorithm in SAS and STATA are presented that can be easily used on any sample.

Pages: 23
Date: 2013-06-07
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://www.bu.edu/econ/files/2016/01/Ellis_Mookim_R2paper_20130605.pdf
Our link check indicates that this URL is bad, the error code is: 403 Forbidden (http://www.bu.edu/econ/files/2016/01/Ellis_Mookim_R2paper_20130605.pdf [301 Moved Permanently]--> https://www.bu.edu/econ/files/2016/01/Ellis_Mookim_R2paper_20130605.pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:bos:wpaper:wp2013-026

Access Statistics for this paper

More papers in Boston University - Department of Economics - Working Papers Series from Boston University - Department of Economics Contact information at EDIRC.
Bibliographic data for series maintained by Program Coordinator ().