Assessing the predictive ability of a multilevel binary regression model

R. Van Oirbeek and E. Lesaffre

Computational Statistics & Data Analysis, 2012, vol. 56, issue 6, 1966-1980

Abstract: An adaptation of the Brier score and the concordance probability is proposed for the two-level and the three-level random intercept binary regression model. This results in 2 different Brier scores and 3 different C-indices for the two-level binary regression model and 4 different Brier scores and 7 different C-indices for the three-level binary regression model. The ensemble of these measures offers a better view on how the different elements of the random effects model, i.e. the covariates and the random effects, affect the predictive ability of the model separately, evaluated on a within-cluster, between-cluster and global level. For all measures, an estimation procedure using Bayesian and likelihood estimation methods was developed, including a percentile and a BCa non-parametric bootstrap step to construct credible/confidence intervals. In a simulation study, the likelihood estimation procedure showed difficulties in estimating unbiasedly the predictive ability of the random effects, while the Bayesian estimation procedure resulted in good estimation properties for all of the developed measures. The BCa non-parametric bootstrap method resulted in confidence/credible intervals with better coverage properties than the percentile non-parametric bootstrap method. The proposals are applied to a real-life binary data set with a three-level structure using the Bayesian estimation procedure.

Keywords: Brier score; C-index; Concordance probability; Multilevel; Binary regression (search for similar items in EconPapers)
Date: 2012
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0167947311004221
Full text for ScienceDirect subscribers only.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:csdana:v:56:y:2012:i:6:p:1966-1980

DOI: 10.1016/j.csda.2011.11.023

Access Statistics for this article

Computational Statistics & Data Analysis is currently edited by S.P. Azen

More articles in Computational Statistics & Data Analysis from Elsevier
Bibliographic data for series maintained by Catherine Liu ().