Beyond the AUC: A comprehensive workflow in Stata for evaluating clinical prediction models—application to a multicenter cardiovascular risk model

Shengnan Li
Additional contact information
Shengnan Li: Instituto Ramón y Cajal investigación sanitaria (IRYCIS)

Spanish Stata Conference 2026 from Stata Users Group

Abstract: Background: With the increasing use of clinical prediction models in medical research, evaluating model performance has become an essential component of the development and validation process. However, most studies still rely primarily on the area under the ROC curve (AUC) to assess model performance, paying less attention to key indicators such as calibration, incremental model improvement, clinical value, and external generalizability. Guidelines for predictive model evaluation, such as TRIPOD, recommend a multidimensional assessment; however, in Stata, these analyses typically require multiple independent commands. Objective: To develop a comprehensive Stata workflow for evaluating clinical prediction models and demonstrate its application using a real-world multicenter dataset, thereby providing a standardized and reproducible analytical framework for predictive model evaluation. Methods: An analysis workflow was constructed covering the main dimensions of predictive model evaluation, including discrimination (ROC curve and AUC), calibration (calibration plots, calibration in the large [CITL], and calibration slope), incremental predictive value (net reclassification improvement [NRI] and integrated discrimination improvement [IDI]), clinical utility (decision curve analysis [DCA]), and model validation. A multicenter cohort comprising 3,169 patients from 5 European countries was used to build a perioperative cardiovascular risk prediction model based on the revised cardiac risk index (RCRI) combined with NT-proBNP; this workflow was then applied to systematically evaluate the model's performance. Results: The entire evaluation process was conducted within the Stata environment, achieving a standardized and reproducible assessment of model performance. Following the inclusion of NT-proBNP, the model's discriminative ability improved compared with the base model (AUC increased from 0.58 to 0.67). The NRI was 0.38 and the IDI was 0.02, suggesting an improvement in patient risk reclassification. Calibration analysis, clinical decision curve analysis, and external validation provided additional relevant information that cannot be captured by the AUC alone. Conclusion: This study establishes a comprehensive workflow in Stata for evaluating clinical prediction models, integrating discriminative ability, calibration, incremental improvement, clinical utility, and model validation into a unified analytical framework. This approach enables a more standardized and reproducible assessment of predictive models, facilitates the practical application of TRIPOD recommendations, and provides a useful methodological tool for multicenter studies of clinical prediction models.

Date: 2026-10-31
References: Add references at CitEc
Citations:

There are no downloads for this item, see the EconPapers FAQ for hints about obtaining it.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:boc:spai26:04

Access Statistics for this paper

More papers in Spanish Stata Conference 2026 from Stata Users Group Contact information at EDIRC.
Bibliographic data for series maintained by Christopher F Baum ().