Methods for analytical validation of novel digital clinical measures: A simulation study

Simon Turner, Chen Chen, Rolando Acosta, Rachell Chon, Eric J Daza, Lysbeth Floden, Joss Langford, Leif Simmatis, Berend Terluin, Benjamin Vandendriessche and Piper Fromy

PLOS ONE, 2026, vol. 21, issue 5, 1-21

Abstract: Analytical validation is a crucial step in the evaluation of algorithms that process data from sensor-based digital health technologies (sDHTs). Analytical validation of novel digital measures can be complicated when reference measures with directly comparable units are not available. To address this, we conducted a simulation study. Data was simulated assuming a latent physical ability trait, indirectly accessed through an sDHT-derived target measure collecting step count data, and the items of a clinical outcome assessment (COA) measuring self-reported physical activity. We quantified the ability of two methods to assess the latent relationship between reference and target measures: the Pearson Correlation Coefficient (PCC) and factor correlations from a two-factor confirmatory factor analysis (CFA) model. Additionally, three multiple linear regression models were used to evaluate if multiple COA reference measures can more completely represent a target measure of interest. Our findings show that PCC was more stable, easier to compute, and relatively robust with respect to violations of parametric assumptions than CFA, particularly with small sample sizes. However, CFA was less biased than PCC in all scenarios investigated. We demonstrate that using both PCC and CFA generates more confidence in the results of a target and reference measure comparison. Finally, regression results suggest that incorporating multiple reference measures with more frequent collection time points can provide a more complete presentation of the sDHT’s analytical validity. Novel digital measures are being developed at an accelerating pace and promise to revolutionize patient care and medical product development. Our findings provide investigators with crucial information for choosing appropriate methods to perform rigorous analytical validation of these novel measures, including an open-access simulation toolkit.

Date: 2026
References: Add references at CitEc
Citations:

Downloads: (external link)
https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0308190 (text/html)
https://journals.plos.org/plosone/article/file?id= ... 08190&type=printable (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:plo:pone00:0308190

DOI: 10.1371/journal.pone.0308190

Access Statistics for this article

More articles in PLOS ONE from Public Library of Science
Bibliographic data for series maintained by plosone ().