New validation metrics for models with multiple correlated responses

Wei Li, Wei Chen, Zhen Jiang, Zhenzhou Lu and Yu Liu

Reliability Engineering and System Safety, 2014, vol. 127, issue C, 1-11

Abstract: Validating models with correlated multivariate outputs involves the comparison of multiple stochastic quantities. Considering both uncertainty and correlations among multiple responses from model and physical observations imposes challenges. Existing marginal comparison methods and the hypothesis testing-based methods either ignore correlations among responses or only reach Boolean conclusions (yes or no) without accounting for the amount of discrepancy between a model and the underlying reality. A new validation metric is needed to quantitatively characterize the overall agreement of multiple responses considering correlations among responses and uncertainty in both model predictions and physical observations. In this paper, by extending the concept of “area metric†and the “u-pooling method†developed for validating a single response, we propose new model validation metrics for validating correlated multiple responses using the multivariate probability integral transformation (PIT). One new metric is the PIT area metric for validating multi-responses at a single validation site. The other is the t-pooling metric that allows for pooling observations of multiple responses observed at multiple validation sites to assess the global predictive capability. The proposed metrics have many favorable properties that are well suited for validation assessment of models with correlated responses. The two metrics are examined and compared with the direct area metric and the marginal u-pooling method respectively through numerical case studies and an engineering example to illustrate their validity and potential benefits.

Keywords: Model validation; Uncertainty; Correlation; Area metric; Multivariate probability integral transformation; Multi-response (search for similar items in EconPapers)
Date: 2014
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (7)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0951832014000313
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:reensy:v:127:y:2014:i:c:p:1-11

DOI: 10.1016/j.ress.2014.02.002

Access Statistics for this article

Reliability Engineering and System Safety is currently edited by Carlos Guedes Soares

More articles in Reliability Engineering and System Safety from Elsevier
Bibliographic data for series maintained by Catherine Liu ().