EconPapers    
Economics at your fingertips  
 

Evaluating the reliability of machine-learning-based predictions used in nuclear power plant instrumentation and control systems

Edward Chen, Han Bao and Nam Dinh

Reliability Engineering and System Safety, 2024, vol. 250, issue C

Abstract: The field of data-driven, neural-network-based machine learning (ML) has seen significant growth, with applications in various information and control systems. Despite promising real-world uses, the reliability of models remains questionable. Conventionally, reliability is assessed based on predictive fidelity, accuracy, and training effectiveness; however, quality developmental procedures and excellent training performance metrics do not guarantee operational reliability. Instead, an ML model's predictive performance depends on the training set's representativeness to the intended operational space. It is known that ML algorithms excel at interpolation but struggle with extrapolation tasks. Anomalies and feature drift can also reduce operational performance. Determining whether a new sample is an interpolation or extrapolation task involves out-of-distribution (OOD) detection for assessing its proximity to the existing training data. Thus, we present a real-time, model-agnostic individual prediction reliability evaluation method called Data Auditing for Reliability Evaluation (DARE) for applying OOD detection to the training dataset. We demonstrate on a feedforward neural network ML-integrated digital twin for predicting fuel centerline temperatures during loss-of-flow transients. DARE acts as a “data supervisor†in determining the model's applicability under different operating conditions. In this manner, we demonstrate how training data can serve as inductive evidence to support the reliability of ML predictions.

Keywords: Machine learning; Reliability; Trustworthiness; Out-of-distribution detection (search for similar items in EconPapers)
Date: 2024
References: Add references at CitEc
Citations:

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0951832024003387
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:reensy:v:250:y:2024:i:c:s0951832024003387

DOI: 10.1016/j.ress.2024.110266

Access Statistics for this article

Reliability Engineering and System Safety is currently edited by Carlos Guedes Soares

More articles in Reliability Engineering and System Safety from Elsevier
Bibliographic data for series maintained by Catherine Liu (repec@elsevier.com).

 
Page updated 2024-12-28
Handle: RePEc:eee:reensy:v:250:y:2024:i:c:s0951832024003387