Coherent quality management for big data systems: a dynamic approach for stochastic time consistency

Yi-Ting Chen, Edward Sun and Yi-Bing Lin
Additional contact information
Yi-Ting Chen: University of Mannheim
Yi-Bing Lin: National Chiao Tung University (NCTU)

Annals of Operations Research, 2019, vol. 277, issue 1, No 2, 3-32

Abstract: Abstract Big data systems for reinforcement learning have often exhibited problems (e.g., failures or errors) when their components involve stochastic nature with the continuous control actions of reliability and quality. The complexity of big data systems and their stochastic features raise the challenge of uncertainty. This article proposes a dynamic coherent quality measure focusing on an axiomatic framework by characterizing the probability of critical errors that can be used to evaluate if the conveyed information of big data interacts efficiently with the integrated system (i.e., system of systems) to achieve desired performance. Herein, we consider two new measures that compute the higher-than-expected error,—that is, the tail error and its conditional expectation of the excessive error (conditional tail error)—as a quality measure of a big data system. We illustrate several properties (that suffice stochastic time-invariance) of the proposed dynamic coherent quality measure for a big data system. We apply the proposed measures in an empirical study with three wavelet-based big data systems in monitoring and forecasting electricity demand to conduct the reliability and quality management in terms of minimizing decision-making errors. Performance of using our approach in the assessment illustrates its superiority and confirms the efficiency and robustness of the proposed method.

Keywords: Big data; Dynamic coherent measure; Optimal decision; Quality management; Time consistency (search for similar items in EconPapers)
JEL-codes: C02 C10 C63 (search for similar items in EconPapers)
Date: 2019
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (5)

Downloads: (external link)
http://link.springer.com/10.1007/s10479-018-2795-1 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:annopr:v:277:y:2019:i:1:d:10.1007_s10479-018-2795-1

Ordering information: This journal article can be ordered from
http://www.springer.com/journal/10479

DOI: 10.1007/s10479-018-2795-1

Access Statistics for this article

Annals of Operations Research is currently edited by Endre Boros

More articles in Annals of Operations Research from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().