Deep probabilistic graphical modeling for robust multivariate time series anomaly detection with missing data

Jingyu Yang, Zuogong Yue and Ye Yuan

Reliability Engineering and System Safety, 2023, vol. 238, issue C

Abstract: Multivariate time series anomaly detection with missing data is one of the most pending issues for industrial monitoring. Due to scarcity of labeled anomalies, most advanced data-driven anomaly detection approaches fall in the unsupervised learning paradigm. As a premise in the presence of missing data, one needs to improve the data quality through data imputation with a separate model. Our concern lies in the consistency between data imputation and unsupervised learning for robust anomaly detection, regarding accurately discovering the spatiotemporal dependence among multiple variables over time. However, the existing practice tends to overlook this consistency and decouple the training process for these two closely linked tasks. This article novelly proposes a probabilistic multivariate time series anomaly detection framework that unifies data imputation and unsupervised learning. A deep probabilistic graphical model abbreviated SCNF is first devised for unsupervised density estimation. A tailored expectation maximization-based optimization scheme is then developed to achieve the joint training of data imputation and unsupervised learning with missing data. The efficacy is experimentally corroborated in several industrial applications, including chemical process, water treatment and network traffic. Briefly, the joint training framework enhances the AUROC of SCNF by averagely 6.34% for three applications under 50% data missing rate.

Keywords: Multivariate time series; Anomaly detection; Missing data; Probabilistic graphical model; Expectation maximization (search for similar items in EconPapers)
Date: 2023
References: Add references at CitEc
Citations: View citations in EconPapers (4)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0951832023003241
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:reensy:v:238:y:2023:i:c:s0951832023003241

DOI: 10.1016/j.ress.2023.109410

Access Statistics for this article

Reliability Engineering and System Safety is currently edited by Carlos Guedes Soares

More articles in Reliability Engineering and System Safety from Elsevier
Bibliographic data for series maintained by Catherine Liu ().