Contextual anomaly detection for high-dimensional data using Dirichlet process variational autoencoder

Hyojoong Kim and Heeyoung Kim

IISE Transactions, 2023, vol. 55, issue 5, 433-444

Abstract: Due to recent advances in sensing technologies, response measurements of various sensors are frequently used for system monitoring purposes. However, response data are often affected by some contextual variables, such as equipment settings and time, resulting in different patterns, even when the system is in the normal state. In this case, anomaly detection methods that do not consider contextual variables may be unable to distinguish between abnormal and normal patterns of the response data affected by the contextual variables. Motivated by this problem, we propose a method for contextual anomaly detection, particularly in the case where the response and contextual variables are both high-dimensional and complex. The proposed method is based on Variational AutoEncoders (VAEs), which are neural-network-based generative models suitable for modeling high-dimensional and complex data. The proposed method combines two VAEs: one for response variables and the other for contextual variables. Specifically, in the latent space of the VAE for contextual variables, we model the latent variables using a Dirichlet process Gaussian mixture model. Consequently, the effects of the contextual variables can be modeled using several clusters, each representing a different contextual environment. The latent contextual variables are then used as additional inputs to the other VAE’s decoder for reconstructing response data from their latent representations. We then detect the anomalies based on the negative reconstruction loss of a new response observation. The effectiveness of the proposed method is demonstrated using several benchmark datasets and a case study based on a global tire company.

Date: 2023
References: Add references at CitEc
Citations:

Downloads: (external link)
http://hdl.handle.net/10.1080/24725854.2021.2024925 (text/html)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:taf:uiiexx:v:55:y:2023:i:5:p:433-444

Ordering information: This journal article can be ordered from
http://www.tandfonline.com/pricing/journal/uiie20

DOI: 10.1080/24725854.2021.2024925

Access Statistics for this article

IISE Transactions is currently edited by Jianjun Shi

More articles in IISE Transactions from Taylor & Francis Journals
Bibliographic data for series maintained by Chris Longhurst ().