Out-of-Distribution Detection with Memory-Augmented Variational Autoencoder

Faezeh Ataeiasad, David Elizondo, Saúl Calderón Ramírez, Sarah Greenfield () and Lipika Deka
Additional contact information
Faezeh Ataeiasad: School of Computer Science and Informatics, De Montfort University, The Gateway, Leicester LE1 9BH, UK
David Elizondo: School of Computer Science and Informatics, De Montfort University, The Gateway, Leicester LE1 9BH, UK
Saúl Calderón Ramírez: School of Computer Engineering, Costa Rica Institute of Technology (ITCR), Cartago 30109, Costa Rica
Sarah Greenfield: School of Computer Science and Informatics, De Montfort University, The Gateway, Leicester LE1 9BH, UK
Lipika Deka: School of Computer Science and Informatics, De Montfort University, The Gateway, Leicester LE1 9BH, UK

Mathematics, 2024, vol. 12, issue 19, 1-16

Abstract: This paper proposes a novel method capable of both detecting OOD data and generating in-distribution data samples. To achieve this, a VAE model is adopted and augmented with a memory module, providing capacities for identifying OOD data and synthesising new in-distribution samples. The proposed VAE is trained on normal data and the memory stores prototypical patterns of the normal data distribution. At test time, the input is encoded by the VAE encoder; this encoding is used as a query to retrieve related memory items, which are then integrated with the input encoding and passed to the decoder for reconstruction. Normal samples reconstruct well and yield low reconstruction errors, while OOD inputs produce high reconstruction errors as their encodings get replaced by retrieved normal patterns. Prior works use memory modules for OOD detection with autoencoders, but this method leverages a VAE architecture to enable generation abilities. Experiments conducted with CIFAR-10 and MNIST datasets show that the memory-augmented VAE consistently outperforms the baseline, particularly where OOD data resembles normal patterns. This notable improvement is due to the enhanced latent space representation provided by the VAE. Overall, the memory-equipped VAE framework excels in identifying OOD and generating creative examples effectively.

Keywords: out-of-distribution detection; variational autoencoder; memory-augmentation; machine learning; integrated external memory; PyTorch; deep learning (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2024
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2227-7390/12/19/3153/pdf (application/pdf)
https://www.mdpi.com/2227-7390/12/19/3153/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:12:y:2024:i:19:p:3153-:d:1494788

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().