Metamorphic Testing of Image Classification and Consistency Analysis Using Clustering

Hemanth Gudaparthi, Prudhviraj Naidu and Nan Niu
Additional contact information
Hemanth Gudaparthi: University of Cincinnati, USA
Prudhviraj Naidu: University of Cincinnati, USA
Nan Niu: University of Cincinnati, USA

International Journal of Multimedia Data Engineering and Management (IJMDEM), 2022, vol. 13, issue 1, 1-20

Abstract: Testing deep learning systems requires expensive labeled data. In recent years, researchers began to leverage metamorphic testing to address this issue. However, metamorphic relations on image data remain poorly understood. To gain a deeper understanding of these metamorphic relations, we survey common image operations modeling covariate shift, manually classify and categorize the underlying metamorphic relations, and conduct experiments to validate our classifications. In our experiments, we train three popular convolutional neural network architectures on an image classification task. Next, we apply metamorphic operations on input test images and measure the change in classification accuracy and cross-entropy loss. A hierarchical clustering algorithm cluster these results and plots a dendrogram. We compare the groups from manual classification and the clusters from the algorithm to provide key insights. We find that Affine and Noise relations are consistent. Furthermore, we recommend metamorphic relationships to save time and better test deep learning systems in the future.

Date: 2022
References: Add references at CitEc
Citations:

Downloads: (external link)
http://services.igi-global.com/resolvedoi/resolve.aspx?doi=10.4018/IJMDEM.304390 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:igg:jmdem0:v:13:y:2022:i:1:p:1-20

Access Statistics for this article

International Journal of Multimedia Data Engineering and Management (IJMDEM) is currently edited by Chengcui Zhang

More articles in International Journal of Multimedia Data Engineering and Management (IJMDEM) from IGI Global
Bibliographic data for series maintained by Journal Editor ().