Robustness Learning via Inference-Softmax Cross Entropy in Misaligned Distribution of Image

Bingbing Song, Ruxin Wang, Wei He and Wei Zhou ()
Additional contact information
Bingbing Song: School of Information Science and Engineering, Yunnan University, Kunming 650091, China
Ruxin Wang: Piolet School of Software, Yunnan University, Kunming 650091, China
Wei He: Piolet School of Software, Yunnan University, Kunming 650091, China
Wei Zhou: Piolet School of Software, Yunnan University, Kunming 650091, China

Mathematics, 2022, vol. 10, issue 19, 1-18

Abstract: Adversarial examples easily mislead vision systems based on deep neural networks (DNNs) trained with softmax cross entropy (SCE) loss. The vulnerability of DNN comes from the fact that SCE drives DNNs to fit on the training examples, whereas the resultant feature distributions between the training and adversarial examples are unfortunately misaligned. Several state-of-the-art methods start from improving the inter-class separability of training examples by modifying loss functions, where we argue that the adversarial examples are ignored, thus resulting in a limited robustness to adversarial attacks. In this paper, we exploited the inference region, which inspired us to apply margin-like inference information to SCE, resulting in a novel inference-softmax cross entropy (I-SCE) loss, which is intuitively appealing and interpretable. The inference information guarantees that it is difficult for neural networks to cross the decision boundary under an adversarial attack, and guarantees both the inter-class separability and the improved generalization to adversarial examples, which was further demonstrated and proved under the min-max framework. Extensive experiments show that the DNN models trained with the proposed I-SCE loss achieve a superior performance and robustness over the state-of-the-arts under different prevalent adversarial attacks; for example, the accuracy of I-SCE is 63% higher than SCE under the P G D 50 u n attack on the MNIST dataset. These experiments also show that the inference region can effectively solve the misaligned distribution.

Keywords: neural networks; robustness learning; loss function; adversarial examples (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2022
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2227-7390/10/19/3716/pdf (application/pdf)
https://www.mdpi.com/2227-7390/10/19/3716/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:10:y:2022:i:19:p:3716-:d:938320

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().