Composing Diverse Ensembles of Convolutional Neural Networks by Penalization

Balazs Harangi (), Agnes Baran, Marcell Beregi-Kovacs and Andras Hajdu
Additional contact information
Balazs Harangi: Faculty of Informatics, University of Debrecen, H-4028 Debrecen, Hungary
Agnes Baran: Faculty of Informatics, University of Debrecen, H-4028 Debrecen, Hungary
Marcell Beregi-Kovacs: Faculty of Informatics, University of Debrecen, H-4028 Debrecen, Hungary
Andras Hajdu: Faculty of Informatics, University of Debrecen, H-4028 Debrecen, Hungary

Mathematics, 2023, vol. 11, issue 23, 1-19

Abstract: Ensemble-based systems are well known to have the capacity to outperform individual approaches if the ensemble members are sufficiently accurate and diverse. This paper investigates how an efficient ensemble of deep convolutional neural networks (CNNs) can be created by forcing them to adjust their parameters during the training process to increase diversity in their decisions. As a new theoretical approach to reach this aim, we join the member neural architectures via a fully connected layer and insert a new correlation penalty term in the loss function to obstruct their similar operation. With this complementary term, we implement the standard guideline of ensemble creation to increase the members’ diversity for CNNs in a more detailed and flexible way than similar existing techniques. As for applicability, we show that our approach can be efficiently used in various classification tasks. More specifically, we demonstrate its performance in challenging medical image analysis and natural image classification problems. Besides the theoretical considerations and foundations, our experimental findings suggest that the proposed technique is competitive. Namely, on the one hand, the classification rate of the ensemble trained in this way outperformed all the individual accuracies of the state-of-the-art member CNNs according to the standard error functions of these application domains. On the other hand, it is also validated that the ensemble members get more diverse and their accuracies are raised by adding the penalization term. Moreover, we performed a full comparative analysis, including other state-of-the-art ensemble-based approaches recommended for the same classification tasks. This comparative study also confirmed the superiority of our method, as it overcame the current solutions.

Keywords: ensemble-based network; penalization; loss function; image classification; diversity (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2023
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2227-7390/11/23/4730/pdf (application/pdf)
https://www.mdpi.com/2227-7390/11/23/4730/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:11:y:2023:i:23:p:4730-:d:1285583

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().