Advanced Optimization Techniques for Federated Learning on Non-IID Data

Filippos Efthymiadis, Aristeidis Karras (), Christos Karras () and Spyros Sioutas
Additional contact information
Filippos Efthymiadis: Computer Engineering and Informatics Department, University of Patras, 26504 Patras, Greece
Aristeidis Karras: Computer Engineering and Informatics Department, University of Patras, 26504 Patras, Greece
Christos Karras: Computer Engineering and Informatics Department, University of Patras, 26504 Patras, Greece
Spyros Sioutas: Computer Engineering and Informatics Department, University of Patras, 26504 Patras, Greece

Future Internet, 2024, vol. 16, issue 10, 1-31

Abstract: Federated learning enables model training on multiple clients locally, without the need to transfer their data to a central server, thus ensuring data privacy. In this paper, we investigate the impact of Non-Independent and Identically Distributed (non-IID) data on the performance of federated training, where we find a reduction in accuracy of up to 29% for neural networks trained in environments with skewed non-IID data. Two optimization strategies are presented to address this issue. The first strategy focuses on applying a cyclical learning rate to determine the learning rate during federated training, while the second strategy develops a sharing and pre-training method on augmented data in order to improve the efficiency of the algorithm in the case of non-IID data. By combining these two methods, experiments show that the accuracy on the CIFAR-10 dataset increased by about 36% while achieving faster convergence by reducing the number of required communication rounds by 5.33 times. The proposed techniques lead to improved accuracy and faster model convergence, thus representing a significant advance in the field of federated learning and facilitating its application to real-world scenarios.

Keywords: federated learning; optimization strategies; non-IID data; deep learning; cyclical learning rate; pre-training; augmented data; faster model convergence; IoT; decision making (search for similar items in EconPapers)
JEL-codes: O3 (search for similar items in EconPapers)
Date: 2024
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/1999-5903/16/10/370/pdf (application/pdf)
https://www.mdpi.com/1999-5903/16/10/370/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jftint:v:16:y:2024:i:10:p:370-:d:1497836

Access Statistics for this article

Future Internet is currently edited by Ms. Grace You

More articles in Future Internet from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().