A Heterogeneous Federated Transfer Learning Approach with Extreme Aggregation and Speed

Tarek Berghout, Toufik Bentrcia, Mohamed Amine Ferrag () and Mohamed Benbouzid
Additional contact information
Tarek Berghout: Laboratory of Automation and Manufacturing Engineering, University of Batna 2, Batna 05000, Algeria
Toufik Bentrcia: Laboratory of Automation and Manufacturing Engineering, University of Batna 2, Batna 05000, Algeria
Mohamed Amine Ferrag: Department of Computer Science, University of Guelma, Guelma 24000, Algeria
Mohamed Benbouzid: Institut de Recherche Dupuy de Lôme (UMR CNRS 6027), University of Brest, 29238 Brest, France

Mathematics, 2022, vol. 10, issue 19, 1-16

Abstract: Federated learning (FL) is a data-privacy-preserving, decentralized process that allows local edge devices of smart infrastructures to train a collaborative model independently while keeping data localized. FL algorithms, encompassing a well-structured average of the training parameters (e.g., the weights and biases resulting from training-based stochastic gradient descent variants), are subject to many challenges, namely expensive communication, systems heterogeneity, statistical heterogeneity, and privacy concerns. In this context, our paper targets the four aforementioned challenges while focusing on reducing communication and computational costs by involving recursive least squares (RLS) training rules. Accordingly, to the best of our knowledge, this is the first time that the RLS algorithm is modified to completely accommodate non-independent and identically distributed data (non-IID) for federated transfer learning (FTL). Furthermore, this paper also introduces a newly generated dataset capable of emulating such real conditions and of making data investigation available on ordinary commercial computers with quad-core microprocessors and less need for higher computing hardware. Applications of FTL-RLS on the generated data under different levels of complexity closely related to different levels of cardinality lead to a variety of conclusions supporting its performance for future uses.

Keywords: federated learning; federated transfer learning; heterogeneous systems; non identical independent data; recursive least squares (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2022
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2227-7390/10/19/3528/pdf (application/pdf)
https://www.mdpi.com/2227-7390/10/19/3528/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:10:y:2022:i:19:p:3528-:d:927530

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().