Meta-Heuristic Optimization Methods for Quaternion-Valued Neural Networks

Jeremiah Bill, Lance Champagne, Bruce Cox and Trevor Bihl
Additional contact information
Jeremiah Bill: Air Force Institute of Technology, Department of Operational Sciences, WPAFB, OH 45433, USA
Lance Champagne: Air Force Institute of Technology, Department of Operational Sciences, WPAFB, OH 45433, USA
Bruce Cox: Air Force Institute of Technology, Department of Operational Sciences, WPAFB, OH 45433, USA
Trevor Bihl: Air Force Research Laboratory, Sensors Directorate, WPAFB, OH 45433, USA

Mathematics, 2021, vol. 9, issue 9, 1-23

Abstract: In recent years, real-valued neural networks have demonstrated promising, and often striking, results across a broad range of domains. This has driven a surge of applications utilizing high-dimensional datasets. While many techniques exist to alleviate issues of high-dimensionality, they all induce a cost in terms of network size or computational runtime. This work examines the use of quaternions, a form of hypercomplex numbers, in neural networks. The constructed networks demonstrate the ability of quaternions to encode high-dimensional data in an efficient neural network structure, showing that hypercomplex neural networks reduce the number of total trainable parameters compared to their real-valued equivalents. Finally, this work introduces a novel training algorithm using a meta-heuristic approach that bypasses the need for analytic quaternion loss or activation functions. This algorithm allows for a broader range of activation functions over current quaternion networks and presents a proof-of-concept for future work.

Keywords: multilayer perceptrons; quaternion neural networks; metaheuristic optimization; genetic algorithms (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2021
References: View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
https://www.mdpi.com/2227-7390/9/9/938/pdf (application/pdf)
https://www.mdpi.com/2227-7390/9/9/938/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:9:y:2021:i:9:p:938-:d:541819

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().