Adaptive multilayer extreme learning machines

Christos K. Filelis-Papadopoulos, John P. Morrison and O’Reilly, Philip

Mathematics and Computers in Simulation (MATCOM), 2025, vol. 231, issue C, 71-98

Abstract: Extreme learning machines is a neural network type that has been utilized in tasks such as regression and classification, due to their efficient training process, which is based on pseudoinverse matrices and randomized weights, avoiding the computationally intensive backpropagation. In order to further improve their performance and reduce their complexity with respect to number of required hyperparameters, especially in the case of multiple layer architectures, a novel multilayer adaptive approach, based on residual networks, is proposed. This approach constructs the network iteratively with respect to error minimization and parsimony using a recursive pseudoinverse matrix framework. A new block approach, using mixed precision arithmetic and Graphics Processing Units (GPU) is proposed. The proposed technique is coupled with a new adaptive penalty criterion to ensure adequate numbers of neurons are included in each layer, while avoiding highly correlated basis. Adaptive regularization, along with scaling, is also incorporated to ensure Symmetric Positive Definiteness (SPD) of the Gram matrix. Several random number distributions for the proposed approach are examined and discussed. Handling of large datasets is discussed and a new batch variant is proposed. The proposed scheme is evaluated for regression and classification tasks in a multitude of datasets and is compared with other neural network architectures.

Keywords: Recursive pseudoinverse matrix; Modeling; Multilayer Extreme Learning Machine; Graphics Processing Unit (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0378475424004750
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:matcom:v:231:y:2025:i:c:p:71-98

DOI: 10.1016/j.matcom.2024.12.004

Access Statistics for this article

Mathematics and Computers in Simulation (MATCOM) is currently edited by Robert Beauwens

More articles in Mathematics and Computers in Simulation (MATCOM) from Elsevier
Bibliographic data for series maintained by Catherine Liu ().