An Adaptive Low Computational Cost Alternating Direction Method of Multiplier for RELM Large-Scale Distributed Optimization

Ke Wang, Shanshan Huo, Banteng Liu (), Zhangquan Wang and Tiaojuan Ren
Additional contact information
Ke Wang: College of Infomation Science and Technology, Zhejiang Shuren University, Hangzhou 310015, China
Shanshan Huo: School of Computer Science and Artificial Intelligence, Changzhou University, Changzhou 213164, China
Banteng Liu: College of Infomation Science and Technology, Zhejiang Shuren University, Hangzhou 310015, China
Zhangquan Wang: College of Infomation Science and Technology, Zhejiang Shuren University, Hangzhou 310015, China
Tiaojuan Ren: College of Infomation Science and Technology, Zhejiang Shuren University, Hangzhou 310015, China

Mathematics, 2023, vol. 12, issue 1, 1-20

Abstract: In a class of large-scale distributed optimization, the calculation of RELM based on the Moore–Penrose inverse matrix is prohibitively expensive, which hinders the formulation of a computationally efficient optimization model. Attempting to improve the model’s convergence performance, this paper proposes a low computing cost Alternating Direction Method of Multipliers (ADMM), where the original update in ADMM is solved inexactly with approximate curvature information. Based on quasi-Newton techniques, the ADMM approach allows us to solve convex optimization with reasonable accuracy and computational effort. By introducing this algorithm into the RELM model, the model fitting problem can be decomposed into a set of subproblems that can be executed in parallel to achieve efficient classification performance. To avoid the storage of expensive Hessian for large problems, BFGS with limited memory is proposed with computational efficiency. And the optimal parameter values of the step-size search method are obtained through Wolfe line search strategy. To demonstrate the superiority of our methods, numerical experiments are conducted on eight real-world datasets. Results on problems arising in machine learning suggest that the proposed method is competitive with other similar methods, both in terms of better computational efficiency as well as accuracy.

Keywords: extreme learning machines; alternating direction method of multipliers; matrix calculation; convex optimization (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2023
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2227-7390/12/1/43/pdf (application/pdf)
https://www.mdpi.com/2227-7390/12/1/43/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:12:y:2023:i:1:p:43-:d:1305899

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().