Improving the Performance of Optimization Algorithms Using the Adaptive Fixed-Time Scheme and Reset Scheme

Yuquan Chen (), Yunkang Sun and Bing Wang
Additional contact information
Yuquan Chen: School of Artificial Intelligence and Automation, Hohai University, Nanjing 210024, China
Yunkang Sun: School of Artificial Intelligence and Automation, Hohai University, Nanjing 210024, China
Bing Wang: School of Artificial Intelligence and Automation, Hohai University, Nanjing 210024, China

Mathematics, 2023, vol. 11, issue 22, 1-16

Abstract: Optimization algorithms have now played an important role in many fields, and the issue of how to design high-efficiency algorithms has gained increasing attention, for which it has been shown that advanced control theories could be helpful. In this paper, the fixed-time scheme and reset scheme are introduced to design high-efficiency gradient descent methods for unconstrained convex optimization problems. At first, a general reset framework for existing accelerated gradient descent methods is given based on the systematic representation, with which both convergence speed and stability are significantly improved. Then, the design of a novel adaptive fixed-time gradient descent, which has fewer tuning parameters and maintains better robustness to initial conditions, is presented. However, its discrete form introduces undesirable overshoot and easily leads to instability, and the reset scheme is then applied to overcome the drawbacks. The linear convergence and better stability of the proposed algorithms are theoretically proven, and several dedicated simulation examples are finally given to validate the effectiveness.

Keywords: optimization algorithms; reset scheme; accelerated gradient descent; fixed-time convergence; adaptive learning rate (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2023
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2227-7390/11/22/4704/pdf (application/pdf)
https://www.mdpi.com/2227-7390/11/22/4704/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:11:y:2023:i:22:p:4704-:d:1284023

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().