A harmonic framework for stepsize selection in gradient methods

Giulia Ferrandi (), Michiel E. Hochstenbach () and Nataša Krejić ()
Additional contact information
Giulia Ferrandi: TU Eindhoven
Michiel E. Hochstenbach: TU Eindhoven
Nataša Krejić: University of Novi Sad

Computational Optimization and Applications, 2023, vol. 85, issue 1, No 3, 75-106

Abstract: Abstract We study the use of inverse harmonic Rayleigh quotients with target for the stepsize selection in gradient methods for nonlinear unconstrained optimization problems. This not only provides an elegant and flexible framework to parametrize and reinterpret existing stepsize schemes, but it also gives inspiration for new flexible and tunable families of steplengths. In particular, we analyze and extend the adaptive Barzilai–Borwein method to a new family of stepsizes. While this family exploits negative values for the target, we also consider positive targets. We present a convergence analysis for quadratic problems extending results by Dai and Liao (IMA J Numer Anal 22(1):1–10, 2002), and carry out experiments outlining the potential of the approaches.

Keywords: Unconstrained optimization; Harmonic Rayleigh quotient; Gradient methods; Framework for steplength selection; ABB method; Hessian spectral properties; 65K05; 90C20; 90C30; 65F15; 65F10 (search for similar items in EconPapers)
Date: 2023
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
http://link.springer.com/10.1007/s10589-023-00455-6 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:coopap:v:85:y:2023:i:1:d:10.1007_s10589-023-00455-6

Ordering information: This journal article can be ordered from
http://www.springer.com/math/journal/10589

DOI: 10.1007/s10589-023-00455-6

Access Statistics for this article

Computational Optimization and Applications is currently edited by William W. Hager

More articles in Computational Optimization and Applications from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().