Revisiting Nesterov’s acceleration via high-resolution differential equations

Shuo Chen (), Bin Shi () and Ya-xiang Yuan ()
Additional contact information
Shuo Chen: Chinese Academy of Sciences
Bin Shi: Center for Mathematics and Interdisciplinary Sciences
Ya-xiang Yuan: Chinese Academy of Sciences

Journal of Global Optimization, 2025, vol. 93, issue 2, No 9, 569 pages

Abstract: Abstract Nesterov’s accelerated gradient descent (NAG) stands as a landmark in the development of first-order optimization algorithms. Its acceleration mechanism, which originates from a gradient correction term, was elucidated through the high-resolution ordinary differential equation (ODE) framework, as referenced in [1]. This framework has been vital in demystifying the effectiveness of NAG. Moreover, it is worth noting that the construction of Lyapunov functions within this framework is methodical and principled. In this paper, we leverage this framework to conduct an in-depth analysis of the convergence properties of NAG for $$\mu $$ μ -strongly convex functions. First, we refine the proof of the gradient-correction scheme, streamlining the process with straightforward calculations akin to that in [2]. This also allows us to enlarge the step size to $$s=1/L$$ s = 1 / L with only slight modifications. Furthermore, our analysis via the implicit-velocity scheme reveals that its associated Lyapunov function is more succinct to construct, as it simplifies the structure and eases the computation of the iterative difference. This resulting simplicity, coupled with the optimal step size derived, indicates the superiority of the implicit velocity scheme over the gradient correction scheme within the high-resolution ODE framework.

Keywords: NAG; High-resolution ODE; Lyapunov function; Gradient-correction scheme; implicit-velocity scheme; $$\mu $$ μ -strongly convex function; 34E10; 68Q25; 65K05; 90C25; 90C30 (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s10898-025-01543-z Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:jglopt:v:93:y:2025:i:2:d:10.1007_s10898-025-01543-z

Ordering information: This journal article can be ordered from
http://www.springer. ... search/journal/10898

DOI: 10.1007/s10898-025-01543-z

Access Statistics for this article

Journal of Global Optimization is currently edited by Sergiy Butenko

More articles in Journal of Global Optimization from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().