Intermediate Gradient Methods with Relative Inexactness

Nikita Kornilov (), Mohammad Alkousa (), Eduard Gorbunov, Fedor Stonyakin, Pavel Dvurechensky and Alexander Gasnikov
Additional contact information
Nikita Kornilov: Moscow Institute of Physics and Technology
Mohammad Alkousa: Innopolis University
Eduard Gorbunov: Mohamed bin Zayed University of Artificial Intelligence
Fedor Stonyakin: Moscow Institute of Physics and Technology
Pavel Dvurechensky: Weierstrass Institute for Applied Analysis and Stochastics
Alexander Gasnikov: Moscow Institute of Physics and Technology

Journal of Optimization Theory and Applications, 2025, vol. 207, issue 3, No 22, 42 pages

Abstract: Abstract This paper is devoted to studying first-order methods for smooth convex optimization with inexact gradients. Unlike the majority of the literature on this topic, we consider the setting of relative rather than absolute inexactness. More precisely, we assume that the additive error in the gradient is proportional to the gradient norm, rather than being globally bounded by some small quantity. We propose a novel analysis of the accelerated gradient method under relative inexactness and strong convexity, improving the bound on the maximum admissible error that preserves the algorithm’s linear convergence. In other words, we analyze the robustness of the accelerated gradient method to relative gradient inexactness. Furthermore, using the Performance Estimation Problem (PEP) technique, we demonstrate that the obtained result is tight up to a numerical constant. Motivated by existing intermediate methods with absolute error, i.e., methods which convergence rates interpolate between the slower but more robust non-accelerated algorithms and the faster yet less robust accelerated algorithms, we propose an adaptive variant of the intermediate gradient method with relative gradient error.

Keywords: Accelerated methods; Intermediate method; Inexact gradient; Relative noise; Performance Estimation Problem; 90C25; 90C30; 90C06 (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s10957-025-02809-y Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:joptap:v:207:y:2025:i:3:d:10.1007_s10957-025-02809-y

Ordering information: This journal article can be ordered from
http://www.springer. ... cs/journal/10957/PS2

DOI: 10.1007/s10957-025-02809-y

Access Statistics for this article

Journal of Optimization Theory and Applications is currently edited by Franco Giannessi and David G. Hull

More articles in Journal of Optimization Theory and Applications from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().