Stochastic ISTA/FISTA Adaptive Step Search Algorithms for Convex Composite Optimization

Lam M. Nguyen (), Katya Scheinberg () and Trang H. Tran ()
Additional contact information
Lam M. Nguyen: IBM Research
Katya Scheinberg: Georgia Institute of Technology
Trang H. Tran: Cornell University

Journal of Optimization Theory and Applications, 2025, vol. 205, issue 1, No 10, 37 pages

Abstract: Abstract We develop and analyze stochastic variants of ISTA and a full backtracking FISTA algorithms (Beck and Teboulle in SIAM J Imag Sci 2(1):183–202, 2009; Scheinberg et al. in Found Comput Math 14(3):389–417, 2014) for composite optimization without the assumption that stochastic gradient is an unbiased estimator. This work extends analysis of inexact fixed step ISTA/FISTA in Schmidt et al. (Convergence rates of inexact proximal-gradient methods for convex optimization, 2022. arXiv:1109.2415 ) to the case of stochastic gradient estimates and adaptive step-size parameter chosen by backtracking. It also extends the framework for analyzing stochastic line-search method in Cartis and Scheinberg (Math Program 169(2):337-375, 2018) to the proximal gradient framework as well as to the accelerated first order methods.

Keywords: Stochastic optimization algorithm; Convex optimization; 90C25; 90-08 (search for similar items in EconPapers)
Date: 2025
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s10957-025-02621-8 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:joptap:v:205:y:2025:i:1:d:10.1007_s10957-025-02621-8

Ordering information: This journal article can be ordered from
http://www.springer. ... cs/journal/10957/PS2

DOI: 10.1007/s10957-025-02621-8

Access Statistics for this article

Journal of Optimization Theory and Applications is currently edited by Franco Giannessi and David G. Hull

More articles in Journal of Optimization Theory and Applications from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().