Improving the stochastically controlled stochastic gradient method by the bandwidth-based stepsize

Chenchen Liu, Yakui Huang () and Dan Wang
Additional contact information
Chenchen Liu: Hebei University of Technology
Yakui Huang: Hebei University of Technology
Dan Wang: Hebei University of Technology

Computational Optimization and Applications, 2025, vol. 90, issue 3, No 11, 968 pages

Abstract: Abstract Stepsize plays an important role in the stochastic gradient method. The bandwidth-based stepsize allows us to adjust the stepsize within a banded region determined by some boundary functions. Based on the bandwidth-based stepsize, we propose a new method, namely SCSG-BD, for smooth non-convex finite-sum optimization problems. For the boundary functions 1/t, $$1/(t\log (t + 1))$$ 1 / ( t log ( t + 1 ) ) and $$1/t^p$$ 1 / t p ( $$p\in (0,1)$$ p ∈ ( 0 , 1 ) ), SCSG-BD converges sublinearly to a stationary point at a faster rate than the stochastically controlled stochastic gradient (SCSG) method under certain conditions. Moreover, SCSG-BD is able to converge linearly to the solution if the objective function satisfies the Polyak–Łojasiewicz condition. We also introduce the 1/t-Barzilai–Borwein stepsize for practical computation. Numerical experiments demonstrate that SCSG-BD performs better than SCSG and its variants.

Keywords: Stochastic gradient method; Bandwidth-based stepsize; Barzilai–Borwein stepsize; Non-convex finite-sum optimization; 90C20; 90C25; 90C30 (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s10589-025-00651-6 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:coopap:v:90:y:2025:i:3:d:10.1007_s10589-025-00651-6

Ordering information: This journal article can be ordered from
http://www.springer.com/math/journal/10589

DOI: 10.1007/s10589-025-00651-6

Access Statistics for this article

Computational Optimization and Applications is currently edited by William W. Hager

More articles in Computational Optimization and Applications from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().