The continuous stochastic gradient method: part II–application and numerics

Max Grieshammer (), Lukas Pflug (), Michael Stingl () and Andrian Uihlein ()
Additional contact information
Max Grieshammer: Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU)
Lukas Pflug: Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU)
Michael Stingl: Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU)
Andrian Uihlein: Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU)

Computational Optimization and Applications, 2024, vol. 87, issue 3, No 10, 977-1008

Abstract: Abstract In this contribution, we present a numerical analysis of the continuous stochastic gradient (CSG) method, including applications from topology optimization and convergence rates. In contrast to standard stochastic gradient optimization schemes, CSG does not discard old gradient samples from previous iterations. Instead, design dependent integration weights are calculated to form a convex combination as an approximation to the true gradient at the current design. As the approximation error vanishes in the course of the iterations, CSG represents a hybrid approach, starting off like a purely stochastic method and behaving like a full gradient scheme in the limit. In this work, the efficiency of CSG is demonstrated for practically relevant applications from topology optimization. These settings are characterized by both, a large number of optimization variables and an objective function, whose evaluation requires the numerical computation of multiple integrals concatenated in a nonlinear fashion. Such problems could not be solved by any existing optimization method before. Lastly, with regards to convergence rates, first estimates are provided and confirmed with the help of numerical experiments.

Keywords: Stochastic gradient scheme; Convergence analysis; Step size rule; Backtracking line search; Constant step size; 65K05; 90C06; 90C15; 90C30 (search for similar items in EconPapers)
Date: 2024
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s10589-023-00540-w Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:coopap:v:87:y:2024:i:3:d:10.1007_s10589-023-00540-w

Ordering information: This journal article can be ordered from
http://www.springer.com/math/journal/10589

DOI: 10.1007/s10589-023-00540-w

Access Statistics for this article

Computational Optimization and Applications is currently edited by William W. Hager

More articles in Computational Optimization and Applications from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().