Economics at your fingertips  

Stochastic relaxed inertial forward-backward-forward splitting for monotone inclusions in Hilbert spaces

Shisheng Cui (), Uday Shanbhag (), Mathias Staudigl () and Phan Vuong ()
Additional contact information
Shisheng Cui: Pennsylvania State University
Uday Shanbhag: Pennsylvania State University
Mathias Staudigl: Maastricht University
Phan Vuong: University of Southampton

Computational Optimization and Applications, 2022, vol. 83, issue 2, No 4, 465-524

Abstract: Abstract We consider monotone inclusions defined on a Hilbert space where the operator is given by the sum of a maximal monotone operator T and a single-valued monotone, Lipschitz continuous, and expectation-valued operator V. We draw motivation from the seminal work by Attouch and Cabot (Attouch in AMO 80:547–598, 2019, Attouch in MP 184: 243–287) on relaxed inertial methods for monotone inclusions and present a stochastic extension of the relaxed inertial forward–backward-forward method. Facilitated by an online variance reduction strategy via a mini-batch approach, we show that our method produces a sequence that weakly converges to the solution set. Moreover, it is possible to estimate the rate at which the discrete velocity of the stochastic process vanishes. Under strong monotonicity, we demonstrate strong convergence, and give a detailed assessment of the iteration and oracle complexity of the scheme. When the mini-batch is raised at a geometric (polynomial) rate, the rate statement can be strengthened to a linear (suitable polynomial) rate while the oracle complexity of computing an $$\epsilon $$ ϵ -solution improves to $${\mathcal {O}}(1/\epsilon )$$ O ( 1 / ϵ ) . Importantly, the latter claim allows for possibly biased oracles, a key theoretical advancement allowing for far broader applicability. By defining a restricted gap function based on the Fitzpatrick function, we prove that the expected gap of an averaged sequence diminishes at a sublinear rate of $${\mathcal {O}}(1/k)$$ O ( 1 / k ) while the oracle complexity of computing a suitably defined $$\epsilon $$ ϵ -solution is $${\mathcal {O}}(1/\epsilon ^{1+a})$$ O ( 1 / ϵ 1 + a ) where $$a > 1$$ a > 1 . Numerical results on two-stage games and an overlapping group Lasso problem illustrate the advantages of our method compared to competitors.

Keywords: Monotone operator splitting; Stochastic approximation; Complexity; Variance reduction; Dynamic sampling (search for similar items in EconPapers)
Date: 2022
References: View references in EconPapers View complete reference list from CitEc
Citations: Track citations by RSS feed

Downloads: (external link) Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link:

Ordering information: This journal article can be ordered from

DOI: 10.1007/s10589-022-00399-3

Access Statistics for this article

Computational Optimization and Applications is currently edited by William W. Hager

More articles in Computational Optimization and Applications from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().

Page updated 2022-11-27
Handle: RePEc:spr:coopap:v:83:y:2022:i:2:d:10.1007_s10589-022-00399-3