New Penalized Stochastic Gradient Methods for Linearly Constrained Strongly Convex Optimization

Meng Li (), Paul Grigas () and Alper Atamtürk ()
Additional contact information
Meng Li: University of California, Berkeley
Paul Grigas: University of California, Berkeley
Alper Atamtürk: University of California, Berkeley

Journal of Optimization Theory and Applications, 2025, vol. 205, issue 2, No 10, 40 pages

Abstract: Abstract For minimizing a strongly convex objective function subject to linear inequality constraints, we consider a penalty approach that allows one to utilize stochastic methods for problems with a moderate to large number of constraints and/or objective function terms. We provide upper bounds on the distance between the solutions to the original constrained problem and the penalty reformulations, guaranteeing the convergence of the proposed approach. We consider a static method that uses a fixed smoothness parameter for the penalty function as well as a dynamic nested method with a novel way for updating the smoothness parameter of the penalty function and the step-size. In both cases, we apply accelerated stochastic gradient methods and study the expected incremental/stochastic gradient iteration complexity to produce a solution within an expected distance of $$\epsilon $$ ϵ to the optimal solution of the original problem. We show that this complexity is proportional to $$m\sqrt{\frac{m}{\mu \epsilon }}$$ m m μ ϵ , where m is the number of constraints and $$\mu $$ μ is the strong convexity parameter of the objective function, which improves upon existing results when m is not too large. We also show how to query an approximate dual solution after stochastically solving the penalty reformulations, leading to results on the convergence of the duality gap. Moreover, the nested structure of the algorithm and upper bounds on the distance to the optimal solutions allows one to safely eliminate constraints that are inactive at an optimal solution throughout the algorithm, which leads to improved complexity results. Finally, we present computational results that demonstrate the effectiveness and robustness of our algorithm.

Keywords: Convex optimization; Penalty method; Stochastic gradient; Duality gap; Screening procedure; 90C30; 90C25; 65K05 (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s10957-025-02646-z Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:joptap:v:205:y:2025:i:2:d:10.1007_s10957-025-02646-z

Ordering information: This journal article can be ordered from
http://www.springer. ... cs/journal/10957/PS2

DOI: 10.1007/s10957-025-02646-z

Access Statistics for this article

Journal of Optimization Theory and Applications is currently edited by Franco Giannessi and David G. Hull

More articles in Journal of Optimization Theory and Applications from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().