A Momentum-Based Adaptive Primal–Dual Stochastic Gradient Method for Non-Convex Programs with Expectation Constraints

Rulei Qi, Dan Xue () and Yujia Zhai
Additional contact information
Rulei Qi: School of Mathematics and Statistics, Qingdao University, Qingdao 266071, China
Dan Xue: School of Mathematics and Statistics, Qingdao University, Qingdao 266071, China
Yujia Zhai: School of Mathematics and Statistics, Qingdao University, Qingdao 266071, China

Mathematics, 2024, vol. 12, issue 15, 1-26

Abstract: In this paper, we propose a stochastic primal-dual adaptive method based on an inexact augmented Lagrangian function to solve non-convex programs, referred to as the SPDAM. Different from existing methods, SPDAM incorporates adaptive step size and momentum-based search directions, which improve the convergence rate. At each iteration, an inexact augmented Lagrangian subproblem is solved to update the primal variables. A post-processing step is designed to adjust the primal variables to meet the accuracy requirement, and the adjusted primal variable is used to compute the dual variable. Under appropriate assumptions, we prove that the method converges to the ε -KKT point of the primal problem, and a complexity result of SPDAM less than O ( ε − 11 2 ) is established. This is better than the most famous O ( ε − 6 ) result. The numerical experimental results validate that this method outperforms several existing methods with fewer iterations and a lower running time.

Keywords: non-convex stochastic optimization; expectation-constrained; stochastic gradient method; adaptive method; momentum-based search direction (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2024
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2227-7390/12/15/2393/pdf (application/pdf)
https://www.mdpi.com/2227-7390/12/15/2393/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:12:y:2024:i:15:p:2393-:d:1447312

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().