Accelerated Stochastic Conjugate Gradient for a class of convex optimization

Lulu He and Yanan Du

PLOS ONE, 2025, vol. 20, issue 12, 1-21

Abstract: The conjugate gradient method is widely recognized as a foundational technique for large-scale unconstrained optimization. In this work, we introduce an Accelerated Stochastic Conjugate Gradient (ASCG) algorithm, specifically designed for a class of convex empirical risk minimization problems. The proposed ASCG method integrates a variance-reduced gradient estimator-inspired by modern stochastic variance reduction techniques-to control noise and improve stability in the optimization process. Moreover, the ASCG algorithm incorporates a novel acceleration mechanism via a deflation factor on the step size, which is shown to achieve faster practical convergence compared to the baseline stochastic FR method. We provide a rigorous theoretical analysis demonstrating that ASCG achieves an expected linear convergence rate under strong convexity assumptions and attains a superior reduction in function values compared to non-accelerated stochastic counterparts. Extensive numerical experiments on four widely-used benchmark datasets confirm that ASCG consistently outperforms state-of-the-art stochastic optimization methods.

Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0338720 (text/html)
https://journals.plos.org/plosone/article/file?id= ... 38720&type=printable (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:plo:pone00:0338720

DOI: 10.1371/journal.pone.0338720

Access Statistics for this article

More articles in PLOS ONE from Public Library of Science
Bibliographic data for series maintained by plosone ().