Distributed stochastic compositional optimization problems over directed networks

Shengchao Zhao () and Yongchao Liu ()
Additional contact information
Shengchao Zhao: Dalian University of Technology
Yongchao Liu: Dalian University of Technology

Computational Optimization and Applications, 2024, vol. 87, issue 1, No 8, 249-288

Abstract: Abstract We study the distributed stochastic compositional optimization problems over directed communication networks in which agents privately own a stochastic compositional objective function and collaborate to minimize the sum of all objective functions. We propose a distributed stochastic compositional gradient descent method, where the gradient tracking and the stochastic correction techniques are employed to adapt to the networks’ directed structure and increase the accuracy of inner function estimation. When the objective function is smooth, the proposed method achieves the convergence rate $${\mathcal {O}}\left( k^{-1/2}\right) $$ O k - 1 / 2 and sample complexity $${\mathcal {O}}\left( \frac{1}{\epsilon ^2}\right) $$ O 1 ϵ 2 for finding the ( $$\epsilon $$ ϵ )-stationary point. When the objective function is strongly convex, the convergence rate is improved to $${\mathcal {O}}\left( k^{-1}\right) $$ O k - 1 . Moreover, the asymptotic normality of Polyak-Ruppert averaged iterates of the proposed method is also presented. We demonstrate the empirical performance of the proposed method on model-agnostic meta-learning problem and logistic regression problem.

Keywords: Distributed stochastic compositional optimization; Directed communication networks; Gradient tracking; Asymptotic normality (search for similar items in EconPapers)
Date: 2024
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s10589-023-00512-0 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:coopap:v:87:y:2024:i:1:d:10.1007_s10589-023-00512-0

Ordering information: This journal article can be ordered from
http://www.springer.com/math/journal/10589

DOI: 10.1007/s10589-023-00512-0

Access Statistics for this article

Computational Optimization and Applications is currently edited by William W. Hager

More articles in Computational Optimization and Applications from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().