Proximal Gradient Methods with Adaptive Subspace Sampling

Dmitry Grishchenko (), Franck Iutzeler () and Jérôme Malick ()
Additional contact information
Dmitry Grishchenko: Laboratoire Jean Kuntzmann, Université Grenoble Alpes, 38402 Saint-Martin-d’Heres, France; Laboratoire d’Informatique de Grenoble, Université Grenoble Alpes, 38401 Saint-Martin-d’Heres, France
Franck Iutzeler: Laboratoire Jean Kuntzmann, Université Grenoble Alpes, 38402 Saint-Martin-d’Heres, France
Jérôme Malick: Laboratoire Jean Kuntzmann, Université Grenoble Alpes, 38402 Saint-Martin-d’Heres, France; Centre National de la Recherche Scientifique, 75016 Paris, France

Mathematics of Operations Research, 2021, vol. 46, issue 4, 1303-1323

Abstract: Many applications in machine learning or signal processing involve nonsmooth optimization problems. This nonsmoothness brings a low-dimensional structure to the optimal solutions. In this paper, we propose a randomized proximal gradient method harnessing this underlying structure. We introduce two key components: (i) a random subspace proximal gradient algorithm; and (ii) an identification-based sampling of the subspaces. Their interplay brings a significant performance improvement on typical learning problems in terms of dimensions explored.

Keywords: Primary: 65K10; secondary: 90C30; programming: non-linear; nonsmooth optimization; identification; proximal gradient algorithm; randomized methods (search for similar items in EconPapers)
Date: 2021
References: Add references at CitEc
Citations:

Downloads: (external link)
http://dx.doi.org/10.1287/moor.2020.1092 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:ormoor:v:46:y:2021:i:4:p:1303-1323

Access Statistics for this article

More articles in Mathematics of Operations Research from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().