Gauss–Southwell Type Descent Methods for Low-Rank Matrix Optimization

Guillaume Olikier (), André Uschmajew () and Bart Vandereycken ()
Additional contact information
Guillaume Olikier: Université Côte d’Azur and Inria
André Uschmajew: University of Augsburg
Bart Vandereycken: University of Geneva

Journal of Optimization Theory and Applications, 2025, vol. 206, issue 1, No 6, 32 pages

Abstract: Abstract We consider gradient-related methods for low-rank matrix optimization with a smooth cost function. The methods operate on single factors of the low-rank factorization and share aspects of both alternating and Riemannian optimization. Two possible choices for the search directions based on Gauss–Southwell type selection rules are compared: one using the gradient of a factorized non-convex formulation, the other using the Riemannian gradient. While both methods provide gradient convergence guarantees that are similar to the unconstrained case, numerical experiments on a quadratic cost function indicate that the version based on the Riemannian gradient is significantly more robust with respect to small singular values and the condition number of the cost function. As a side result of our approach, we also obtain new convergence results for the alternating least squares method.

Keywords: Low-rank matrix optimization; First-order method; Riemannian optimization; Convergence analysis; Critical point; 65K05; 65K10; 65F55; 90C26; 90C30 (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s10957-025-02682-9 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:joptap:v:206:y:2025:i:1:d:10.1007_s10957-025-02682-9

Ordering information: This journal article can be ordered from
http://www.springer. ... cs/journal/10957/PS2

DOI: 10.1007/s10957-025-02682-9

Access Statistics for this article

Journal of Optimization Theory and Applications is currently edited by Franco Giannessi and David G. Hull

More articles in Journal of Optimization Theory and Applications from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().