Convex optimization via inertial algorithms with vanishing Tikhonov regularization: fast convergence to the minimum norm solution

Hedy Attouch () and Szilárd Csaba László ()
Additional contact information
Hedy Attouch: IMAG, Univ. Montpellier, CNRS
Szilárd Csaba László: Technical University of Cluj-Napoca

Mathematical Methods of Operations Research, 2024, vol. 99, issue 3, No 4, 307-347

Abstract: Abstract In a Hilbertian framework, for the minimization of a general convex differentiable function f, we introduce new inertial dynamics and algorithms that generate trajectories and iterates that converge fastly towards the minimizer of f with minimum norm. Our study is based on the non-autonomous version of the Polyak heavy ball method, which, at time t, is associated with the strongly convex function obtained by adding to f a Tikhonov regularization term with vanishing coefficient $$\varepsilon (t)$$ ε ( t ) . In this dynamic, the damping coefficient is proportional to the square root of the Tikhonov regularization parameter $$\varepsilon (t)$$ ε ( t ) . By adjusting the speed of convergence of $$\varepsilon (t)$$ ε ( t ) towards zero, we will obtain both rapid convergence towards the infimal value of f, and the strong convergence of the trajectories towards the element of minimum norm of the set of minimizers of f. In particular, we obtain an improved version of the dynamic of Su-Boyd-Candès for the accelerated gradient method of Nesterov. This study naturally leads to corresponding first-order algorithms obtained by temporal discretization. In the case of a proper lower semicontinuous and convex function f, we study the proximal algorithms in detail, and show that they benefit from similar properties.

Keywords: Accelerated gradient methods; Convex optimization; Damped inertial dynamics; Minimum norm solution; Nesterov accelerated gradient method; Tikhonov approximation; 37N40; 46N10; 49M30; 65B99; 65K05; 65K10; 90B50; 90C25 (search for similar items in EconPapers)
Date: 2024
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
http://link.springer.com/10.1007/s00186-024-00867-y Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:mathme:v:99:y:2024:i:3:d:10.1007_s00186-024-00867-y

Ordering information: This journal article can be ordered from
http://www.springer.com/economics/journal/00186

DOI: 10.1007/s00186-024-00867-y

Access Statistics for this article

Mathematical Methods of Operations Research is currently edited by Oliver Stein

More articles in Mathematical Methods of Operations Research from Springer, Gesellschaft für Operations Research (GOR), Nederlands Genootschap voor Besliskunde (NGB)
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().