Rates of superlinear convergence for classical quasi-Newton methods

Anton, Rodomanov () and Yurii, Nesterov ()
Additional contact information
Anton, Rodomanov: Université catholique de Louvain, CORE, Belgium
Yurii, Nesterov: Université catholique de Louvain, CORE, Belgium

No 2020013, LIDAM Discussion Papers CORE from Université catholique de Louvain, Center for Operations Research and Econometrics (CORE)

Abstract: We present a new theoretical analysis of local superlinear convergence of the classical quasi-Newton methods from the convex Broyden class. Our analysis is based on the potential function involving the logarithm of determinant of Hessian approximation and the trace of inverse Hessian approximation. For the well-known DFP and BFGS methods, we obtain the rates of the form $\left[\frac L \mu \left(exp \left\{\frac n k \ln \frac L \mu\right\} - 1\right)\right]^{k/2}$ and $\left[exp\left\{\frac n k \ln \frac L \mu\right\} - 1\right]^{k/2}$ respectively, where k is the iteration counter, n is the dimension of the problem, μ is the strong convexity parameter, and L is the Lipschitz constant of the gradient. Currently, these are the best known superlinear convergence rates for these methods. In particular, our results show that the starting moment of superlinear convergence of BFGS method depends on the logarithm of the condition number $\frac L μ$ in the worst case.

Keywords: quasi-Newton methods; convex Broyden class; DFP; BFGS; superlinear convergence; local convergence; rate of convergence (search for similar items in EconPapers)
Date: 2020-02-11
References: Add references at CitEc
Citations:

Downloads: (external link)
https://sites.uclouvain.be/core/publications/coredp/coredp2020.html (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:cor:louvco:2020013

Access Statistics for this paper

More papers in LIDAM Discussion Papers CORE from Université catholique de Louvain, Center for Operations Research and Econometrics (CORE) Voie du Roman Pays 34, 1348 Louvain-la-Neuve (Belgium). Contact information at EDIRC.
Bibliographic data for series maintained by Alain GILLIS ().