Adaptive Low-Rank Hessian Approximation for Large-Scale Optimization: Theory, Algorithms, and Applications

Arush Rao Lagudu ()

International Journal of Innovative Science and Research Technology (IJISRT), 2026, vol. 11, issue 03, 2226-2238

Abstract: We introduce the Adaptive Low-rank Hessian Approximation (ALHA) algorithm, a novel quasi-Newton method that dynamically adjusts the rank of the Hessian approximation based on local curvature information and approximation quality. Unlike classical limited-memory BFGS (L-BFGS) which maintains a fixed memory parameter m, ALHA adaptively increases or decreases the effective rank to balance computational efficiency with approximation accuracy. We establish rigorous convergence guarantees under standard assumptions: for L-smooth and µ-strongly convex functions, ALHA achieves linear convergence at rate O((1 − µ/L)^k), matching the optimal rate for first-order methods with curvature information. Our analysis introduces a novel spectral approximation quality metric that governs rank adaptation and provides explicit bounds on the approximation error. We prove that the adaptive mechanism maintains bounded rank while ensuring sufficient descent at each iteration. Comprehensive numerical experiments on quadratic optimization, logistic regression on MNIST, Rosenbrock function minimization, and neural network training demonstrate that ALHA consistently outperforms fixed-rank methods, achieving up to 40% reduction in iterations while maintaining comparable per-iteration cost. The algorithm is particularly effective for ill-conditioned problems where adaptive rank selection captures essential curvature directions.

Keywords: Quasi-Newton Methods; L-BFGS; Adaptive Algorithms; Low-Rank Approximation; Large-Scale Optimization; Convergence Analysis; Hessian Approximation. (search for similar items in EconPapers)
Date: 2026
References: Add references at CitEc
Citations:

Downloads: (external link)
https://www.ijisrt.com/adaptive-lowrank-hessian-ap ... hms-and-applications (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:cvr:ijisrt:2026:03:ijisrt26mar1179

DOI: 10.38124/ijisrt/26mar1179

Access Statistics for this article

More articles in International Journal of Innovative Science and Research Technology (IJISRT) from IJISRT Publication
Bibliographic data for series maintained by Rahul Goyel ().