Applications of fractional gradient descent method with adaptive momentum in BP neural networks

Xiaohui Han and Jianping Dong

Applied Mathematics and Computation, 2023, vol. 448, issue C

Abstract: A novel fractional gradient descent method with adaptive momentum is presented in this paper to improve the convergence speed and stability for BP neural network training. The fractional Grünwald-Letnikov derivative is used for the fractional gradient. The coefficient of the momentum term is set as an adaptive variable, depending on the fractional gradient of the current step and the weight change of the previous step. We give a detailed convergence proof of the proposed method. Experiments on MNIST data sets and XOR problem demonstrate that the fractional gradient descent method with adaptive momentum term can effectively improve convergence speed, maintain stability of BP neural network training, help escape from local minimum points, and enlarge the selection range of the learning rate.

Keywords: Fractional calculus; Gradient methods; Adaptive momentum; Convergence analysis (search for similar items in EconPapers)
Date: 2023
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0096300323001133
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:apmaco:v:448:y:2023:i:c:s0096300323001133

DOI: 10.1016/j.amc.2023.127944

Access Statistics for this article

Applied Mathematics and Computation is currently edited by Theodore Simos

More articles in Applied Mathematics and Computation from Elsevier
Bibliographic data for series maintained by Catherine Liu ().