Fractional-order gradient approach for optimizing neural networks: A theoretical and empirical analysis

Priyanka Harjule, Rinki Sharma and Rajesh Kumar

Chaos, Solitons & Fractals, 2025, vol. 192, issue C

Abstract: This article proposes a modified fractional gradient descent algorithm to enhance the learning capabilities of neural networks, comprising the benefits of a metaheuristic optimizer. The use of fractional derivatives, which possess memory properties, offers an additional degree of adaptability to the network. The convergence of the fractional gradient descent algorithm, incorporating the Caputo derivative in the neural network’s backpropagation process, is thoroughly examined, and a detailed convergence analysis is provided which indicates that it enables a more gradual and controlled adaptation of the network to the data. Additionally, the optimal fractional order has been found for each dataset, a contribution that has not been previously explored in the literature, which has a significant impact on the training of neural networks with fractional gradient backpropagation. In the experiments, four classification datasets and one regression dataset were used, and the results consistently show that the proposed hybrid algorithm achieves faster convergence across all cases. The empirical results with the proposed algorithm are supported by theoretical convergence analysis. Empirical results demonstrate that the proposed optimizer with optimal order yields more accurate results compared to existing optimizers.

Keywords: Fractional derivative; Gradient descent; Backpropagation; Metaheuristic optimization; Neural networks (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0960077925000220
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:chsofr:v:192:y:2025:i:c:s0960077925000220

DOI: 10.1016/j.chaos.2025.116009

Access Statistics for this article

Chaos, Solitons & Fractals is currently edited by Stefano Boccaletti and Stelios Bekiros

More articles in Chaos, Solitons & Fractals from Elsevier
Bibliographic data for series maintained by Thayer, Thomas R. ().