Transformer operator network with fast difference gradient positive–negative momentum for solving Navier–Stokes equations

Ruslan Abdulkadirov, Pavel Lyakhov, Maxim Bergerman and Nikolay Nagornov

Chaos, Solitons & Fractals, 2025, vol. 200, issue P1

Abstract: Modern machine learning approaches solve many problems in human activity. Recent models of neural networks find applications in solving equations of mathematical physics. Along with traditional numerical methods, physics-informed learning builds the approximate solution with a relatively small L2 and mean squared errors. In this paper, we propose a fast difference-gradient positive–negative momentum optimizer that achieves a global minimum of the loss function with a convergence rate O(logT) and stability O(T). This optimization algorithm solves loss function minimization problems such as gradient discontinuity, vanishing gradient and local minimum traversal. Analysis on the set of test functions confirms the superiority of the proposed optimizer over state-of-the-art methods. Through the modified positive–negative moment estimation, the proposed optimizer gives more appropriate weight updates in physics-informed, deep operator, and transformer operator neural networks in Navier–Stokes equation solving. In particular, the proposed optimizer minimizes the loss function better than known analogs in solving 2d-Kovasznay, (2d+t)-Taylor–Green, (3d+t)-Beltrami, and (2d+t) circular cylindrical flows. Fast difference gradient positive–negative moment estimation allows the physics-informed model to reduce the L2 and mean squared errors solution by 14.6−53.9 percentage points. The proposed fast difference gradient positive–negative momentum can increase the approximate solutions of partial differential equations in physics-informed neural, deep operator, and transformer operator networks.

Keywords: Optimization; Transformers; Deep neural operator; Physics-informed neural networks; Navier–Stokes equations (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0960077925009774
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:chsofr:v:200:y:2025:i:p1:s0960077925009774

DOI: 10.1016/j.chaos.2025.116964

Access Statistics for this article

Chaos, Solitons & Fractals is currently edited by Stefano Boccaletti and Stelios Bekiros

More articles in Chaos, Solitons & Fractals from Elsevier
Bibliographic data for series maintained by Thayer, Thomas R. ().