Stability of Gated Recurrent Unit Neural Networks: Convex Combination Formulation Approach

Dušan M. Stipanović (), Mirna N. Kapetina (), Milan R. Rapaić () and Boris Murmann ()
Additional contact information
Dušan M. Stipanović: University of Illinois at Urbana-Champaign
Mirna N. Kapetina: University of Novi Sad
Milan R. Rapaić: University of Novi Sad
Boris Murmann: Stanford University

Journal of Optimization Theory and Applications, 2021, vol. 188, issue 1, No 14, 306 pages

Abstract: Abstract In this paper, a particular discrete-time nonlinear and time-invariant system represented as a vector difference equation is analyzed for its stability properties. The motivation for analyzing this particular system is that it models gated recurrent unit neural networks commonly used and well known in machine learning applications. From the technical perspective, the analyses exploit the systems similarities to a convex combination of discrete-time systems, where one of the systems is trivial, and thus, the overall properties are mostly dependent on the other one. Stability results are formulated for the nonlinear system and its linearization with respect to the systems, in general, multiple equilibria. To motivate and illustrate the potential of these results in applications, some particular results are derived for the gated recurrent unit neural network models and a connection between local stability analysis and learning is provided.

Keywords: Stability; Difference equations; Neural networks; 39A11; 37F99; 37N99 (search for similar items in EconPapers)
Date: 2021
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s10957-020-01776-w Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:joptap:v:188:y:2021:i:1:d:10.1007_s10957-020-01776-w

Ordering information: This journal article can be ordered from
http://www.springer. ... cs/journal/10957/PS2

DOI: 10.1007/s10957-020-01776-w

Access Statistics for this article

Journal of Optimization Theory and Applications is currently edited by Franco Giannessi and David G. Hull

More articles in Journal of Optimization Theory and Applications from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().