Realization theory of recurrent neural ODEs using polynomial system embeddings

DOI10.1016/J.SYSCONLE.2023.105468MaRDI QIDQ6160839zbMATH OpenOpenAlexFDO

Authors Martin Gonzalez, Thibault Defourneau, Hatem Hajri, Mihály Petreczky

Publication date 26 June 2023

Published in Systems \& Control Letters (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2205.11989

zbMATH Keywords

long short-term memoryneural ODEsrealization theoryrecurrent neural networkssystem identification

Mathematics Subject Classification ID

Attainable sets, reachability (93B03) Observability (93B07) Realizations from input-output data (93B15) System identification (93B30) Control/observation systems governed by ordinary differential equations (93C15)

Abstract: In this paper we show that neural ODE analogs of recurrent (ODE-RNN) and Long Short-Term Memory (ODE-LSTM) networks can be algorithmically embeddeded into the class of polynomial systems. This embedding preserves input-output behavior and can suitably be extended to other neural DE architectures. We then use realization theory of polynomial systems to provide necessary conditions for an input-output map to be realizable by an ODE-LSTM and sufficient conditions for minimality of such systems. These results represent the first steps towards realization theory of recurrent neural ODE architectures, which is is expected be useful for model reduction and learning algorithm analysis of recurrent neural ODEs.

Recommendations

• ODE-RU: a dynamical system view on recurrent neural networks
• Approximation of dynamical systems by recurrent neural networks
• On the differential equations of recurrent neural networks
• scientific article; zbMATH DE number 1190938
• Constructing general partial differential equations using polynomial and neural networks
• Modeling of continuous time dynamical systems with input by recurrent neural networks
• scientific article; zbMATH DE number 720643
• Synthesis of recurrent neural networks for dynamical system simulation

Cites work

• Algebraic Differential Equations and Rational Control Systems
• Further results on controllability of recurrent neural networks
• scientific article; zbMATH DE number 3985055 (Why is no real title available?)
• scientific article; zbMATH DE number 3668435 (Why is no real title available?)
• scientific article; zbMATH DE number 1877272 (Why is no real title available?)
• Linear Stochastic Systems
• Mathematical Description of Linear Dynamical Systems
• Minimal polynomial realizations
• Nonlinear control systems: An introduction
• Nonlinear controllability and observability
• Rational systems and observation fields
• Realization theory for rational systems: minimal rational realizations
• Realization Theory for Rational Systems: The Existence of Rational Realizations
• Realization theory of Nash systems
• State observability in recurrent neural networks

Cited in

(3)

• Dimension reduction in recurrent networks by canonicalization
• ODE-RU: a dynamical system view on recurrent neural networks
• Learning proper orthogonal decomposition of complex dynamics using heavy-ball neural ODEs

This page was built for publication: Realization theory of recurrent neural ODEs using polynomial system embeddings

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6160839)