A framework for machine learning of model error in dynamical systems
From MaRDI portal
Publication:6076655
dynamical systemsmodel errorrandom featuresrecurrent neural networksreservoir computingstatistical learning
Ergodic theorems, spectral theory, Markov operators (37A30) Time series analysis of dynamical systems (37M10) Computational methods for ergodic theory (approximation of invariant measures, computation of Lyapunov exponents, entropy, etc.) (37M25) Approximation by other special function classes (41A30) Numerical methods for initial value problems involving ordinary differential equations (65L05) Knowledge representation (68T30)
Abstract: The development of data-informed predictive models for dynamical systems is of widespread interest in many disciplines. We present a unifying framework for blending mechanistic and machine-learning approaches to identify dynamical systems from noisily and partially observed data. We compare pure data-driven learning with hybrid models which incorporate imperfect domain knowledge. Our formulation is agnostic to the chosen machine learning model, is presented in both continuous- and discrete-time settings, and is compatible both with model errors that exhibit substantial memory and errors that are memoryless. First, we study memoryless linear (w.r.t. parametric-dependence) model error from a learning theory perspective, defining excess risk and generalization error. For ergodic continuous-time systems, we prove that both excess risk and generalization error are bounded above by terms that diminish with the square-root of T, the time-interval over which training data is specified. Secondly, we study scenarios that benefit from modeling with memory, proving universal approximation theorems for two classes of continuous-time recurrent neural networks (RNNs): both can learn memory-dependent model error. In addition, we connect one class of RNNs to reservoir computing, thereby relating learning of memory-dependent error to recent work on supervised learning between Banach spaces using random features. Numerical results are presented (Lorenz '63, Lorenz '96 Multiscale systems) to compare purely data-driven and hybrid approaches, finding hybrid methods less data-hungry and more parametrically efficient. Finally, we demonstrate numerically how data assimilation can be leveraged to learn hidden dynamics from noisy, partially-observed data, and illustrate challenges in representing memory by this approach, and in the training of such models.
Recommendations
- Using data assimilation to train a hybrid forecast system that combines machine-learning and knowledge-based components
- Deep learning of dynamics and signal-noise decomposition with time-stepping constraints
- Time-series machine-learning error models for approximate solutions to parameterized dynamical systems
- A nonintrusive hybrid neural-physics modeling of incomplete dynamical systems: Lorenz equations
- Learning latent dynamics for partially observed chaotic systems
Cites work
- \textit{A priori} estimation of memory effects in reduced-order models of nonlinear systems using the Mori-Zwanzig formalism
- A computational strategy for multiscale systems with applications to Lorenz 96 model
- A dynamic subgrid scale model for large eddy simulations based on the Mori-Zwanzig formalism
- A family of embedded Runge-Kutta formulae
- A priori estimates of the population risk for two-layer neural networks
- Analysis of the 3DVAR filter for the partially observed Lorenz '63 model
- Approximation by superpositions of a sigmoidal function
- Asymptotic analysis for periodic structures
- Augmenting physical models with deep networks for complex dynamics forecasting*
- Autodifferentiable ensemble Kalman filters
- Bagging predictors
- Bayesian solution uncertainty quantification for differential equations
- Central limit theorems and invariance principles for Lorenz attractors
- Consistency of maximum likelihood estimation for some dynamical systems
- Data Assimilation
- Data assimilation: methods, algorithms, and applications
- Data-based stochastic model reduction for the Kuramoto-Sivashinsky equation
- Data-driven discovery of coordinates and governing equations
- Data-driven model reduction, Wiener projections, and the Koopman-Mori-Zwanzig formalism
- Data-driven spectral analysis of the Koopman operator
- Deep learning
- Deterministic Nonperiodic Flow
- Discovering governing equations from data by sparse identification of nonlinear dynamical systems
- Discovery of dynamics using linear multistep methods
- Echo state networks are universal
- Error bounds of the invariant statistics in machine learning of ergodic Itô diffusions
- Extracting Sparse High-Dimensional Dynamics from Limited Data
- Extracting Structured Dynamical Systems Using Sparse Optimization With Very Few Samples
- Fundamentals of recurrent neural network (RNN) and long short-term memory (LSTM) network
- Gaussian processes for machine learning.
- scientific article; zbMATH DE number 7225149 (Why is no real title available?)
- scientific article; zbMATH DE number 3810550 (Why is no real title available?)
- scientific article; zbMATH DE number 3752429 (Why is no real title available?)
- scientific article; zbMATH DE number 43985 (Why is no real title available?)
- scientific article; zbMATH DE number 67633 (Why is no real title available?)
- scientific article; zbMATH DE number 477682 (Why is no real title available?)
- scientific article; zbMATH DE number 1086828 (Why is no real title available?)
- scientific article; zbMATH DE number 1086057 (Why is no real title available?)
- scientific article; zbMATH DE number 823069 (Why is no real title available?)
- scientific article; zbMATH DE number 7625195 (Why is no real title available?)
- Inverse acoustic and electromagnetic scattering theory
- Kernel Analog Forecasting: Multiscale Test Problems
- Kernel-based prediction of non-Markovian time series
- Learning dynamical systems from data: a simple cross-validation perspective. I: Parametric kernel flows
- Learning latent dynamics for partially observed chaotic systems
- Learning representations by back-propagating errors
- Learning stochastic closures using ensemble Kalman inversion
- Machine learning for prediction with missing dynamics
- Machine-learning error models for approximate solutions to parameterized systems of nonlinear equations
- Model reduction and neural networks for parametric PDEs
- Model reduction with memory and the machine learning of dynamical systems
- Modeling of missing dynamical systems: deriving parametric models using a nonparametric framework
- Modelling and control of dynamic systems using Gaussian process models
- Modelling spatiotemporal dynamics from Earth observation data with neural differential equations
- Multiscale Methods
- Neural networks and dynamical systems
- New directions in statistical signal processing: from systems to brain
- Numerical techniques for multi-scale dynamical systems with stochastic effects
- On dynamic mode decomposition: theory and applications
- On Information and Sufficiency
- On the estimation of the Mori-Zwanzig memory integral
- Operator-theoretic framework for forecasting nonlinear time series with kernel analog techniques
- Optimal prediction and the Mori–Zwanzig representation of irreversible processes
- Orthogonal Gaussian process models
- Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations
- Probabilistic Forecasting and Bayesian Data Assimilation
- Recurrent neural network closure of parametric POD-Galerkin reduced-order models based on the Mori-Zwanzig formalism
- Reservoir computing approaches to recurrent neural network training
- SINDy-PI: a robust algorithm for parallel implicit sparse identification of nonlinear dynamics
- Smoothing noisy data with spline functions
- Solving and learning nonlinear PDEs with Gaussian processes
- Stable \textit{a posteriori} LES of 2D turbulence using convolutional neural networks: backscattering analysis and generalization to higher \(Re\) via transfer learning
- Stable architectures for deep neural networks
- Statistical and computational inverse problems.
- Statistical inference for ergodic diffusion processes.
- Supervised learning from noisy observations: combining machine-learning techniques with data assimilation
- The discovery of dynamics via linear multistep methods and deep learning: error estimation
- The Random Feature Model for Input-Output Maps between Banach Spaces
- Turbulence modeling in the age of data
- Using machine learning to replicate chaotic attractors and calculate Lyapunov exponents from data
- Variance continuity for Lorenz flows
Cited in
(24)- Machine-learning-based modeling of coarse-scale error, with application to uncertainty quantification
- Machine-learning error models for approximate solutions to parameterized systems of nonlinear equations
- Robust modeling of unknown dynamical systems via ensemble averaged learning
- Bridging the gap: machine learning to resolve improperly modeled dynamics
- Machine learning for prediction with missing dynamics
- A proposal on machine learning via dynamical systems
- Reduced-order autodifferentiable ensemble Kalman filters
- Mitigating Model Error via a Multimodel Method and Application to Tropical Intraseasonal Oscillations
- Reservoir computing with error correction: long-term behaviors of stochastic dynamical systems
- Quantum Mechanics for Closure of Dynamical Systems
- Fourier series-based approximation of time-varying parameters in ordinary differential equations
- Physics-constrained data-driven variational method for discrepancy modeling
- Discrepancy Modeling Framework: Learning Missing Physics, Modeling Systematic Residuals, and Disambiguating between Deterministic and Random Effects
- Data-informed reservoir computing for efficient time-series prediction
- Filtering dynamical systems using observations of statistics
- Learning about structural errors in models of complex dynamical systems
- Hybrid modeling design patterns
- Learning enhanced ensemble filters
- Bayesian experimental design for model discrepancy calibration: an auto-differentiable ensemble Kalman inversion approach
- On the choice of the non-trainable internal weights in random feature maps for forecasting chaotic dynamical systems
- How more data can hurt: instability and regularization in next-generation reservoir computing
- Error approximation and bias correction in dynamic problems using a recurrent neural network/finite element hybrid model
- Scientific machine learning for closure models in multiscale problems: a review
- Inference of Hamiltonian and Lie-Poisson structures from vector field samples
This page was built for publication: A framework for machine learning of model error in dynamical systems
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6076655)
