A theoretical analysis of deep neural networks and parametric PDEs

DOI10.1007/s00365-021-09551-4OpenAlexW3167031531WikidataQ114229767 ScholiaQ114229767MaRDI QIDQ2117329

Mones Raslan, Philipp Petersen, Gitta Kutyniok, Reinhold Schneider

Publication date: 21 March 2022

Published in: Constructive Approximation (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/1904.00377

zbMATH Keywords

reduced basis methodparametric PDEsapproximation ratesdeep neural networks

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05) Finite element, Rayleigh-Ritz and Galerkin methods for boundary value problems involving PDEs (65N30) Theoretical approximation in context of PDEs (35A35) Rate of convergence, degree of approximation (41A25) Elliptic equations and elliptic systems (35J99) Approximation by arbitrary nonlinear expressions; widths and entropy (41A46) Numerical analysis (65-XX)

Related Items (46)

Linear/ridge expansions: enhancing linear approximations by ridge functions ⋮ Reduced basis methods for time-dependent problems ⋮ A Proof that Artificial Neural Networks Overcome the Curse of Dimensionality in the Numerical Approximation of Black–Scholes Partial Differential Equations ⋮ A neural network multigrid solver for the Navier-Stokes equations ⋮ Two-Layer Neural Networks with Values in a Banach Space ⋮ Discretization of parameter identification in PDEs using neural networks ⋮ A deep learning approach to Reduced Order Modelling of parameter dependent partial differential equations ⋮ Deep neural network approximations for solutions of PDEs based on Monte Carlo algorithms ⋮ Overcoming the curse of dimensionality in the numerical approximation of parabolic partial differential equations with gradient-dependent nonlinearities ⋮ Variational physics informed neural networks: the role of quadratures and test functions ⋮ Approximations with deep neural networks in Sobolev time-space ⋮ Designing rotationally invariant neural networks from PDEs and variational methods ⋮ Full error analysis for the training of deep neural networks ⋮ Unbiased Deep Solvers for Linear Parametric PDEs ⋮ Deep Neural Network Surrogates for Nonsmooth Quantities of Interest in Shape Uncertainty Quantification ⋮ Deep ReLU neural network approximation in Bochner spaces and applications to parametric PDEs ⋮ Deep Surrogate Accelerated Delayed-Acceptance Hamiltonian Monte Carlo for Bayesian Inference of Spatio-Temporal Heat Fluxes in Rotating Disc Systems ⋮ An introduction to the mathematics of deep learning ⋮ Neural Network Approximation of Coarse-Scale Surrogates in Numerical Homogenization ⋮ Deep learning methods for partial differential equations and related parameter identification problems ⋮ Mesh-informed neural networks for operator learning in finite element spaces ⋮ Sparse polynomial approximations for affine parametric saddle point problems ⋮ A New Certified Hierarchical and Adaptive RB-ML-ROM Surrogate Model for Parametrized PDEs ⋮ Learning high frequency data via the coupled frequency predictor-corrector triangular DNN ⋮ Exponential Convergence of Deep Operator Networks for Elliptic Partial Differential Equations ⋮ Space-time error estimates for deep neural network approximations for differential equations ⋮ Limitations of neural network training due to numerical instability of backpropagation ⋮ Deep convolutional Ritz method: parametric PDE surrogates without labeled data ⋮ Collocation approximation by deep neural ReLU networks for parametric and stochastic PDEs with lognormal inputs ⋮ Connections between numerical algorithms for PDEs and neural networks ⋮ An artificial neural network approach to bifurcating phenomena in computational fluid dynamics ⋮ DeepBND: a machine learning approach to enhance multiscale solid mechanics ⋮ Learning the random variables in Monte Carlo simulations with stochastic gradient descent: Machine learning for parametric PDEs and financial derivative pricing ⋮ On the spectral bias of coupled frequency predictor-corrector triangular DNN: the convergence analysis ⋮ DNN-MG: a hybrid neural network/finite element method with applications to 3D simulations of the Navier-Stokes equations ⋮ A machine learning approach to enhance the SUPG stabilization method for advection-dominated differential problems ⋮ wPINNs: Weak Physics Informed Neural Networks for Approximating Entropy Solutions of Hyperbolic Conservation Laws ⋮ The mathematics of artificial intelligence ⋮ Error assessment of an adaptive finite elements -- neural networks method for an elliptic parametric PDE ⋮ Computation and learning in high dimensions. Abstracts from the workshop held August 1--7, 2021 (hybrid meeting) ⋮ Approximation properties of residual neural networks for Kolmogorov PDEs ⋮ An overview on deep learning-based approximation methods for partial differential equations ⋮ Solving traveltime tomography with deep learning ⋮ Unnamed Item ⋮ Solving parametric partial differential equations with deep rectified quadratic unit neural networks ⋮ PDE-Aware Deep Learning for Inverse Problems in Cardiac Electrophysiology

Uses Software

• redbKIT
• DGM

Cites Work

• Unnamed Item
• Breaking the curse of dimensionality in sparse polynomial approximation of parametric PDEs
• Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations
• Data driven approximation of parametrized PDEs by reduced basis and neural networks
• Reduced basis approximation and a posteriori error estimation for affinely parametrized elliptic coercive partial differential equations. Application to transport and continuum mechanics.
• Reduced basis method for quadratically nonlinear transport equations
• On making large nonlinear problems small
• Lower bounds for approximation by MLP neural networks
• Non-intrusive reduced order modeling of nonlinear problems using neural networks
• Multilayer feedforward networks are universal approximators
• Provable approximation properties for deep neural networks
• Approximation properties of a multilayered feedforward artificial neural network
• DGM: a deep learning algorithm for solving partial differential equations
• Numerical solution of the parametric diffusion equation by deep neural networks
• Proof that deep artificial neural networks overcome the curse of dimensionality in the numerical approximation of Kolmogorov partial differential equations with constant diffusion and nonlinear drift coefficients
• Optimal approximation of piecewise smooth functions using deep ReLU neural networks
• A proof that rectified deep neural networks overcome the curse of dimensionality in the numerical approximation of semilinear heat equations
• Model reduction of dynamical systems on nonlinear manifolds using deep convolutional autoencoders
• Error bounds for approximations with deep ReLU networks
• Variational Monte Carlo -- bridging concepts of machine learning and high-dimensional partial differential equations
• Greedy algorithms for reduced bases in Banach spaces
• Gaussian elimination is not optimal
• On the mathematical foundations of learning
• How To Best Sample a Solution Manifold?
• Certified Reduced Basis Methods for Parametrized Partial Differential Equations
• Introduction to Uncertainty Quantification
• Kolmogorov widths and low-rank approximations of parametric elliptic PDEs
• Sparse polynomial approximation of parametric elliptic PDEs. Part I: affine coefficients
• Convergence Rates for Greedy Algorithms in Reduced Basis Methods
• Relu Deep Neural Networks and Linear Finite Elements
• Efficient reduced-basis treatment of nonaffine and nonlinear partial differential equations
• A Posteriori Error Analysis of the Reduced Basis Method for Nonaffine Parametrized Nonlinear PDEs
• Recent advances in reduction methods for nonlinear problems
• Universal approximation bounds for superpositions of a sigmoidal function
• Parametric PDEs: sparse or low-rank approximations?
• Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations
• Deep learning in high dimension: Neural network expression rates for generalized polynomial chaos expansions in UQ
• Solving high-dimensional partial differential equations using deep learning
• Multilevel approximation of parametric and stochastic PDES
• Deep Neural Network Approximation Theory
• Solving parametric PDE problems with artificial neural networks
• Optimal Approximation with Sparsely Connected Deep Neural Networks
• Analysis of the Generalization Error: Empirical Risk Minimization over Deep Artificial Neural Networks Overcomes the Curse of Dimensionality in the Numerical Approximation of Black--Scholes Partial Differential Equations
• Deep ReLU networks and high-order finite element methods
• Error bounds for approximations with deep ReLU neural networks in Ws,p norms
• Rectified deep neural networks overcome the curse of dimensionality for nonsmooth value functions in zero-sum games of nonlinear stiff systems
• Equivalence of approximation by convolutional neural networks and fully-connected networks
• Approximation of high-dimensional parametric PDEs
• Fully Discrete Approximation of Parametric and Stochastic Elliptic PDEs
• Reduced Basis Methods for Partial Differential Equations
• Approximation by superpositions of a sigmoidal function

This page was built for publication: A theoretical analysis of deep neural networks and parametric PDEs