Deep ReLU networks and high-order finite element methods

DOI10.1142/S0219530519410136zbMath1452.65354OpenAlexW2998107928MaRDI QIDQ5132226

Philipp Petersen, Joost A. A. Opschoor, Christoph Schwab

Publication date: 10 November 2020

Published in: Analysis and Applications (Search for Journal in Brave)

Full work available at URL: https://doi.org/10.1142/s0219530519410136

zbMATH Keywords

singularitiesfinite element methodsGevrey regularitydeep neural networksexponential covergence

Mathematics Subject Classification ID

Numerical computation using splines (65D07) Artificial neural networks and deep learning (68T07) Smoothness and regularity of solutions to PDEs (35B65) Neural networks for/in biological studies, artificial life and related topics (92B20) Stability and convergence of numerical methods for boundary value problems involving PDEs (65N12) Finite element, Rayleigh-Ritz and Galerkin methods for boundary value problems involving PDEs (65N30) Rate of convergence, degree of approximation (41A25) Approximation by arbitrary nonlinear expressions; widths and entropy (41A46) PDEs on graphs and networks (ramified or polygonal spaces) (35R02)

Related Items (45)

Neural network approximation ⋮ PFNN-2: A Domain Decomposed Penalty-Free Neural Network Method for Solving Partial Differential Equations ⋮ Thermodynamically consistent physics-informed neural networks for hyperbolic systems ⋮ Physically motivated structuring and optimization of neural networks for multi-physics modelling of solid oxide fuel cells ⋮ Approximation properties of deep ReLU CNNs ⋮ ReLU deep neural networks from the hierarchical basis perspective ⋮ Variational physics informed neural networks: the role of quadratures and test functions ⋮ Approximations with deep neural networks in Sobolev time-space ⋮ Simultaneous neural network approximation for smooth functions ⋮ On the approximation of functions by tanh neural networks ⋮ Probabilistic partition of unity networks for high‐dimensional regression problems ⋮ Neural Network Approximation of Coarse-Scale Surrogates in Numerical Homogenization ⋮ Three ways to solve partial differential equations with neural networks — A review ⋮ A convergent deep learning algorithm for approximation of polynomials ⋮ Galerkin neural network approximation of singularly-perturbed elliptic systems ⋮ Approximation theory of tree tensor networks: tensorized univariate functions ⋮ Deep ReLU neural networks in high-dimensional approximation ⋮ Exponential ReLU neural network approximation rates for point and edge singularities ⋮ Exponential Convergence of Deep Operator Networks for Elliptic Partial Differential Equations ⋮ Deep Learning in High Dimension: Neural Network Expression Rates for Analytic Functions in \(\pmb{L^2(\mathbb{R}^d,\gamma_d)}\) ⋮ ReLU neural network Galerkin BEM ⋮ Growing axons: greedy learning of neural networks with application to function approximation ⋮ Physics-constrained data-driven variational method for discrepancy modeling ⋮ Computation and learning in high dimensions. Abstracts from the workshop held August 1--7, 2021 (hybrid meeting) ⋮ A functional equation with polynomial solutions and application to neural networks ⋮ Deep neural network expression of posterior expectations in Bayesian PDE inversion ⋮ Deep ReLU network expression rates for option prices in high-dimensional, exponential Lévy models ⋮ Better Approximations of High Dimensional Smooth Functions by Deep Neural Networks with Rectified Power Units ⋮ Adaptive quadratures for nonlinear approximation of low-dimensional PDEs using smooth neural networks ⋮ Efficient approximation of solutions of parametric linear transport equations by ReLU DNNs ⋮ Unnamed Item ⋮ Numerical solution of the parametric diffusion equation by deep neural networks ⋮ Approximation rates for neural networks with encodable weights in smoothness spaces ⋮ POD-DL-ROM: enhancing deep learning-based reduced order models for nonlinear parametrized PDEs by proper orthogonal decomposition ⋮ Constructive deep ReLU neural network approximation ⋮ Exact imposition of boundary conditions with distance functions in physics-informed deep neural networks ⋮ The Gap between Theory and Practice in Function Approximation with Deep Neural Networks ⋮ Solving PDEs by variational physics-informed neural networks: an a posteriori error analysis ⋮ CAS4DL: Christoffel adaptive sampling for function approximation via deep learning ⋮ Higher-Order Quasi-Monte Carlo Training of Deep Neural Networks ⋮ Approximation of Smoothness Classes by Deep Rectifier Networks ⋮ Deep ReLU neural networks overcome the curse of dimensionality for partial integrodifferential equations ⋮ Deep neural networks can stably solve high-dimensional, noisy, non-linear inverse problems ⋮ A theoretical analysis of deep neural networks and parametric PDEs ⋮ Exponential ReLU DNN expression of holomorphic maps in high dimension

Uses Software

• DGM
• FPINNs

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Theory of function spaces
• On optimal global error bounds obtained by scaled local error estimates
• Best approximation by piecewise polynomials with variable knots and degrees
• Exponential convergence of \(hp\)-FEM for elliptic problems in polyhedra: mixed boundary conditions and anisotropic polynomial degrees
• Multilayer feedforward networks are universal approximators
• Provable approximation properties for deep neural networks
• The Deep Ritz Method: a deep learning-based numerical algorithm for solving variational problems
• On the degree of nonlinear spline approximation in Besov-Sobolev spaces
• The h, p and h-p versions of the finite element method in 1 dimension. II. The error analysis of the h- and h-p versions
• Approximation properties of a multilayered feedforward artificial neural network
• Exponential convergence of the deep neural network approximation for analytic functions
• Anisotropic radial basis function methods for continental size ice sheet simulations
• DGM: a deep learning algorithm for solving partial differential equations
• Optimal approximation of piecewise smooth functions using deep ReLU neural networks
• Error bounds for approximations with deep ReLU networks
• Machine learning approximation algorithms for high-dimensional fully nonlinear partial differential equations and second-order backward stochastic differential equations
• Exponential convergence ofhpquadrature for integral operators with Gevrey kernels
• A Remark on Stirling's Formula
• hp FEM for Reaction-Diffusion Equations I: Robust Exponential Convergence
• Analytic Regularity for a Singularly Perturbed Problem
• Universal approximation bounds for superpositions of a sigmoidal function
• Neural Networks for Localized Approximation
• Deep learning in high dimension: Neural network expression rates for generalized polynomial chaos expansions in UQ
• The $p$ and $hp$ versions of the finite element method for problems with boundary layers
• Solving high-dimensional partial differential equations using deep learning
• Optimal Approximation with Sparsely Connected Deep Neural Networks
• New Error Bounds for Deep ReLU Networks Using Sparse Grids
• fPINNs: Fractional Physics-Informed Neural Networks
• Deep neural networks for rotation-invariance approximation and learning
• Scattered Data Approximation

This page was built for publication: Deep ReLU networks and high-order finite element methods