Convergence rate analysis for deep Ritz method

DOI10.4208/CICP.OA-2021-0195MaRDI QIDQ5077692zbMATH OpenOpenAlexFDO

Authors Chenguang Duan, Y. M. Lai, Dingwei Li, Xiliang Lu, Yu Ling Jiao, Jerry Zhijian Yang

Publication date 19 May 2022

Published in Communications in Computational Physics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2103.13330

zbMATH Keywords

convergence rate analysisdeep neural networksdeep Ritz method

Mathematics Subject Classification ID

Variational methods applied to PDEs (35A15) Second-order elliptic equations (35J15) Boundary value problems for second-order elliptic equations (35J25) PDEs on graphs and networks (ramified or polygonal spaces) (35R02) Nonparametric estimation (62G05) Stability and convergence of numerical methods for boundary value problems involving PDEs (65N12) Error bounds for boundary value problems involving PDEs (65N15) Artificial neural networks and deep learning (68T07) Neural networks for/in biological studies, artificial life and related topics (92B20)

Abstract: Using deep neural networks to solve PDEs has attracted a lot of attentions recently. However, why the deep learning method works is falling far behind its empirical success. In this paper, we provide a rigorous numerical analysis on deep Ritz method (DRM) cite{wan11} for second order elliptic equations with Neumann boundary conditions. We establish the first nonasymptotic convergence rate in $H^1$ norm for DRM using deep networks with $mathrm{ReLU}^2$ activation functions. In addition to providing a theoretical justification of DRM, our study also shed light on how to set the hyper-parameter of depth and width to achieve the desired convergence rate in terms of number of training samples. Technically, we derive bounds on the approximation error of deep $mathrm{ReLU}^2$ network in $H^1$ norm and on the Rademacher complexity of the non-Lipschitz composition of gradient norm and $mathrm{ReLU}^2$ network, both of which are of independent interest.

Recommendations

• A Rate of Convergence of Physics Informed Neural Networks for the Linear Second Order Elliptic PDEs
• A priori and a posteriori error estimates for the deep Ritz method applied to the Laplace and Stokes problem
• A comparison study of deep Galerkin method and deep Ritz method for elliptic problems with different boundary conditions
• The deep Ritz method: a deep learning-based numerical algorithm for solving variational problems
• Deep Nitsche Method: Deep Ritz Method with Essential Boundary Conditions

Cites work

• A practical guide to splines
• An artificial neural network as a troubled-cell indicator
• Approximation rates for neural networks with general activation functions
• Better approximations of high dimensional smooth functions by deep neural networks with rectified power units
• Concentration inequalities. A nonasymptotic theory of independence
• Conservative physics-informed neural networks on discrete domains for conservation laws: applications to forward and inverse problems
• Controlling oscillations in high-order discontinuous Galerkin schemes using artificial viscosity tuned by neural networks
• Convolution Neural Network Shock Detector for Numerical Solution of Conservation Laws
• DGM: a deep learning algorithm for solving partial differential equations
• Error bounds for approximations with deep ReLU networks
• Error bounds for approximations with deep ReLU neural networks in \(W^{s , p}\) norms
• Extended Physics-Informed Neural Networks (XPINNs): A Generalized Space-Time Domain Decomposition Based Deep Learning Framework for Nonlinear Partial Differential Equations
• Finite neuron method and convergence analysis
• fPINNs: Fractional Physics-Informed Neural Networks
• scientific article; zbMATH DE number 5190601 (Why is no real title available?)
• scientific article; zbMATH DE number 1181255 (Why is no real title available?)
• scientific article; zbMATH DE number 7064043 (Why is no real title available?)
• scientific article; zbMATH DE number 5180707 (Why is no real title available?)
• Learning to Discretize: Solving 1D Scalar Conservation Laws via Deep Reinforcement Learning
• Machine learning approximation algorithms for high-dimensional fully nonlinear partial differential equations and second-order backward stochastic differential equations
• Neural algorithm for solving differential equations
• New error bounds for deep ReLU networks using sparse grids
• Nonlinear approximation via compositions
• nPINNs: nonlocal physics-informed neural networks for a parametrized nonlocal universal Laplacian operator. Algorithms and applications
• On the convergence of physics informed neural networks for linear second-order elliptic and parabolic type PDEs
• On the Uniform Convergence of Relative Frequencies of Events to Their Probabilities
• Optimal control of a stochastic heat equation with boundary-noise and boundary-control
• Optimal global rates of convergence for nonparametric regression
• Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations
• Probability in Banach spaces. Isoperimetry and processes
• The deep Ritz method: a deep learning-based numerical algorithm for solving variational problems
• The finite element methods for elliptic problems.
• The sizes of compact subsets of Hilbert space and continuity of Gaussian processes
• Weak adversarial networks for high-dimensional partial differential equations
• Weak convergence and empirical processes. With applications to statistics

Cited in

(39)

• A priori and a posteriori error estimates for the deep Ritz method applied to the Laplace and Stokes problem
• Solving parametric partial differential equations with deep rectified quadratic unit neural networks
• Uniform convergence guarantees for the deep Ritz method for nonlinear problems
• Discovery of subdiffusion problem with noisy data via deep learning
• A Rate of Convergence of Physics Informed Neural Networks for the Linear Second Order Elliptic PDEs
• Imaging conductivity from current density magnitude using neural networks
• A comparison study of deep Galerkin method and deep Ritz method for elliptic problems with different boundary conditions
• Finite neuron method and convergence analysis
• Deep Nitsche Method: Deep Ritz Method with Essential Boundary Conditions
• A New Function Space from Barron Class and Application to Neural Network Approximation
• Stationary Density Estimation of Itô Diffusions Using Deep Learning
• Solving Elliptic Problems with Singular Sources Using Singularity Splitting Deep Ritz Method
• Exponential Convergence of Deep Operator Networks for Elliptic Partial Differential Equations
• Deep convolutional Ritz method: parametric PDE surrogates without labeled data
• A Rate of Convergence of Weak Adversarial Neural Networks for the Second Order Parabolic PDEs
• Error analysis of deep Ritz methods for elliptic equations
• Convergence Analysis of a Quasi-Monte CarloBased Deep Learning Algorithm for Solving Partial Differential Equations
• HRW: Hybrid Residual and Weak Form Loss for Solving Elliptic Interface Problems with Neural Network
• Improved Analysis of PINNs: Alleviate the CoD for Compositional Solutions
• Deep Ritz method for elliptical multiple eigenvalue problems
• Solving PDEs on unknown manifolds with machine learning
• Generalization error in the deep Ritz method with smooth activation functions
• Current density impedance imaging with PINNs
• Generalization of PINNs for elliptic interface problems
• Convergence analysis for over-parameterized deep learning
• Loss jump during loss switch in solving PDEs with neural networks
• WAN discretization of PDEs: best approximation, stabilization, and essential boundary conditions
• Analysis of deep Ritz methods for semilinear elliptic equations
• Error analysis of the mixed residual method for elliptic equations
• Error analysis for empirical risk minimization over clipped ReLU networks in solving linear Kolmogorov partial differential equations
• Convergence and error control of consistent PINNs for elliptic PDEs
• DRM revisited: a complete error analysis
• A weaker adversarial neural networks method for linear second order elliptic PDEs
• Solving the inverse potential problem in the parabolic equation by the deep neural networks method
• On the optimal approximation of Sobolev and Besov functions using deep ReLU neural networks
• Convergence analysis of PINNs with over-parameterization
• Solving differential equations by neural networks: goal functionals and verification of results
• Conditional well-posedness and data-driven method for the inverse dynamic source problem in a coupled diffusion system from one single boundary measurement
• Neural networks for solving elliptic problems: analysis and adaptivities

Describes a project that uses

Uses Software

• DGM
• XPINNs
• FPINNs
• DeepXDE

This page was built for publication: Convergence rate analysis for deep Ritz method

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5077692)