Convergence rate analysis for deep Ritz method

DOI10.4208/CICP.OA-2021-0195zbMATH Open1491.65117arXiv2103.13330OpenAlexW4220826888MaRDI QIDQ5077692FDOQ5077692

Authors: Chenguang Duan, Y. M. Lai, Dingwei Li, Xiliang Lu, Yu Ling Jiao, Jerry Zhijian Yang 

Publication date: 19 May 2022

Published in: Communications in Computational Physics (Search for Journal in Brave)

Abstract: Using deep neural networks to solve PDEs has attracted a lot of attentions recently. However, why the deep learning method works is falling far behind its empirical success. In this paper, we provide a rigorous numerical analysis on deep Ritz method (DRM) cite{wan11} for second order elliptic equations with Neumann boundary conditions. We establish the first nonasymptotic convergence rate in $H^1$ norm for DRM using deep networks with $mathrm{ReLU}^2$ activation functions. In addition to providing a theoretical justification of DRM, our study also shed light on how to set the hyper-parameter of depth and width to achieve the desired convergence rate in terms of number of training samples. Technically, we derive bounds on the approximation error of deep $mathrm{ReLU}^2$ network in $H^1$ norm and on the Rademacher complexity of the non-Lipschitz composition of gradient norm and $mathrm{ReLU}^2$ network, both of which are of independent interest.

Full work available at URL: https://arxiv.org/abs/2103.13330

Recommendations

• A Rate of Convergence of Physics Informed Neural Networks for the Linear Second Order Elliptic PDEs
• A priori and a posteriori error estimates for the deep Ritz method applied to the Laplace and Stokes problem
• A comparison study of deep Galerkin method and deep Ritz method for elliptic problems with different boundary conditions
• The deep Ritz method: a deep learning-based numerical algorithm for solving variational problems
• Deep Nitsche Method: Deep Ritz Method with Essential Boundary Conditions

zbMATH Keywords

convergence rate analysisdeep neural networksdeep Ritz method

Mathematics Subject Classification ID

Nonparametric estimation (62G05) Artificial neural networks and deep learning (68T07) Neural networks for/in biological studies, artificial life and related topics (92B20) Variational methods applied to PDEs (35A15) Boundary value problems for second-order elliptic equations (35J25) Stability and convergence of numerical methods for boundary value problems involving PDEs (65N12) Error bounds for boundary value problems involving PDEs (65N15) Second-order elliptic equations (35J15) PDEs on graphs and networks (ramified or polygonal spaces) (35R02)

Cites Work

• Weak convergence and empirical processes. With applications to statistics
• DGM: a deep learning algorithm for solving partial differential equations
• fPINNs: Fractional Physics-Informed Neural Networks
• Concentration inequalities. A nonasymptotic theory of independence
• A practical guide to splines
• Optimal global rates of convergence for nonparametric regression
• Probability in Banach spaces. Isoperimetry and processes
• The finite element methods for elliptic problems.
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Approximation rates for neural networks with general activation functions
• The sizes of compact subsets of Hilbert space and continuity of Gaussian processes
• Neural algorithm for solving differential equations
• Optimal control of a stochastic heat equation with boundary-noise and boundary-control
• Machine learning approximation algorithms for high-dimensional fully nonlinear partial differential equations and second-order backward stochastic differential equations
• Title not available (Why is that?)
• The deep Ritz method: a deep learning-based numerical algorithm for solving variational problems
• Error bounds for approximations with deep ReLU networks
• Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations
• An artificial neural network as a troubled-cell indicator
• Weak adversarial networks for high-dimensional partial differential equations
• Controlling oscillations in high-order discontinuous Galerkin schemes using artificial viscosity tuned by neural networks
• On the Uniform Convergence of Relative Frequencies of Events to Their Probabilities
• Conservative physics-informed neural networks on discrete domains for conservation laws: applications to forward and inverse problems
• Error bounds for approximations with deep ReLU neural networks in \(W^{s , p}\) norms
• Extended Physics-Informed Neural Networks (XPINNs): A Generalized Space-Time Domain Decomposition Based Deep Learning Framework for Nonlinear Partial Differential Equations
• Nonlinear approximation via compositions
• New error bounds for deep ReLU networks using sparse grids
• Better approximations of high dimensional smooth functions by deep neural networks with rectified power units
• nPINNs: nonlocal physics-informed neural networks for a parametrized nonlocal universal Laplacian operator. Algorithms and applications
• Finite neuron method and convergence analysis
• Learning to Discretize: Solving 1D Scalar Conservation Laws via Deep Reinforcement Learning
• On the convergence of physics informed neural networks for linear second-order elliptic and parabolic type PDEs
• Convolution Neural Network Shock Detector for Numerical Solution of Conservation Laws

Cited In (30)

• Solving parametric partial differential equations with deep rectified quadratic unit neural networks
• Exponential Convergence of Deep Operator Networks for Elliptic Partial Differential Equations
• Error analysis of deep Ritz methods for elliptic equations
• Uniform convergence guarantees for the deep Ritz method for nonlinear problems
• A Rate of Convergence of Weak Adversarial Neural Networks for the Second Order Parabolic PDEs
• Convergence Analysis of a Quasi-Monte CarloBased Deep Learning Algorithm for Solving Partial Differential Equations
• HRW: Hybrid Residual and Weak Form Loss for Solving Elliptic Interface Problems with Neural Network
• Improved Analysis of PINNs: Alleviate the CoD for Compositional Solutions
• Generalization error in the deep Ritz method with smooth activation functions
• Current density impedance imaging with PINNs
• Generalization of PINNs for elliptic interface problems
• Imaging conductivity from current density magnitude using neural networks
• A New Function Space from Barron Class and Application to Neural Network Approximation
• Convergence analysis for over-parameterized deep learning
• Deep Nitsche Method: Deep Ritz Method with Essential Boundary Conditions
• Stationary Density Estimation of Itô Diffusions Using Deep Learning
• Discovery of subdiffusion problem with noisy data via deep learning
• Deep Ritz method for elliptical multiple eigenvalue problems
• Solving Elliptic Problems with Singular Sources Using Singularity Splitting Deep Ritz Method
• Finite neuron method and convergence analysis
• Loss jump during loss switch in solving PDEs with neural networks
• Solving PDEs on unknown manifolds with machine learning
• A comparison study of deep Galerkin method and deep Ritz method for elliptic problems with different boundary conditions
• WAN discretization of PDEs: best approximation, stabilization, and essential boundary conditions
• Analysis of deep Ritz methods for semilinear elliptic equations
• Error analysis of the mixed residual method for elliptic equations
• Error analysis for empirical risk minimization over clipped ReLU networks in solving linear Kolmogorov partial differential equations
• A Rate of Convergence of Physics Informed Neural Networks for the Linear Second Order Elliptic PDEs
• A priori and a posteriori error estimates for the deep Ritz method applied to the Laplace and Stokes problem
• Deep convolutional Ritz method: parametric PDE surrogates without labeled data

Uses Software

• DeepXDE
• DGM
• FPINNs
• XPINNs

This page was built for publication: Convergence rate analysis for deep Ritz method

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5077692)