When do extended physics-informed neural networks (XPINNs) improve generalization?
From MaRDI portal
Publication:5043367
DOI10.1137/21M1447039zbMATH Open1498.65183arXiv2109.09444MaRDI QIDQ5043367FDOQ5043367
Authors: Zheyuan Hu, Ameya D. Jagtap, Kenji Kawaguchi, George Em Karniadakis 
Publication date: 21 October 2022
Published in: SIAM Journal on Scientific Computing (Search for Journal in Brave)
Abstract: Physics-informed neural networks (PINNs) have become a popular choice for solving high-dimensional partial differential equations (PDEs) due to their excellent approximation power and generalization ability. Recently, Extended PINNs (XPINNs) based on domain decomposition methods have attracted considerable attention due to their effectiveness in modeling multiscale and multiphysics problems and their parallelization. However, theoretical understanding on their convergence and generalization properties remains unexplored. In this study, we take an initial step towards understanding how and when XPINNs outperform PINNs. Specifically, for general multi-layer PINNs and XPINNs, we first provide a prior generalization bound via the complexity of the target functions in the PDE problem, and a posterior generalization bound via the posterior matrix norms of the networks after optimization. Moreover, based on our bounds, we analyze the conditions under which XPINNs improve generalization. Concretely, our theory shows that the key building block of XPINN, namely the domain decomposition, introduces a tradeoff for generalization. On the one hand, XPINNs decompose the complex PDE solution into several simple parts, which decreases the complexity needed to learn each part and boosts generalization. On the other hand, decomposition leads to less training data being available in each subdomain, and hence such model is typically prone to overfitting and may become less generalizable. Empirically, we choose five PDEs to show when XPINNs perform better than, similar to, or worse than PINNs, hence demonstrating and justifying our new theory.
Full work available at URL: https://arxiv.org/abs/2109.09444
Recommendations
- On the convergence of physics informed neural networks for linear second-order elliptic and parabolic type PDEs
- On the eigenvector bias of Fourier feature networks: from regression to solving multi-scale PDEs with physics-informed neural networks
- Error analysis for physics-informed neural networks (PINNs) approximating Kolmogorov PDEs
- Error estimates and physics informed augmentation of neural networks for thermally coupled incompressible Navier Stokes equations
- A Rate of Convergence of Physics Informed Neural Networks for the Linear Second Order Elliptic PDEs
Learning and adaptive systems in artificial intelligence (68T05) Computational learning theory (68Q32) Multigrid methods; domain decomposition for initial value and initial-boundary value problems involving PDEs (65M55) Numerical methods for partial differential equations, initial value and time-dependent initial-boundary value problems (65M99)
Cites Work
- On the limited memory BFGS method for large scale optimization
- 10.1162/153244303321897690
- Handbook of numerical methods for hyperbolic problems. Basic and fundamental issues
- Rayleigh‐Ritz‐Galerkin methods for dirichlet's problem using subspaces without boundary conditions
- Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations
- A priori estimates of the population risk for two-layer neural networks
- Non-invasive inference of thrombus material properties with physics-informed neural networks
- NSFnets (Navier-Stokes flow nets): physics-informed neural networks for the incompressible Navier-Stokes equations
- Conservative physics-informed neural networks on discrete domains for conservation laws: applications to forward and inverse problems
- Extended Physics-Informed Neural Networks (XPINNs): A Generalized Space-Time Domain Decomposition Based Deep Learning Framework for Nonlinear Partial Differential Equations
- Adaptive activation functions accelerate convergence in deep and physics-informed neural networks
- On the convergence of physics informed neural networks for linear second-order elliptic and parabolic type PDEs
- Flow over an espresso cup: inferring 3-D velocity and pressure fields from tomographic background oriented Schlieren via physics-informed neural networks
- Generalization in Deep Learning
Cited In (22)
- Data-driven prediction of soliton solutions of the higher-order NLSE via the strongly-constrained PINN method
- Physics-informed neural networks with parameter asymptotic strategy for learning singularly perturbed convection-dominated problem
- Mixed formulation of physics-informed neural networks for thermo-mechanically coupled systems and heterogeneous domains
- Bright-dark rogue wave transition in coupled ab system via the physics-informed neural networks method
- Domain decomposition learning methods for solving elliptic problems
- Multilevel domain decomposition-based architectures for physics-informed neural networks
- An overview on deep learning-based approximation methods for partial differential equations
- Generalization of PINNs for elliptic interface problems
- A unified scalable framework for causal sweeping strategies for physics-informed neural networks (PINNs) and their temporal decompositions
- Physics-informed neural networks for approximating dynamic (hyperbolic) PDEs of second order in time: error analysis and algorithms
- \textit{hp}-VPINNs: variational physics-informed neural networks with domain decomposition
- Enhancing training of physics-informed neural networks using domain decomposition-based preconditioning strategies
- wPINNs: Weak Physics Informed Neural Networks for Approximating Entropy Solutions of Hyperbolic Conservation Laws
- Physical informed neural networks with soft and hard boundary constraints for solving advection-diffusion equations using Fourier expansions
- Physics-Informed Neural Networks for Solving Dynamic Two-Phase Interface Problems
- Tackling the curse of dimensionality in fractional and tempered fractional PDEs with physics-informed neural networks
- A framework based on symbolic regression coupled with eXtended physics-informed neural networks for gray-box learning of equations of motion from data
- Improving weak PINNs for hyperbolic conservation laws: dual norm computation, boundary conditions and systems
- Deep learning-accelerated computational framework based on physics informed neural network for the solution of linear elasticity
- A peridynamic-informed neural network for continuum elastic displacement characterization
- On the eigenvector bias of Fourier feature networks: from regression to solving multi-scale PDEs with physics-informed neural networks
- An unsupervised latent/output physics-informed convolutional-LSTM network for solving partial differential equations using peridynamic differential operator
Uses Software
This page was built for publication: When do extended physics-informed neural networks (XPINNs) improve generalization?
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5043367)
