When and why PINNs fail to train: a neural tangent kernel perspective

From MaRDI portal

Publication:2136450

Jump to:navigation, search

DOI10.1016/j.jcp.2021.110768OpenAlexW3205073660MaRDI QIDQ2136450

Paris Perdikaris, Xinling Yu, Sifan Wang

Publication date: 10 May 2022

Published in: Journal of Computational Physics (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/2007.14527

zbMATH Keywords

gradient descent multi-task learning physics-informed neural networks scientific machine learning spectral bias

Mathematics Subject Classification ID

Artificial intelligence (68Txx) Numerical methods for partial differential equations, initial value and time-dependent initial-boundary value problems (65Mxx) Partial differential equations of mathematical physics and other areas of application (35Qxx)

Related Items

An efficient neural network method with plane wave activation functions for solving Helmholtz equation, Hybrid FEM-NN models: combining artificial neural networks with the finite element method, Physics-informed neural network simulation of multiphase poroelasticity using stress-split sequential training, PINNsNTK, Improved architectures and training algorithms for deep operator networks, Solving multiscale steady radiative transfer equation using neural networks with uniform stability, Lagrangian dual framework for conservative neural network solutions of kinetic equations, Numerical approximation of partial differential equations by a variable projection method with artificial neural networks, Physics-informed neural networks for learning the homogenized coefficients of multiscale elliptic equations, Scientific machine learning through physics-informed neural networks: where we are and what's next, RPINNs: rectified-physics informed neural networks for solving stationary partial differential equations, Data-driven rogue waves and parameters discovery in nearly integrable \(\mathcal{PT}\)-symmetric Gross-Pitaevskii equations via PINNs deep learning, Physics-informed Karhunen-Loéve and neural network approximations for solving inverse differential equation problems, A-PINN: auxiliary physics informed neural networks for forward and inverse problems of nonlinear integro-differential equations, On computing the hyperparameter of extreme learning machines: algorithm and application to computational PDEs, and comparison with classical and high-order finite elements, A physics-informed neural network technique based on a modified loss function for computational 2D and 3D solid mechanics, A unified scalable framework for causal sweeping strategies for physics-informed neural networks (PINNs) and their temporal decompositions, Physics-informed neural networks with parameter asymptotic strategy for learning singularly perturbed convection-dominated problem, Physics-data combined machine learning for parametric reduced-order modelling of nonlinear dynamical systems in small-data regimes, Kolmogorov n-width and Lagrangian physics-informed neural networks: a causality-conforming manifold for convection-dominated PDEs, Physics-Informed Neural Networks for Solving Dynamic Two-Phase Interface Problems, VC-PINN: variable coefficient physics-informed neural network for forward and inverse problems of PDEs with variable coefficient, Enhanced physics‐informed neural networks for hyperelasticity, A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networks, Physics-informed neural networks for approximating dynamic (hyperbolic) PDEs of second order in time: error analysis and algorithms, BINN: a deep learning approach for computational mechanics problems based on boundary integral equations, Adaptive weighting of Bayesian physics informed neural networks for multitask and multiscale forward and inverse problems, Solving Elliptic Problems with Singular Sources Using Singularity Splitting Deep Ritz Method, A framework based on symbolic regression coupled with eXtended physics-informed neural networks for gray-box learning of equations of motion from data, Physics-informed radial basis network (PIRBN): a local approximating neural network for solving nonlinear partial differential equations, Extension of dynamic mode decomposition for dynamic systems with incomplete information based on t-model of optimal prediction, FDM-PINN: Physics-informed neural network based on fictitious domain method, Pre-training strategy for solving evolution equations based on physics-informed neural networks, Physics-informed neural networks with adaptive localized artificial viscosity, Physics-informed neural networks combined with polynomial interpolation to solve nonlinear partial differential equations, Error convergence and engineering-guided hyperparameter search of PINNs: towards optimized I-FENN performance, Surrogate modeling for Bayesian inverse problems based on physics-informed neural networks, Long-time integration of parametric evolution equations with physics-informed DeepONets, HomPINNs: homotopy physics-informed neural networks for solving the inverse problems of nonlinear differential equations with multiple solutions, Adaptive task decomposition physics-informed neural networks, An extreme learning machine-based method for computational PDEs in higher dimensions, Respecting causality for training physics-informed neural networks, Isogeometric neural networks: a new deep learning approach for solving parameterized partial differential equations, A deep Fourier residual method for solving PDEs using neural networks, Transfer learning based physics-informed neural networks for solving inverse problems in engineering structures under different loading scenarios, A deep double Ritz method (\(\mathrm{D^2RM}\)) for solving partial differential equations using neural networks, Loss-attentional physics-informed neural networks, Koopman operator learning using invertible neural networks, Pre-training physics-informed neural network with mixed sampling and its application in high-dimensional systems, Physics-informed neural networks with two weighted loss function methods for interactions of two-dimensional oceanic internal solitary waves, Deep convolutional Ritz method: parametric PDE surrogates without labeled data, Physics-informed neural networks for analysis of 2D thin-walled structures, Learning Specialized Activation Functions for Physics-Informed Neural Networks, Variable separated physics-informed neural networks based on adaptive weighted loss functions for blood flow model, Error analysis of deep Ritz methods for elliptic equations, Physics-integrated neural differentiable (PiNDiff) model for composites manufacturing, Numerical computation of partial differential equations by hidden-layer concatenated extreme learning machine, Improved training of physics-informed neural networks for parabolic differential equations with sharply perturbed initial conditions, Physics informed and data-based augmented learning in structural health diagnosis, Finite basis physics-informed neural networks (FBPINNs): a scalable domain decomposition approach for solving differential equations, Adaptive Learning Rate Residual Network Based on Physics-Informed for Solving Partial Differential Equations, Inverse modeling of nonisothermal multiphase poromechanics using physics-informed neural networks, Failure-Informed Adaptive Sampling for PINNs, A Neural Network Approach for Homogenization of Multiscale Problems, On the spectral bias of coupled frequency predictor-corrector triangular DNN: the convergence analysis, Feature-adjacent multi-fidelity physics-informed machine learning for partial differential equations, Less Emphasis on Hard Regions: Curriculum Learning of PINNs for Singularly Perturbed Convection-Diffusion-Reaction Problems, Spectral operator learning for parametric PDEs without data reliance, Physics-informed machine learning method with space-time Karhunen-Loève expansions for forward and inverse partial differential equations, Controlled descent training, Learning physical models that can respect conservation laws, Is the neural tangent kernel of PINNs deep learning general partial differential equations always convergent?, Data-driven nonparametric identification of material behavior based on physics-informed neural network with full-field data, CNN-DP: composite neural network with differential propagation for impulsive nonlinear dynamics, Physical informed neural networks with soft and hard boundary constraints for solving advection-diffusion equations using Fourier expansions, Optimization of physics-informed neural networks for solving the nolinear Schrödinger equation, An unsupervised latent/output physics-informed convolutional-LSTM network for solving partial differential equations using peridynamic differential operator, Physics-informed neural networks based on adaptive weighted loss functions for Hamilton-Jacobi equations, CENN: conservative energy method based on neural networks with subdomains for solving variational problems involving heterogeneous and complex geometries, Physics-informed neural networks for shell structures, Data-driven prediction of soliton solutions of the higher-order NLSE via the strongly-constrained PINN method, Self-adaptive physics-informed neural networks, A deep domain decomposition method based on Fourier features

Uses Software

Cites Work

Retrieved from "https://portal.mardi4nfdi.de/w/index.php?title=Publication:2136450&oldid=14641078"