Adaptive activation functions accelerate convergence in deep and physics-informed neural networks

From MaRDI portal

Publication:2223034

Jump to:navigation, search

DOI10.1016/j.jcp.2019.109136zbMath1453.68165arXiv1906.01170OpenAlexW2948551291WikidataQ126761242 ScholiaQ126761242MaRDI QIDQ2223034

Kenji Kawaguchi, Ameya D. Jagtap, George Em. Karniadakis

Publication date: 28 January 2021

Published in: Journal of Computational Physics (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/1906.01170

zbMATH Keywords

inverse problems partial differential equations machine learning physics-informed neural networks bad minima deep learning benchmarks

Mathematics Subject Classification ID

Artificial neural networks and deep learning (68T07) Algorithms for approximation of functions (65D15) Numerical methods for inverse problems for initial value and initial-boundary value problems involving PDEs (65M32)

Related Items (93)

An efficient neural network method with plane wave activation functions for solving Helmholtz equation ⋮ The data-driven localized wave solutions of the derivative nonlinear Schrödinger equation by using improved PINN approach ⋮ NSFnets (Navier-Stokes flow nets): physics-informed neural networks for the incompressible Navier-Stokes equations ⋮ When Do Extended Physics-Informed Neural Networks (XPINNs) Improve Generalization? ⋮ Weak form theory-guided neural network (TgNN-wf) for deep learning of subsurface single- and two-phase flow ⋮ On obtaining sparse semantic solutions for inverse problems, control, and neural network training ⋮ PFNN-2: A Domain Decomposed Penalty-Free Neural Network Method for Solving Partial Differential Equations ⋮ Generalization Error Analysis of Neural Networks with Gradient Based Regularization ⋮ Theory-guided hard constraint projection (HCP): a knowledge-based data-driven scientific machine learning method ⋮ Parallel physics-informed neural networks via domain decomposition ⋮ Hybrid FEM-NN models: combining artificial neural networks with the finite element method ⋮ Physics-informed neural networks for the shallow-water equations on the sphere ⋮ DeLISA: deep learning based iteration scheme approximation for solving PDEs ⋮ Learning time-dependent PDEs with a linear and nonlinear separate convolutional neural network ⋮ A two-stage physics-informed neural network method based on conserved quantities and applications in localized wave solutions ⋮ On the Exact Computation of Linear Frequency Principle Dynamics and Its Generalization ⋮ Meta-learning PINN loss functions ⋮ Physics-informed neural networks for inverse problems in supersonic flows ⋮ Scientific machine learning through physics-informed neural networks: where we are and what's next ⋮ Data-driven rogue waves and parameters discovery in nearly integrable \(\mathcal{PT}\)-symmetric Gross-Pitaevskii equations via PINNs deep learning ⋮ HomPINNs: Homotopy physics-informed neural networks for learning multiple solutions of nonlinear elliptic differential equations ⋮ Surrogate modeling for fluid flows based on physics-constrained deep learning without simulation data ⋮ A-PINN: auxiliary physics informed neural networks for forward and inverse problems of nonlinear integro-differential equations ⋮ Adaptive deep neural networks methods for high-dimensional partial differential equations ⋮ Wasserstein generative adversarial uncertainty quantification in physics-informed neural networks ⋮ A gradient-based deep neural network model for simulating multiphase flow in porous media ⋮ Active Neuron Least Squares: A Training Method for Multivariate Rectified Neural Networks ⋮ ModalPINN: an extension of physics-informed neural networks with enforced truncated Fourier decomposition for periodic flow reconstruction using a limited number of imperfect sensors ⋮ Conservative physics-informed neural networks on discrete domains for conservation laws: applications to forward and inverse problems ⋮ Improved deep neural networks with domain decomposition in solving partial differential equations ⋮ A machine learning approach to calculating the non-equilibrium diffusion coefficients in the state-to-state solution of the Navier-Stokes equations ⋮ A unified scalable framework for causal sweeping strategies for physics-informed neural networks (PINNs) and their temporal decompositions ⋮ Distributionally robust unsupervised domain adaptation ⋮ Application of the dynamical system method and the deep learning method to solve the new (3+1)-dimensional fractional modified Benjamin-Bona-Mahony equation ⋮ Physics-informed neural networks with parameter asymptotic strategy for learning singularly perturbed convection-dominated problem ⋮ Three ways to solve partial differential equations with neural networks — A review ⋮ Physics-Informed Neural Networks for Solving Dynamic Two-Phase Interface Problems ⋮ VC-PINN: variable coefficient physics-informed neural network for forward and inverse problems of PDEs with variable coefficient ⋮ A nonlocal energy‐informed neural network for isotropic elastic solids with cracks under thermomechanical loads ⋮ Phase-field DeepONet: physics-informed deep operator neural network for fast simulations of pattern formation governed by gradient flows of free-energy functionals ⋮ Physical restriction neural networks with restarting strategy for solving mathematical model of thermal heat equation for early diagnose breast cancer ⋮ BINN: a deep learning approach for computational mechanics problems based on boundary integral equations ⋮ A dimension-augmented physics-informed neural network (DaPINN) with high level accuracy and efficiency ⋮ On the use of neural networks for full waveform inversion ⋮ FDM-PINN: Physics-informed neural network based on fictitious domain method ⋮ A decoupled physics-informed neural network for recovering a space-dependent force function in the wave equation from integral overdetermination data ⋮ Learning high frequency data via the coupled frequency predictor-corrector triangular DNN ⋮ An extended physics informed neural network for preliminary analysis of parametric optimal control problems ⋮ A continuous convolutional trainable filter for modelling unstructured data ⋮ Physics-informed neural networks combined with polynomial interpolation to solve nonlinear partial differential equations ⋮ Addressing discontinuous root-finding for subsequent differentiability in machine learning, inverse problems, and control ⋮ Variable linear transformation improved physics-informed neural networks to solve thin-layer flow problems ⋮ ADLGM: an efficient adaptive sampling deep learning Galerkin method ⋮ Adaptive task decomposition physics-informed neural networks ⋮ Respecting causality for training physics-informed neural networks ⋮ A novel physics-informed deep learning strategy with local time-updating discrete scheme for multi-dimensional forward and inverse consolidation problems ⋮ Transfer learning based physics-informed neural networks for solving inverse problems in engineering structures under different loading scenarios ⋮ Data-driven forward-inverse problems for Yajima-Oikawa system using deep learning with parameter regularization ⋮ A new method for solving nonlinear partial differential equations based on liquid time-constant networks ⋮ Pre-training physics-informed neural network with mixed sampling and its application in high-dimensional systems ⋮ Parallel physics-informed neural networks method with regularization strategies for the forward-inverse problems of the variable coefficient modified KdV equation ⋮ Physics-informed neural networks for analysis of 2D thin-walled structures ⋮ Learning Specialized Activation Functions for Physics-Informed Neural Networks ⋮ Deep neural networks learning forward and inverse problems of two-dimensional nonlinear wave equations with rational solitons ⋮ PI-VEGAN: Physics Informed Variational Embedding Generative Adversarial Networks for Stochastic Differential Equations ⋮ Subspace decomposition based DNN algorithm for elliptic type multi-scale PDEs ⋮ Adaptive Learning Rate Residual Network Based on Physics-Informed for Solving Partial Differential Equations ⋮ Adaptive transfer learning for PINN ⋮ Physical information neural networks for 2D and 3D nonlinear Biot model and simulation on the pressure of brain ⋮ Comparative study of rogue wave solutions for autonomous and non-autonomous saturable discrete nonlinear Schrödinger equation ⋮ A Neural Network Approach for Homogenization of Multiscale Problems ⋮ On the spectral bias of coupled frequency predictor-corrector triangular DNN: the convergence analysis ⋮ Learning stiff chemical kinetics using extended deep neural operators ⋮ An Adaptive Physics-Informed Neural Network with Two-Stage Learning Strategy to Solve Partial Differential Equations ⋮ Neural Control of Parametric Solutions for High-Dimensional Evolution PDEs ⋮ Physical informed neural networks with soft and hard boundary constraints for solving advection-diffusion equations using Fourier expansions ⋮ Residual-based attention in physics-informed neural networks ⋮ An unsupervised latent/output physics-informed convolutional-LSTM network for solving partial differential equations using peridynamic differential operator ⋮ Physics-informed neural networks based on adaptive weighted loss functions for Hamilton-Jacobi equations ⋮ An overview on deep learning-based approximation methods for partial differential equations ⋮ DeepXDE: A Deep Learning Library for Solving Differential Equations ⋮ Locally adaptive activation functions with slope recovery for deep and physics-informed neural networks ⋮ A Multi-Scale DNN Algorithm for Nonlinear Elliptic Equations with Multiple Scales ⋮ Extended Physics-Informed Neural Networks (XPINNs): A Generalized Space-Time Domain Decomposition Based Deep Learning Framework for Nonlinear Partial Differential Equations ⋮ Deep autoencoder based energy method for the bending, vibration, and buckling analysis of Kirchhoff plates with transfer learning ⋮ Mosaic flows: a transferable deep learning framework for solving PDEs on unseen domains ⋮ CENN: conservative energy method based on neural networks with subdomains for solving variational problems involving heterogeneous and complex geometries ⋮ A physically-informed deep-learning model using time-reversal for locating a source from sparse and highly noisy sensors data ⋮ Galerkin Neural Networks: A Framework for Approximating Variational Equations with Error Control ⋮ A non-gradient method for solving elliptic partial differential equations with deep neural networks ⋮ Physics-informed neural networks for shell structures ⋮ On stability and regularization for data-driven solution of parabolic inverse source problems ⋮ Neural eikonal solver: improving accuracy of physics-informed neural networks for solving eikonal equation in case of caustics

Uses Software

Cites Work

This page was built for publication: Adaptive activation functions accelerate convergence in deep and physics-informed neural networks

Retrieved from "https://portal.mardi4nfdi.de/w/index.php?title=Publication:2223034&oldid=14766023"