Reliable extrapolation of deep neural operators informed by physics or sparse observations
From MaRDI portal
Publication:6097626
Abstract: Deep neural operators can learn nonlinear mappings between infinite-dimensional function spaces via deep neural networks. As promising surrogate solvers of partial differential equations (PDEs) for real-time prediction, deep neural operators such as deep operator networks (DeepONets) provide a new simulation paradigm in science and engineering. Pure data-driven neural operators and deep learning models, in general, are usually limited to interpolation scenarios, where new predictions utilize inputs within the support of the training set. However, in the inference stage of real-world applications, the input may lie outside the support, i.e., extrapolation is required, which may result to large errors and unavoidable failure of deep learning models. Here, we address this challenge of extrapolation for deep neural operators. First, we systematically investigate the extrapolation behavior of DeepONets by quantifying the extrapolation complexity via the 2-Wasserstein distance between two function spaces and propose a new behavior of bias-variance trade-off for extrapolation with respect to model capacity. Subsequently, we develop a complete workflow, including extrapolation determination, and we propose five reliable learning methods that guarantee a safe prediction under extrapolation by requiring additional information -- the governing PDEs of the system or sparse new observations. The proposed methods are based on either fine-tuning a pre-trained DeepONet or multifidelity learning. We demonstrate the effectiveness of the proposed framework for various types of parametric PDEs. Our systematic comparisons provide practical guidelines for selecting a proper extrapolation method depending on the available information, desired accuracy, and required inference speed.
Recommendations
- A comprehensive and fair comparison of two neural operators (with practical extensions) based on FAIR data
- Physics informed WNO
- Predictions of transient vector solution fields with sequential deep operator network
- On the training and generalization of deep operator networks
- Neural operator prediction of linear instability waves in high-speed boundary layers
Cites work
- A composite neural network that learns from multi-fidelity data: application to function approximation and inverse PDE problems
- A comprehensive and fair comparison of two neural operators (with practical extensions) based on FAIR data
- A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networks
- A physics-informed operator regression framework for extracting data-driven continuum models
- A physics-informed variational DeepONet for predicting crack path in quasi-brittle materials
- A seamless multiscale operator neural network for inferring bubble dynamics
- B-DeepONet: an enhanced Bayesian deeponet for solving noisy parametric PDEs using accelerated replica exchange SGLD
- Bi-fidelity modeling of uncertain and partially unknown systems using DeepONets
- Deep double descent: where bigger models and more data hurt*
- Deep solution operators for variational inequalities via proximal neural networks
- DeepM\&Mnet for hypersonics: predicting the coupled flow and finite-rate chemistry behind a normal shock using neural-network approximation of operators
- DeepM\&Mnet: inferring the electroconvection multiphysics fields based on operator approximation by neural networks
- DeepXDE: a deep learning library for solving differential equations
- Error analysis for physics-informed neural networks (PINNs) approximating Kolmogorov PDEs
- Error estimates for DeepONets: a deep learning framework in infinite dimensions
- Exponential convergence of mixed \(hp\)-DGFEM for the incompressible Navier-Stokes equations in \(\mathbb{R}^2\)
- scientific article; zbMATH DE number 7626805 (Why is no real title available?)
- Interfacing finite elements with deep neural operators for fast multiscale modeling of mechanics problems
- Locally adaptive activation functions with slope recovery for deep and physics-informed neural networks
- MIONet: Learning Multiple-Input Operators via Tensor Product
- Multilayer feedforward networks are universal approximators
- Neural operator prediction of linear instability waves in high-speed boundary layers
- Nonlocal kernel network (NKN): a stable and resolution-independent deep neural network
- On a Formula for the L2 Wasserstein Metric between Measures on Euclidean and Hilbert Spaces
- On the influence of over-parameterization in manifold based surrogates and deep neural operators
- Overcoming catastrophic forgetting in neural networks
- Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations
- Predicting the output from a complex computer code when fast approximations are available
- Quantifying the generalization error in deep learning in terms of data distribution and neural network smoothness
- Reconciling modern machine-learning practice and the classical bias-variance trade-off
- Scalable uncertainty quantification for deep operator networks using randomized priors
- Uncertainty quantification in scientific machine learning: methods, metrics, and comparisons
Cited in
(32)- Neural operator prediction of linear instability waves in high-speed boundary layers
- Conditional physics informed neural networks
- Learning to Predict Physical Properties using Sums of Separable Functions
- Fourier-DeepONet: Fourier-enhanced deep operator networks for full waveform inversion with improved accuracy, generalizability, and robustness
- Branched latent neural maps
- A super-real-time three-dimension computing method of digital twins in space nuclear power
- Local neural operator for solving transient partial differential equations on varied domains
- D2NO: efficient handling of heterogeneous input function spaces with distributed deep neural operators
- Solving parametric elliptic interface problems via interfaced operator network
- Predictions of transient vector solution fields with sequential deep operator network
- MODNO: multi-operator learning with distributed neural operators
- PDE generalization of in-context operator networks: a study on 1D scalar nonlinear conservation laws
- PTPI-DL-ROMs: pre-trained physics-informed deep learning-based reduced order models for nonlinear parametrized PDEs
- Conformalized-DeepONet: a distribution-free framework for uncertainty quantification in deep operator networks
- DeepONet as a multi-operator extrapolation model: distributed pretraining with physics-informed fine-tuning
- Solving forward and inverse partial differential equation problems on unknown manifolds via physics-informed neural operators
- Generalizability of local neural operator: example for elastodynamic problems
- Fast meta-solvers for 3D complex-shape scatterers using neural operators trained on a non-scattering problem
- Physics-informed neural operators for efficient modeling of infiltration in porous media
- Discovery of linear representations for nonautonomous translation-invariant problems
- Tensor decomposition-based neural operator with dynamic mode decomposition for parameterized time-dependent problems
- Artificial to spiking neural networks conversion with calibration in scientific machine learning
- Dual-branch neural operator for enhanced out-of-distribution generalization
- IB-UQ: information bottleneck based uncertainty quantification for neural function regression and neural operator learning
- Latent neural PDE solver: a reduced-order modeling framework for partial differential equations
- Predicting bright and dark solitons in (2+1)-dimensional Bose-Einstein condensates by two types of neural operator networks
- Neural Green's operators for parametric partial differential equations
- Active operator learning with predictive uncertainty quantification for partial differential equations
- Perturbative-NeuSA: A Structured Spectral Framework for Time-Dependent PDEs
- Deep-Learning Solvers and Surrogates for Infinity and p-Laplace Problems
- Why Directly Learning Periodic Trajectories Can Fail
- Learning constitutive relations from indirect observations using deep neural networks
This page was built for publication: Reliable extrapolation of deep neural operators informed by physics or sparse observations
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6097626)
