Large-scale Bayesian optimal experimental design with derivative-informed projected neural network

DOI10.1007/S10915-023-02145-1zbMATH Open1519.35348arXiv2201.07925OpenAlexW4323567155MaRDI QIDQ6159007FDOQ6159007

Authors: Ke-yi Wu, Thomas O'Leary-Roseberry, Peng Chen, Omar Ghattas 

Publication date: 20 June 2023

Published in: Journal of Scientific Computing (Search for Journal in Brave)

Abstract: We address the solution of large-scale Bayesian optimal experimental design (OED) problems governed by partial differential equations (PDEs) with infinite-dimensional parameter fields. The OED problem seeks to find sensor locations that maximize the expected information gain (EIG) in the solution of the underlying Bayesian inverse problem. Computation of the EIG is usually prohibitive for PDE-based OED problems. To make the evaluation of the EIG tractable, we approximate the (PDE-based) parameter-to-observable map with a derivative-informed projected neural network (DIPNet) surrogate, which exploits the geometry, smoothness, and intrinsic low-dimensionality of the map using a small and dimension-independent number of PDE solves. The surrogate is then deployed within a greedy algorithm-based solution of the OED problem such that no further PDE solves are required. We analyze the EIG approximation error in terms of the generalization error of the DIPNet and show they are of the same order. Finally, the efficiency and accuracy of the method are demonstrated via numerical experiments on OED problems governed by inverse scattering and inverse reactive transport with up to 16,641 uncertain parameters and 100 experimental design variables, where we observe up to three orders of magnitude speedup relative to a reference double loop Monte Carlo method.

Full work available at URL: https://arxiv.org/abs/2201.07925

Recommendations

• A Fast and Scalable Computational Framework for Large-Scale High-Dimensional Bayesian Optimal Experimental Design
• Bayesian sequential optimal experimental design for nonlinear models using policy gradient reinforcement learning
• An ODE-based neural network with Bayesian optimization
• Unbiased MLMC stochastic gradient-based optimization of Bayesian experimental designs
• Nesterov-aided stochastic gradient methods using Laplace approximation for Bayesian design optimization
• Optimal nonparametric inference via deep neural network
• Goal-oriented optimal design of experiments for large-scale Bayesian linear inverse problems
• BayesOpt: a Bayesian optimization library for nonlinear optimization, experimental design and bandits
• Bayesian inference of non-linear multiscale model parameters accelerated by a deep neural network

zbMATH Keywords

surrogategreedy algorithmBayesian inverse problemneural networkBayesian optimal experimental designexpected information gain

Mathematics Subject Classification ID

Random number generation in numerical analysis (65C10) Gaussian processes (60G15) Artificial neural networks and deep learning (68T07) Numerical computation of eigenvalues and eigenvectors of matrices (65F15) Partial differential equations of mathematical physics and other areas of application (35Q99) Integro-partial differential equations (35R09) Inverse problems for PDEs (35R30) PDEs with randomness, stochastic partial differential equations (35R60) Probabilistic methods, particle methods, etc. for initial value and initial-boundary value problems involving PDEs (65M75)

Cites Work

• Reduced basis methods for partial differential equations. An introduction
• hIPPYlib
• Randomized matrix-free trace and log-determinant estimators
• Optimal Measurement Methods for Distributed Parameter System Identification
• Inverse problems: a Bayesian perspective
• A computational framework for infinite-dimensional Bayesian inverse problems. I: The linearized case, with application to global seismic inversion
• An analysis of infinite dimensional Bayesian inverse shape acoustic scattering and its numerical approximation
• Fast algorithms for Bayesian uncertainty quantification in large-scale linear inverse problems based on low-rank partial Hessian approximations
• Model reduction and neural networks for parametric PDEs
• A computational framework for infinite-dimensional Bayesian inverse problems. II: stochastic Newton MCMC with application to ice sheet flow inverse problems
• Where did the tumor start? An inverse solver with sparse localization for tumor growth models
• On Bayesian A- and D-optimal experimental designs in infinite dimensions
• Dimension-independent likelihood-informed MCMC
• Dimensionality reduction of parameter-dependent problems through proper orthogonal decomposition
• A Hessian-based method for uncertainty quantification in global Ocean state estimation
• GRADIENT-BASED STOCHASTIC OPTIMIZATION METHODS IN BAYESIAN EXPERIMENTAL DESIGN
• Scalable and efficient algorithms for the propagation of uncertainty from data through inference to prediction for large-scale problems, with application to flow of the antarctic ice sheet
• Fast estimation of expected information gains for Bayesian experimental designs based on Laplace approximations
• A-optimal design of experiments for infinite-dimensional Bayesian linear inverse problems with regularized \(\ell_0\)-sparsification
• A Fast and Scalable Method for A-Optimal Design of Experiments for Infinite-dimensional Bayesian Nonlinear Inverse Problems
• Goal-oriented optimal design of experiments for large-scale Bayesian linear inverse problems
• Hessian-based adaptive sparse quadrature for infinite-dimensional Bayesian inverse problems
• Certified dimension reduction in nonlinear Bayesian inverse problems
• Geometric MCMC for infinite-dimensional inverse problems
• Fast Bayesian optimal experimental design for seismic source inversion
• POD-DL-ROM: enhancing deep learning-based reduced order models for nonlinear parametrized PDEs by proper orthogonal decomposition
• Fast Bayesian experimental design: Laplace-based importance sampling for the expected information gain
• Gradient-based dimension reduction of multivariate vector-valued functions
• Derivative-informed projected neural networks for high-dimensional parametric maps governed by PDEs
• A-optimal encoding weights for nonlinear inverse problems, with application to the Helmholtz inverse problem
• Learning physics-based models from data: perspectives from inverse problems and model reduction
• The Random Feature Model for Input-Output Maps between Banach Spaces
• Learning high-dimensional parametric maps via reduced basis adaptive residual networks
• Title not available (Why is that?)

Cited In (14)

• A framework for strategic discovery of credible neural network surrogate models under uncertainty
• A Fast and Scalable Computational Framework for Large-Scale High-Dimensional Bayesian Optimal Experimental Design
• A greedy sensor selection algorithm for hyperparameterized linear Bayesian inverse problems with correlated noise models
• Residual-based error correction for neural operator accelerated Infinite-dimensional Bayesian inverse problems
• Optimal experimental design: formulations and computations
• PyOED: an extensible suite for data assimilation and model-constrained optimal design of experiments
• Stochastic learning approach for binary optimization: application to Bayesian optimal design of experiments
• A scalable framework for multi-objective PDE-constrained design of building insulation under uncertainty
• Residual-based error corrector operator to enhance accuracy and reliability of neural operator surrogates of nonlinear variational boundary-value problems
• Optimal Bayesian experimental design for subsurface flow problems
• Derivative-informed neural operator: an efficient framework for high-dimensional parametric derivative learning
• Optimal experimental design for prediction based on push-forward probability measures
• Variational Bayesian optimal experimental design with normalizing flows
• Bayesian sequential optimal experimental design for nonlinear models using policy gradient reinforcement learning

This page was built for publication: Large-scale Bayesian optimal experimental design with derivative-informed projected neural network

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6159007)