A backward SDE method for uncertainty quantification in deep learning

DOI10.3934/DCDSS.2022062MaRDI QIDQ2676245zbMATH OpenOpenAlexFDO

Authors Richard Archibald, Feng Bao, Yanzhao Cao, He Zhang

Publication date 27 September 2022

Published in Discrete and Continuous Dynamical Systems. Series S (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2011.14145

zbMATH Keywords

probabilistic machine learningstochastic gradient descentstochastic neural networksstochastic optimal control

Mathematics Subject Classification ID

Computational methods for stochastic equations (aspects of stochastic analysis) (60H35) Artificial neural networks and deep learning (68T07) Optimal stochastic control (93E20)

Abstract: We develop a probabilistic machine learning method, which formulates a class of stochastic neural networks by a stochastic optimal control problem. An efficient stochastic gradient descent algorithm is introduced under the stochastic maximum principle framework. Numerical experiments for applications of stochastic neural networks are carried out to validate the effectiveness of our methodology.

Recommendations

• Gradient convergence of deep learning-based numerical methods for BSDEs
• Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations
• Stochastic training of residual networks in deep learning
• Deep backward schemes for high-dimensional nonlinear PDEs
• Bayesian deep convolutional encoder-decoder networks for surrogate modeling and uncertainty quantification

Cites work

• A General Stochastic Maximum Principle for Optimal Control Problems
• A mean-field optimal control formulation of deep learning
• A New Kind of Accurate Numerical Method for Backward Stochastic Differential Equations
• A numerical scheme for BSDEs
• An efficient gradient projection method for stochastic optimal control problems
• Backward Stochastic Differential Equations in Finance
• scientific article; zbMATH DE number 1325009 (Why is no real title available?)
• Iterative importance sampling algorithms for parameter estimation
• Non-convex optimization for machine learning
• Numerical Algorithms for Forward-Backward Stochastic Differential Equations
• On some recent aspects of stochastic control and their applications
• Recent developments in numerical methods for fully nonlinear second order partial differential equations
• Representation theorems for backward stochastic differential equations
• Solving forward-backward stochastic differential equations explicitly -- a four step scheme
• Stable architectures for deep neural networks

Cited in

(9)

• Nonparametric uncertainty quantification for stochastic gradient flows
• Stochastic training of residual networks in deep learning
• scientific article; zbMATH DE number 7326170 (Why is no real title available?)
• Epistemic uncertainty quantification in deep learning classification by the delta method
• Numerical Analysis for Convergence of a Sample-Wise Backpropagation Method for Training Stochastic Neural Networks
• An optimal-order backpropagation algorithm for stochastic neural networks
• Non-Markovian optimal switching control for hybrid renewable energy systems
• Stochastic operator network: a stochastic maximum principle based approach to operator learning
• Finite expression method with TranNet-based function learning for high-dimensional partial differential equations

Describes a project that uses

Uses Software

• SBEED

This page was built for publication: A backward SDE method for uncertainty quantification in deep learning

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2676245)