Deep learning: a Bayesian perspective

DOI10.1214/17-BA1082MaRDI QIDQ1699718zbMATH OpenFDO

Authors Nicholas G. Polson, Vadim Sokolov

Publication date 23 February 2018

Published in Bayesian Analysis (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1706.00473, https://projecteuclid.org/euclid.ba/1510801992

zbMATH Keywords

tensorflowpredictiondeep learningmachine learningartificial intelligencepattern matchingBayesian hierarchical modelsLSTM models

Mathematics Subject Classification ID

Bayesian inference (62F15) Learning and adaptive systems in artificial intelligence (68T05)

Abstract: Deep learning is a form of machine learning for nonlinear high dimensional pattern matching and prediction. By taking a Bayesian probabilistic perspective, we provide a number of insights into more efficient algorithms for optimisation and hyper-parameter tuning. Traditional high-dimensional data reduction techniques, such as principal component analysis (PCA), partial least squares (PLS), reduced rank regression (RRR), projection pursuit regression (PPR) are all shown to be shallow learners. Their deep learning counterparts exploit multiple deep layers of data reduction which provide predictive performance gains. Stochastic gradient descent (SGD) training optimisation and Dropout (DO) regularization provide estimation and variable selection. Bayesian regularization is central to finding weights and connections in networks to optimize the predictive bias-variance trade-off. To illustrate our methodology, we provide an analysis of international bookings on Airbnb. Finally, we conclude with directions for future research.

Recommendations

• Deep learning: an introduction for applied mathematicians
• Deep learning
• Learning sparse deep neural networks with a spike-and-slab prior
• A Bayesian perspective of statistical machine learning for big data
• Consistent Sparse Deep Learning: Theory and Computation

Cited in

(25)

• Comparison of deep neural networks and deep hierarchical models for spatio-temporal data
• Weighted Bayesian bootstrap for scalable posterior distributions
• Multimodel ensemble analysis with neural network Gaussian processes
• Uncertainty quantification in scientific machine learning: methods, metrics, and comparisons
• Deep learning for spatio-temporal modeling: dynamic traffic flows and high frequency trading
• Deep learning for energy markets
• Bayesian Deep Net GLM and GLMM
• Deep learning: computational aspects
• Marginally Calibrated Deep Distributional Regression
• Challenges in Markov chain Monte Carlo for Bayesian neural networks
• Understanding Deep Learning with Statistical Relevance
• Copula deep learning control chart for multivariate zero inflated count response variables
• Computer model calibration with time series data using deep learning and quantile regression
• Industrial Forecasting with Exponentially Smoothed Recurrent Neural Networks
• The application of Bayesian inference under SAFE model
• Detecting conflicting summary statistics in likelihood-free inference
• Bayesian distillation of deep learning models
• Deep learning: a statistical viewpoint
• Marginally calibrated response distributions for end-to-end learning in autonomous driving
• Deep learning from a statistical perspective
• Learning Summary Statistic for Approximate Bayesian Computation via Deep Neural Network
• scientific article; zbMATH DE number 7626754 (Why is no real title available?)
• Bayesian quickest detection of credit card fraud
• Informative Bayesian neural network priors for weak signals
• Data augmentation for Bayesian deep learning

Describes a project that uses

Uses Software

• BASS
• XGBoost
• TensorFlow

This page was built for publication: Deep learning: a Bayesian perspective

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1699718)