User-friendly Introduction to PAC-Bayes Bounds

DOI10.1561/2200000100arXiv2110.11216OpenAlexW3207223249WikidataQ129595315 ScholiaQ129595315MaRDI QIDQ6126000

Pierre Alquier

Publication date: 9 April 2024

Published in: Foundations and Trends® in Machine Learning (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/2110.11216

zbMATH Keywords

Bayesian learningonline learningstatistical learning theorymodel choicevariational inferencedeep learningclassification and predictioninformation theory and statisticslearning and statistical methods

Mathematics Subject Classification ID

Nonparametric regression and quantile regression (62G08) Classification and discrimination; cluster analysis (statistical aspects) (62H30) Inequalities; stochastic orderings (60E15) Artificial neural networks and deep learning (68T07) Bayesian inference (62F15) Learning and adaptive systems in artificial intelligence (68T05) Research exposition (monographs, survey articles) pertaining to computer science (68-02)

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Sparse regression learning by aggregation and Langevin Monte-Carlo
• Model selection for weakly dependent time series forecasting
• Pseudo-Bayesian quantum tomography with rank-adaptation
• Deviation inequalities for sums of weakly dependent time series
• Robust linear least squares regression
• Deviation optimal learning using greedy \(Q\)-aggregation
• PAC-Bayesian bounds for randomized empirical risk minimizers
• Learning by mirror averaging
• Gibbs posterior for variable selection in high-dimensional classification and data mining
• Aggregation by exponential weighting, sharp PAC-Bayesian bounds and sparsity
• Modeling by shortest data description
• Microchoice bounds and self bounding learning algorithms
• PAC-Bayesian stochastic model selection
• Aggregating regression procedures to improve performance
• Smooth discrimination analysis
• 1-bit matrix completion: PAC-Bayesian analysis of a variational approximation
• Simpler PAC-Bayesian bounds for hostile data
• Inconsistency of Bayesian inference for misspecified linear models, and a proposal for repairing it
• Bayesian fractional posteriors
• Robust dimension-free Gram operator estimates
• Consistency of variational Bayes inference for estimation and model selection in mixtures
• On the exponentially weighted aggregate with the Laplace prior
• Functional aggregation for nonparametric regression.
• Concentration of measure inequalities for Markov chains and \(\Phi\)-mixing processes.
• Challenging the empirical mean and empirical variance: a deviation study
• Tighter PAC-Bayes bounds through distribution-dependent priors
• Sharp oracle inequalities for aggregation of affine estimators
• PAC-Bayesian estimation and prediction in sparse additive models
• PAC-Bayesian bounds for sparse regression estimation with exponential weights
• On the optimality of the aggregate with exponential weights for low temperatures
• Exponential weights in multivariate regression and a low-rankness favoring prior
• \(\alpha\)-variational inference with statistical guarantees
• Concentration of tempered posteriors and of their variational approximations
• Convergence rates of variational posterior distributions
• On PAC-Bayesian bounds for random forests
• A Bayesian approach for noisy matrix completion: optimal rate under general sampling distribution
• Fast learning rates in statistical inference through aggregation
• PAC-Bayesian risk bounds for group-analysis sparse regression by exponential weighting
• Optimal rates and adaptation in the single-index model using aggregation
• Aggregation of affine estimators
• Empirical minimization
• Weak dependence. With examples and applications.
• Gibbs posterior concentration rates under sub-exponential type losses
• On some recent advances on high dimensional Bayesian statistics
• Bayesian Methods for Low-Rank Matrix Estimation: Short Survey and Theoretical Study
• On the properties of variational approximations of Gibbs posteriors
• PAC-Bayesian Inequalities for Martingales
• PAC-Bayesian Generalisation Error Bounds for Gaussian Process Classification
• Asymptotically Minimax Adaptive Estimation. I: Upper Bounds. Optimally Adaptive Estimates
• Online Learning and Online Convex Optimization
• Information-theoretic upper and lower bounds for statistical estimation
• Information Theory and Mixing Least-Squares Regressions
• Sequential Procedures for Aggregating Arbitrary Estimators of a Conditional Mean
• A theory of the learnable
• Asymptotic evaluation of certain Markov process expectations for large time—III
• How to use expert advice
• Adaptive Regression by Mixing
• Inégalités de Hoeffding pour les fonctions lipschitziennes de suites dépendantes
• Learners that Use Little Information
• High-Dimensional Statistics
• A PAC-Bayesian margin bound for linear classifiers
• The minimum description length principle in coding and modeling
• 10.1162/1532443041424300
• Distribution-Dependent PAC-Bayes Priors
• A PAC-Bayes Bound for Tailored Density Estimation
• Calibrating general posterior credible regions
• Transfer Meta-Learning: Information- Theoretic Bounds and Information Meta-Risk Minimization
• How Much Does Your Data Exploration Overfit? Controlling Bias via Information Usage
• Learning Theory and Kernel Machines
• A General Framework for Updating Belief Distributions
• Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems
• Sparse single-index model
• Prediction of time series by statistical learning: general losses and fast rates
• Occam’s Hammer
• Prediction, Learning, and Games
• Fundamentals of Nonparametric Bayesian Inference
• Convexity, Classification, and Risk Bounds