PAC-Bayesian bounds for randomized empirical risk minimizers

DOI10.3103/S1066530708040017zbMATH Open1260.62038arXiv0712.1698OpenAlexW2067940717MaRDI QIDQ734547FDOQ734547

Pierre Alquier

Publication date: 13 October 2009

Published in: Mathematical Methods of Statistics (Search for Journal in Brave)

Abstract: The aim of this paper is to generalize the PAC-Bayesian theorems proved by Catoni in the classification setting to more general problems of statistical inference. We show how to control the deviations of the risk of randomized estimators. A particular attention is paid to randomized estimators drawn in a small neighborhood of classical estimators, whose study leads to control the risk of the latter. These results allow to bound the risk of very general estimation procedures, as well as to perform model selection.

Full work available at URL: https://arxiv.org/abs/0712.1698

Recommendations

• A strongly quasiconvex PAC-Bayesian bound
• PAC-Bayes bounds with data dependent priors
• A PAC-Bayes Bound for Tailored Density Estimation
• Tighter PAC-Bayes bounds through distribution-dependent priors
• scientific article; zbMATH DE number 7219345
• On the bounded regret of empirical bayes estimators
• scientific article
• scientific article; zbMATH DE number 7625158
• On Bayes risk lower bounds
• Data dependent priors in PAC-Bayes bounds

zbMATH Keywords

classificationstatistical learningregression estimationadaptive inferenceempirical boundrandomized estimator

Mathematics Subject Classification ID

Nonparametric regression and quantile regression (62G08) Classification and discrimination; cluster analysis (statistical aspects) (62H30) Learning and adaptive systems in artificial intelligence (68T05)

Cites Work

• The elements of statistical learning. Data mining, inference, and prediction
• Support-vector networks
• Title not available (Why is that?)
• Boosting the margin: a new explanation for the effectiveness of voting methods
• Title not available (Why is that?)
• Probability Inequalities for Sums of Bounded Random Variables
• Title not available (Why is that?)
• Smooth discrimination analysis
• Statistical behavior and consistency of classification methods based on convex risk minimization.
• Modeling by shortest data description
• Optimal rates and adaptation in the single-index model using aggregation
• Optimal aggregation of classifiers in statistical learning.
• Theory of Classification: a Survey of Some Recent Advances
• Aggregated estimators and empirical complexity for least square regression

Cited In (20)

• From robust tests to Bayes-like posterior distributions
• Challenging the empirical mean and empirical variance: a deviation study
• A PAC-Bayes Bound for Tailored Density Estimation
• Sparse regression learning by aggregation and Langevin Monte-Carlo
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• On some recent advances on high dimensional Bayesian statistics
• Mirror averaging with sparsity priors
• Model selection for weakly dependent time series forecasting
• PAC-Bayesian high dimensional bipartite ranking
• PAC-Bayesian bounds for sparse regression estimation with exponential weights
• PAC-Bayesian estimation and prediction in sparse additive models
• Estimation from nonlinear observations via convex programming with application to bilinear regression
• Gibbs posterior inference on multivariate quantiles
• Tighter PAC-Bayes bounds through distribution-dependent priors
• Prediction of time series by statistical learning: general losses and fast rates
• Gibbs posterior concentration rates under sub-exponential type losses
• Model misspecification, Bayesian versus credibility estimation, and Gibbs posteriors
• User-friendly Introduction to PAC-Bayes Bounds

This page was built for publication: PAC-Bayesian bounds for randomized empirical risk minimizers

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q734547)