Sparse regression learning by aggregation and Langevin Monte-Carlo

DOI10.1016/j.jcss.2011.12.023zbMath1244.62054arXiv0903.1223OpenAlexW1547983335MaRDI QIDQ439987

Arnak S. Dalalyan, Alexandre B. Tsybakov

Publication date: 17 August 2012

Published in: Journal of Computer and System Sciences (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/0903.1223

zbMATH Keywords

logistic regressionregression estimationoracle inequalitiessparsity priorsparse learning

Mathematics Subject Classification ID

Nonparametric regression and quantile regression (62G08) Generalized linear models (logistic models) (62J12) Monte Carlo methods (65C05) Learning and adaptive systems in artificial intelligence (68T05)

Related Items

Entropic optimal transport is maximum-likelihood deconvolution, Weighted multilevel Langevin simulation of invariant measures, Noisy Monte Carlo: convergence of Markov chains with approximate transition kernels, Probabilistic learning inference of boundary value problem with uncertainties based on Kullback-Leibler divergence under implicit constraints, PAC-Bayesian high dimensional bipartite ranking, Sparse recovery under matrix uncertainty, Exponential weights in multivariate regression and a low-rankness favoring prior, PAC-Bayesian risk bounds for group-analysis sparse regression by exponential weighting, Functional inequalities for perturbed measures with applications to log-concave measures and to some Bayesian problems, A reduced-rank approach to predicting multiple binary responses through machine learning, Ergodicity of supercritical SDEs driven by \(\alpha \)-stable processes and heavy-tailed sampling, User-friendly Introduction to PAC-Bayes Bounds, Sharp oracle inequalities for aggregation of affine estimators, Simple proof of the risk bound for denoising by exponential weights for asymmetric noise distributions, Non-convex penalized estimation in high-dimensional models with single-index structure, PAC-Bayesian estimation and prediction in sparse additive models, Sparse regression learning by aggregation and Langevin Monte-Carlo, PAC-Bayesian bounds for sparse regression estimation with exponential weights, Mirror averaging with sparsity priors, Statistical inference in compound functional models, Optimal learning with \textit{Q}-aggregation, Entropy-based closure for probabilistic learning on manifolds, Probabilistic learning on manifolds constrained by nonlinear partial differential equations for small datasets, On Stochastic Gradient Langevin Dynamics with Dependent Data Streams: The Fully Nonconvex Case, Exponential screening and optimal rates of sparse estimation, Prediction error bounds for linear regression with the TREX, The tamed unadjusted Langevin algorithm, A quasi-Bayesian perspective to online clustering, Approximate models and robust decisions, Selection of KL neighbourhood in robust Bayesian inference, Sparse estimation by exponential weighting, On the exponentially weighted aggregate with the Laplace prior, Sharp oracle inequalities for low-complexity priors, Sampling from non-smooth distributions through Langevin diffusion

Uses Software

• GGMselect
• EBayesThresh

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Nearly unbiased variable selection under minimax concave penalty
• Graph Selection with GGMselect
• Sparse regression learning by aggregation and Langevin Monte-Carlo
• Mirror averaging with sparsity priors
• Markov chains and stochastic stability
• PAC-Bayesian bounds for randomized empirical risk minimizers
• Nonlinear estimation over weak Besov spaces and minimax Bayes
• The sparsity and bias of the LASSO selection in high-dimensional linear regression
• Learning by mirror averaging
• Aggregation by exponential weighting, sharp PAC-Bayesian bounds and sparsity
• Sparse recovery in convex hulls via entropy penalization
• Prediction of stochastic sequences
• The weighted majority algorithm
• PAC-Bayesian stochastic model selection
• Langevin diffusions and Metropolis-Hastings algorithms
• Aggregating regression procedures to improve performance
• Least angle regression. (With discussion)
• Statistical learning theory and stochastic optimization. Ecole d'Eté de Probabilitiés de Saint-Flour XXXI -- 2001.
• On the optimality of the aggregate with exponential weights for low temperatures
• Fast learning rates in statistical inference through aggregation
• Simultaneous analysis of Lasso and Dantzig selector
• High-dimensional generalized linear models and the lasso
• Sparsity oracle inequalities for the Lasso
• Optimal rates and adaptation in the single-index model using aggregation
• Aggregation for Gaussian regression
• On optimality of Bayesian testimation in the normal means problem
• The Dantzig selector: statistical estimation when \(p\) is much larger than \(n\). (With discussions and rejoinder).
• High-dimensional graphs and variable selection with the Lasso
• Empirical Bayes selection of wavelet thresholds
• On the Generalization Ability of On-Line Learning Algorithms
• Stable recovery of sparse overcomplete representations in the presence of noise
• Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies?
• Information Theory and Mixing Least-Squares Regressions
• Time-reversible diffusions
• How to use expert advice
• An Empirical Bayesian Strategy for Solving the Simultaneous Sparse Approximation Problem
• Sequential prediction of individual sequences under general loss functions
• Aggregation and Sparsity Via ℓ1 Penalized Least Squares
• Regularization and Variable Selection Via the Elastic Net
• Aggregation by Exponential Weighting and Sharp Oracle Inequalities
• Prediction, Learning, and Games