A general framework for Bayes structured linear models
From MaRDI portal
Abstract: High dimensional statistics deals with the challenge of extracting structured information from complex model settings. Compared with the growing number of frequentist methodologies, there are rather few theoretically optimal Bayes methods that can deal with very general high dimensional models. In contrast, Bayes methods have been extensively studied in various nonparametric settings and rate optimal posterior contraction results have been established. This paper provides a unified approach to both Bayes high dimensional statistics and Bayes nonparametrics in a general framework of structured linear models. With the proposed two-step model selection prior, we prove a general theorem of posterior contraction under an abstract setting. The main theorem can be used to derive new results on optimal posterior contraction under many complex model settings including stochastic block model, graphon estimation and dictionary learning. It can also be used to re-derive optimal posterior contraction for problems such as sparse linear regression and nonparametric aggregation, which improve upon previous Bayes results for these problems. The key of the success lies in the proposed two-step prior distribution. The prior on the parameters is an elliptical Laplace distribution that is capable to model signals with large magnitude, and the prior on the models involves an important correction factor that compensates the effect of the normalizing constant of the elliptical Laplace distribution.
The paper under review provides a unified methodology and theory for both Bayes high dimensional statistics and Bayes nonparametric statistics in a general framework of structured linear models. The authors first introduce a unified view of various high dimensional and nonparametric models, and then propose a single prior distribution for all models in the considered framework. Optimal rates of convergence of the posterior distributions are established under appropriate conditions. The results directly lead to exact minimax posterior contraction rates in stochastic block model, biclustering, sparse linear regression, regression with group sparsity, multitask learning and dictionary learning. Moreover, a general posterior oracle inequality, which allows arbitrary model misspecification, is also derived. The main results are illustrated by examples ranging from nonparametric estimation to high dimensional statistics.
Recommendations
Cites work
- A Clustering Approach to Learning Sparsely Used Overcomplete Dictionaries
- Adaptive Bayesian inference in the Gaussian sequence model using exponential-variance priors
- Adaptive Minimax Estimation over Sparse \ell_q-Hulls
- Aggregating regression procedures to improve performance
- Aggregation and minimax optimality in high-dimensional estimation
- An asymptotic property of model selection criteria
- Asymptotic equivalence of density estimation and Gaussian white noise
- Asymptotic equivalence of nonparametric regression and white noise
- Bayesian community detection
- Bayesian linear regression with sparse priors
- Combining different procedures for adaptive regression
- Convergence rates of posterior distributions.
- Decoding by Linear Programming
- Empirical Bayes posterior concentration in sparse high-dimensional linear models
- Exponential screening and optimal rates of sparse estimation
- Gaussian model selection
- Graph limits and exchangeable random graphs
- scientific article; zbMATH DE number 45785 (Why is no real title available?)
- scientific article; zbMATH DE number 1321826 (Why is no real title available?)
- scientific article; zbMATH DE number 1522808 (Why is no real title available?)
- scientific article; zbMATH DE number 845714 (Why is no real title available?)
- Information Theory and Mixing Least-Squares Regressions
- Large networks and graph limits
- Learning Theory and Kernel Machines
- Limits of dense graph sequences
- Minimax Rates of Estimation for High-Dimensional Linear Regression Over \ell_q-Balls
- Minimum complexity density estimation
- Misspecification in infinite-dimensional Bayesian statistics
- Model Selection and Estimation in Regression with Grouped Variables
- Needles and straw in a haystack: posterior concentration for possibly sparse sequences
- On adaptive posterior concentration rates
- On Bayesian supremum norm contraction rates
- On the computational complexity of high-dimensional Bayesian variable selection
- On the conditions used to prove oracle results for the Lasso
- On the representation theorem for exchangeable arrays
- Oracle inequalities and optimal inference under group sparsity
- Posterior concentration rates for infinite dimensional exponential families
- Posterior consistency of Dirichlet mixtures in density estimation
- Posterior convergence rates for estimating large precision matrices using graphical models
- Pushing the Limits of Contemporary Statistics: Contributions in Honor of Jayanta K. Ghosh
- Rate exact Bayesian adaptation with modified block priors
- Rate-optimal graphon estimation
- Rate-optimal posterior contraction for sparse PCA
- Rates of contraction of posterior distributions based on Gaussian process priors
- Rates of convergence of posterior distributions.
- Representations for partially exchangeable arrays of random variables
- Risk bounds for model selection via penalization
- Simultaneous analysis of Lasso and Dantzig selector
- Sparse estimation by exponential weighting
- Stable recovery of sparse overcomplete representations in the presence of noise
- Statistical learning theory and stochastic optimization. Ecole d'Eté de Probabilitiés de Saint-Flour XXXI -- 2001.
- Statistics for high-dimensional data. Methods, theory and applications.
- Structured matrix estimation and completion
- Sup-norm convergence rate and sign concentration property of Lasso and Dantzig estimators
- The consistency of posterior distributions in nonparametric problems
- The sparsity and bias of the LASSO selection in high-dimensional linear regression
- Volume Ratio, Sparsity, and Minimaxity Under Unitarily Invariant Norms
Cited in
(28)- Minimax rates in network analysis: graphon estimation, community detection and hypothesis testing
- Oracle posterior contraction rates under hierarchical priors
- Unified Bayesian theory of sparse linear regression with nuisance parameters
- Posterior analysis of n in the binomial (n,p) problem with both parameters unknown -- with applications to quantitative nanoscopy
- On the robustness of minimum norm interpolators and regularized empirical risk minimizers
- Posterior contraction in group sparse logit models for categorical responses
- Iterative algorithm for discrete structure recovery
- Bayesian linear regression for multivariate responses under group sparsity
- Posterior asymptotic normality for an individual coordinate in high-dimensional linear regression
- A uniform framework for the combination of penalties in generalized structured models
- Test on stochastic block model: local smoothing and extreme value theory
- Nearly optimal Bayesian shrinkage for high-dimensional regression
- Optimal Bayesian estimation of Gaussian mixtures with growing number of components
- A general framework for model-based statistics
- Model choice in generalised linear models: a Bayesian approach via Kullback-Leibler projections
- A BAYESIAN HIERARCHICAL LINEAR MODEL WITH EDUCATIONAL APPLICATIONS
- Optimal false discovery control of minimax estimators
- Bayesian learning of graph substructures
- Complexity analysis of Bayesian learning of high-dimensional DAG models and their equivalence classes
- Adaptive variational Bayes: optimality, computation and applications
- Concentration of posterior model probabilities and normalized \({L_0}\) criteria
- Recent advances on mechanisms of network generation: community, exchangeability, and scale-free properties
- Empirical priors and posterior concentration in a piecewise polynomial sequence model
- Adaptive posterior concentration rates for sparse high-dimensional linear regression with random design and unknown error variance
- Concentration of a sparse Bayesian model with horseshoe prior in estimating high-dimensional precision matrix
- Bayesian inference in high-dimensional models
- Posterior ramifications of prior dependence structures
- Spatiotemporal Besov Priors for Bayesian Inverse Problems
This page was built for publication: A general framework for Bayes structured linear models
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2215762)
