Oracle inequalities and optimal inference under group sparsity

DOI10.1214/11-AOS896zbMath1306.62156arXiv1007.1771OpenAlexW3122814465MaRDI QIDQ651028

Massimiliano Pontil, Karim Lounici, Sara van de Geer, Alexandre B. Tsybakov

Publication date: 8 December 2011

Published in: The Annals of Statistics (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/1007.1771

zbMATH Keywords

minimax riskoracle inequalitiesgroup lassogroup sparsitystatistical learningpenalized least squaresmoment inequality

Mathematics Subject Classification ID

Linear regression; mixed models (62J05) Minimax procedures in statistical decision theory (62C20) Statistical ranking and selection procedures (62F07)

Related Items (96)

Minimax estimation in multi-task regression under low-rank structures ⋮ Posterior contraction in group sparse logit models for categorical responses ⋮ Adaptive estimation in multivariate response regression with hidden variables ⋮ The benefit of group sparsity in group inference with de-biased scaled group Lasso ⋮ Solution of linear ill-posed problems using overcomplete dictionaries ⋮ Multivariate sparse group lasso for the multivariate multiple linear regression with an arbitrary group structure ⋮ Penalized least square in sparse setting with convex penalty and non Gaussian errors ⋮ Improved linear regression prediction by transfer learning ⋮ A varying-coefficient panel data model with fixed effects: theory and an application to US commercial banks ⋮ Robust grouped variable selection using distributionally robust optimization ⋮ Trace regression model with simultaneously low rank and row(column) sparse parameter ⋮ Structured variable selection via prior-induced hierarchical penalty functions ⋮ Joint rank and variable selection for parsimonious estimation in a high-dimensional finite mixture regression model ⋮ Variable selection and structure identification for varying coefficient Cox models ⋮ Monte Carlo Simulation for Lasso-Type Problems by Estimator Augmentation ⋮ Bayesian linear regression for multivariate responses under group sparsity ⋮ Grouping strategies and thresholding for high dimensional linear models ⋮ ``Grouping strategies and thresholding for high dimensional linear models: discussion ⋮ ``Grouping strategies and thresholding for high dimensional linear models: rejoinder ⋮ Quantile regression with group Lasso for classification ⋮ Variable selection, monotone likelihood ratio and group sparsity ⋮ Grouped variable selection with discrete optimization: computational and statistical perspectives ⋮ Adaptive bi-level variable selection for multivariate failure time model with a diverging number of covariates ⋮ Generalized linear models with structured sparsity estimators ⋮ Joint association and classification analysis of multi‐view data ⋮ Group sparse recovery via group square-root elastic net and the iterative multivariate thresholding-based algorithm ⋮ Variable selection in heterogeneous panel data models with cross‐sectional dependence ⋮ High-Dimensional Gaussian Graphical Regression Models with Covariates ⋮ Non-asymptotic oracle inequalities for the Lasso and Group Lasso in high dimensional logistic model ⋮ A penalized two-pass regression to predict stock returns with time-varying risk premia ⋮ Individual Data Protected Integrative Regression Analysis of High-Dimensional Heterogeneous Data ⋮ Unnamed Item ⋮ Lasso in Infinite dimension: application to variable selection in functional multivariate linear regression ⋮ Statistical and computational limits for sparse matrix detection ⋮ Multi-Task Learning with High-Dimensional Noisy Images ⋮ Adaptive and robust multi-task learning ⋮ Expectile trace regression via low-rank and group sparsity regularization ⋮ Regularizers for structured sparsity ⋮ A group adaptive elastic-net approach for variable selection in high-dimensional linear regression ⋮ Model selection in regression under structural constraints ⋮ Least squares after model selection in high-dimensional sparse models ⋮ Sparse PCA: optimal rates and adaptive estimation ⋮ A general framework for Bayes structured linear models ⋮ On the finite-sample analysis of \(\Theta\)-estimators ⋮ Estimation and variable selection with exponential weights ⋮ Sparse regression and support recovery with \(\mathbb{L}_2\)-boosting algorithms ⋮ Consistent group selection with Bayesian high dimensional modeling ⋮ On the finite-sample analysis of \(\Theta\)-estimators ⋮ Discrete optimization methods for group model selection in compressed sensing ⋮ Robust inference on average treatment effects with possibly more covariates than observations ⋮ Robust regression with compositional covariates ⋮ Unnamed Item ⋮ Decomposable norm minimization with proximal-gradient homotopy algorithm ⋮ Slope meets Lasso: improved oracle bounds and optimality ⋮ Pivotal estimation via square-root lasso in nonparametric regression ⋮ Estimation of high-dimensional low-rank matrices ⋮ Greedy variance estimation for the LASSO ⋮ Unnamed Item ⋮ Bayesian MIDAS penalized regressions: estimation, selection, and prediction ⋮ Prediction and estimation consistency of sparse multi-class penalized optimal scoring ⋮ Gelfand numbers related to structured sparsity and Besov space embeddings with small mixed smoothness ⋮ Simultaneous feature selection and clustering based on square root optimization ⋮ Group Regularized Estimation Under Structural Hierarchy ⋮ Adaptive estimation of the rank of the coefficient matrix in high-dimensional multivariate response regression models ⋮ Recovery of simultaneous low rank and two-way sparse coefficient matrices, a nonconvex approach ⋮ Structured sparsity through convex optimization ⋮ A selective review of group selection in high-dimensional models ⋮ High-dimensional regression with unknown variance ⋮ A unified framework for high-dimensional analysis of \(M\)-estimators with decomposable regularizers ⋮ Sparse estimation by exponential weighting ⋮ Randomized maximum-contrast selection: subagging for large-scale regression ⋮ Log-Contrast Regression with Functional Compositional Predictors: Linking Preterm Infant's Gut Microbiome Trajectories to Neurobehavioral Outcome ⋮ A Note on Coding and Standardization of Categorical Variables in (Sparse) Group Lasso Regression ⋮ Tuning-Free Heterogeneity Pursuit in Massive Networks ⋮ Robust regression via mutivariate regression depth ⋮ Joint variable and rank selection for parsimonious estimation of high-dimensional matrices ⋮ Tight conditions for consistency of variable selection in the context of high dimensionality ⋮ Accuracy guaranties for \(\ell_{1}\) recovery of block-sparse signals ⋮ The de-biased group Lasso estimation for varying coefficient models ⋮ AIC for the group Lasso in generalized linear models ⋮ Sharp oracle inequalities for low-complexity priors ⋮ Oracle inequalities for weighted group Lasso in high-dimensional misspecified Cox models ⋮ Analysis of generalized Bregman surrogate algorithms for nonsmooth nonconvex statistical learning ⋮ Multidimensional linear functional estimation in sparse Gaussian models and robust estimation of the mean ⋮ Localized Gaussian width of \(M\)-convex hulls with applications to Lasso and convex aggregation ⋮ High-dimensional generalized linear models incorporating graphical structure among predictors ⋮ Quantile regression feature selection and estimation with grouped variables using Huber approximation ⋮ Unnamed Item ⋮ Unnamed Item ⋮ On the robustness of minimum norm interpolators and regularized empirical risk minimizers ⋮ Structured estimation for the nonparametric Cox model ⋮ Optimal estimation and rank detection for sparse spiked covariance matrices ⋮ A no-free-lunch theorem for multitask learning ⋮ Sparse high-dimensional varying coefficient model: nonasymptotic minimax study ⋮ A group VISA algorithm for variable selection ⋮ ISLET: Fast and Optimal Low-Rank Tensor Regression via Importance Sketching

Uses Software

• hgam

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Exponential screening and optimal rates of sparse estimation
• Oracle inequalities in empirical risk minimization and sparse recovery problems. École d'Été de Probabilités de Saint-Flour XXXVIII-2008.
• Some theoretical results on the grouped variables Lasso
• Convex analysis and nonlinear optimization. Theory and examples.
• The sparsity and bias of the LASSO selection in high-dimensional linear regression
• The benefit of group sparsity
• Variable selection in nonparametric additive models
• Moment inequalities for sums of dependent random variables under projective conditions
• High-dimensional additive modeling
• The composite absolute penalties family for grouped and hierarchical variable selection
• Oracle inequalities for inverse problems
• Weak convergence and empirical processes. With applications to statistics
• On the asymptotic properties of the group lasso estimator for linear models
• Simultaneous analysis of Lasso and Dantzig selector
• High-dimensional generalized linear models and the lasso
• Sparsity oracle inequalities for the Lasso
• Sup-norm convergence rate and sign concentration property of Lasso and Dantzig estimators
• Support union recovery in high-dimensional multivariate regression
• Aggregation for Gaussian regression
• The Dantzig selector: statistical estimation when \(p\) is much larger than \(n\). (With discussions and rejoinder).
• Nemirovski's Inequalities Revisited
• Stable recovery of sparse overcomplete representations in the presence of noise
• The Group Lasso for Logistic Regression
• Sharp Thresholds for High-Dimensional and Noisy Sparsity Recovery Using $\ell _{1}$-Constrained Quadratic Programming (Lasso)
• Model Selection and Estimation in Regression with Grouped Variables
• An Efficient Method of Estimating Seemingly Unrelated Regressions and Tests for Aggregation Bias
• Introduction to nonparametric estimation

This page was built for publication: Oracle inequalities and optimal inference under group sparsity