Sharp variable selection of a sparse submatrix in a high-dimensional noisy matrix

DOI10.1051/PS/2014017MaRDI QIDQ2786472zbMATH OpenOpenAlexFDO

Authors Cristina Butucea, Yuri I. Ingster, Irina A. Suslina

Publication date 12 February 2016

Published in European Series in Applied and Industrial Mathematics (ESAIM): Probability and Statistics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1303.5647

zbMATH Keywords

variable selectionestimationminimax testinglarge matricesselection of sparse signalsharp selection bounds

Mathematics Subject Classification ID

Nonparametric estimation (62G05) Asymptotic properties of nonparametric inference (62G20) Signal theory (characterization, reconstruction, filtering, etc.) (94A12)

Abstract: We observe a $NimesM$ matrix of independent, identically distributed Gaussian random variables which are centered except for elements of some submatrix of size $nimesm$ where the mean is larger than some $a>0$. The submatrix is sparse in the sense that $n/N$ and $m/M$ tend to 0, whereas $n,,m,,N$ and $M$ tend to infinity. We consider the problem of selecting the random variables with significantly large mean values. We give sufficient conditions on $a$ as a function of $n,,m,,N$ and $M$ and construct a uniformly consistent procedure in order to do sharp variable selection. We also prove the minimax lower bounds under necessary conditions which are complementary to the previous conditions. The critical values $a^{\ast }$ separating the necessary and sufficient conditions are sharp (we show exact constants). We note a gap between the critical values $a^{\ast }$ for selection of variables and that of detecting that such a submatrix exists given by Butucea and Ingster (2012). When $a^{\ast }$ is in this gap, consistent detection is possible but no consistent selector of the corresponding variables can be found.

Recommendations

• Detection of a sparse submatrix of a high-dimensional noisy matrix
• High-dimensional variable selection with sparse random projections: measurement sparsity and statistical efficiency
• Subset selection in sparse matrices
• Sparse covariance thresholding for high-dimensional variable selection
• Variable selection in high-dimension with random designs and orthogonal matching pursuit
• Joint variable and rank selection for parsimonious estimation of high-dimensional matrices
• Coordinate-independent sparse sufficient dimension reduction and variable selection
• Dimension-wise sparse low-rank approximation of a matrix with application to variable selection in high-dimensional integrative analyzes of association
• Variable selection in multivariate linear models with high-dimensional covariance matrix estimation
• Variable selection in high-dimensional sparse multiresponse linear regression models

Cites work

• scientific article; zbMATH DE number 409717 (Why is no real title available?)
• scientific article; zbMATH DE number 720689 (Why is no real title available?)
• A simpler approach to matrix completion
• Adapting to unknown sparsity by controlling the false discovery rate
• Adaptive variable selection in nonparametric sparse regression
• Detection of a signal of known shape in a multichannel system
• Detection of a sparse submatrix of a high-dimensional noisy matrix
• Detection of an anomalous cluster in a network
• Estimation and confidence sets for sparse normal mixtures
• Estimation of high-dimensional low-rank matrices
• Exact matrix completion via convex optimization
• Finding large average submatrices in high dimensional data
• Higher criticism for detecting sparse heterogeneous mixtures.
• Introduction to nonparametric estimation
• Matrix completion from noisy entries
• Minimax risks for sparse regressions: ultra-high dimensional phenomenons
• Near-Optimal Detection of Geometric Objects by Fast Multiscale Methods
• Nonparametric goodness-of-fit testing under Gaussian models
• Nuclear-norm penalization and optimal rates for noisy low-rank matrix completion
• On the maximal size of large-average and ANOVA-fit submatrices in a Gaussian random matrix
• Recovering Low-Rank Matrices From Few Coefficients in Any Basis
• Rodeo: Sparse, greedy nonparametric regression
• Selection of variables and dimension reduction in high-dimensional non-parametric regression
• Sharp detection of smooth signals in a high-dimensional sparse matrix with indirect observations
• Simultaneous analysis of Lasso and Dantzig selector
• Some problems of hypothesis testing leading to infinitely divisible distributions
• Tight Oracle Inequalities for Low-Rank Matrix Recovery From a Minimal Number of Noisy Random Measurements
• Tight conditions for consistency of variable selection in the context of high dimensionality

Cited in

(15)

• Compressed spectral screening for large-scale differential correlation analysis with application in selecting glioblastoma gene modules
• Tensor clustering with planted structures: statistical optimality and computational limits
• Generalized Sparse Precision Matrix Selection for Fitting Multivariate Gaussian Random Fields to Large Data Sets
• On the maximal size of large-average and ANOVA-fit submatrices in a Gaussian random matrix
• Detecting positive correlations in a multivariate sample
• Detection of a sparse submatrix of a high-dimensional noisy matrix
• A goodness-of-fit test on the number of biclusters in a relational data matrix
• Computational barriers to estimation from low-degree polynomials
• Analysis of singular subspaces under random perturbations
• Variable selection with Hamming loss
• The overlap gap property in principal submatrix recovery
• Submatrix localization via message passing
• Computational lower bounds for graphon estimation via low-degree polynomials
• Computational and statistical boundaries for submatrix localization in a large noisy matrix
• Connection between the selection problem for a sparse submatrix of a large-size matrix and the Bayesian problem of hypotheses testing

Describes a project that uses

Uses Software

• LAS

This page was built for publication: Sharp variable selection of a sparse submatrix in a high-dimensional noisy matrix

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2786472)