High-dimensional estimation with geometric constraints
From MaRDI portal
Publication:4603716
Abstract: Consider measuring an n-dimensional vector x through the inner product with several measurement vectors, a_1, a_2, ..., a_m. It is common in both signal processing and statistics to assume the linear response model y_i = <a_i, x> + e_i, where e_i is a noise term. However, in practice the precise relationship between the signal x and the observations y_i may not follow the linear model, and in some cases it may not even be known. To address this challenge, in this paper we propose a general model where it is only assumed that each observation y_i may depend on a_i only through <a_i, x>. We do not assume that the dependence is known. This is a form of the semiparametric single index model, and it includes the linear model as well as many forms of the generalized linear model as special cases. We further assume that the signal x has some structure, and we formulate this as a general assumption that x belongs to some known (but arbitrary) feasible set K. We carefully detail the benefit of using the signal structure to improve estimation. The theory is based on the mean width of K, a geometric parameter which can be used to understand its effective dimension in estimation problems. We determine a simple, efficient two-step procedure for estimating the signal based on this model -- a linear estimation followed by metric projection onto K. We give general conditions under which the estimator is minimax optimal up to a constant. This leads to the intriguing conclusion that in the high noise regime, an unknown non-linearity in the observations does not significantly reduce one's ability to determine the signal, even when the non-linearity may be non-invertible. Our results may be specialized to understand the effect of non-linearities in compressed sensing.
Recommendations
Cites work
- scientific article; zbMATH DE number 1001426 (Why is no real title available?)
- scientific article; zbMATH DE number 3931747 (Why is no real title available?)
- scientific article; zbMATH DE number 3944537 (Why is no real title available?)
- scientific article; zbMATH DE number 49190 (Why is no real title available?)
- scientific article; zbMATH DE number 2149435 (Why is no real title available?)
- 1-bit matrix completion
- 10.1162/153244303321897690
- A Probabilistic and RIPless Theory of Compressed Sensing
- A new perspective on least squares under convex constraint
- A unified framework for high-dimensional analysis of \(M\)-estimators with decomposable regularizers
- Asymptotic formulas for the diameter of sections of symmetric convex bodies
- Capturing ridge functions in high dimensions from point queries
- Consistent Estimation of Scaled Coefficients
- Direct estimation of the index coefficient in a single-index model
- Exact matrix completion via convex optimization
- Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization
- How small can the intersection of a few rotations of a symmetric convex body be?
- How well can we estimate a sparse vector?
- Living on the edge: phase transitions in convex programs with random data
- Matrix Completion From a Few Entries
- Matrix estimation by universal singular value thresholding
- Mean width and diameter of proportional sections of a symmetric convex body
- Minimax Rates of Estimation for High-Dimensional Linear Regression Over $\ell_q$-Balls
- Nuclear-norm penalization and optimal rates for noisy low-rank matrix completion
- One-bit compressed sensing by linear programming
- One-bit compressed sensing with non-Gaussian measurements
- Reconstruction and subgaussian operators in asymptotic geometric analysis
- Regression analysis under link violation
- Robust 1-Bit Compressive Sensing via Binary Stable Embeddings of Sparse Vectors
- Robust 1-bit Compressed Sensing and Sparse Logistic Regression: A Convex Programming Approach
- Self-concordant analysis for logistic regression
- Semiparametric Estimation of Index Coefficients
- Sparse single-index model
- Statistics for high-dimensional data. Methods, theory and applications.
- Subspaces of Small Codimension of Finite-Dimensional Banach Spaces
- The Expected Norm of Random Matrices
- The Generic Chaining
- Tight Oracle Inequalities for Low-Rank Matrix Recovery From a Minimal Number of Noisy Random Measurements
- Tight conditions for consistency of variable selection in the context of high dimensionality
Cited in
(28)- Geometric Representation of High Dimension, Low Sample Size Data
- Rigorous high-dimensional shadowing using containment: the general case
- Localized Gaussian width of \(M\)-convex hulls with applications to Lasso and convex aggregation
- On polyhedral estimation of signals via indirect observations
- Least squares estimation in the monotone single index model
- Estimating covariance and precision matrices along subspaces
- Solving equations of random convex functions via anchored regression
- Just least squares: binary compressive sampling with low generative intrinsic dimension
- Optimal estimation of slope vector in high-dimensional linear transformation models
- Geometric inference for general high-dimensional linear inverse problems
- Concentration of measure bounds for matrix-variate data with missing values
- A unified approach to uniform signal recovery from nonlinear observations
- Understanding Implicit Regularization in Over-Parameterized Single Index Model
- Fast and reliable parameter estimation from nonlinear observations
- High-dimensional varying index coefficient models via Stein's identity
- Uniform recovery guarantees for quantized corrupted sensing using structured or generative priors
- Convergence guarantee for the sparse monotone single index model
- Misspecified nonconvex statistical optimization for sparse phase retrieval
- Preserving injectivity under subgaussian mappings and its application to compressed sensing
- Generic error bounds for the generalized Lasso with sub-exponential data
- Robust Decoding from 1-Bit Compressive Sampling with Ordinary and Regularized Least Squares
- The landscape of empirical risk for nonconvex losses
- Non-Gaussian observations in nonlinear compressed sensing via Stein discrepancies
- Robust parameter estimation of regression models under weakened moment assumptions
- Sharp oracle inequalities for least squares estimators in shape restricted regression
- Optimal combination of linear and spectral estimators for generalized linear models
- Estimation in high dimensions: a geometric perspective
- A simple tool for bounding the deviation of random matrices on geometric sets
This page was built for publication: High-dimensional estimation with geometric constraints
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4603716)
