Generalized linear models with structured sparsity estimators
From MaRDI portal
Abstract: In this paper, we introduce structured sparsity estimators in Generalized Linear Models. Structured sparsity estimators in the least squares loss are introduced by Stucky and van de Geer (2018) recently for fixed design and normal errors. We extend their results to debiased structured sparsity estimators with Generalized Linear Model based loss. Structured sparsity estimation means penalized loss functions with a possible sparsity structure used in the chosen norm. These include weighted group lasso, lasso and norms generated from convex cones. The significant difficulty is that it is not clear how to prove two oracle inequalities. The first one is for the initial penalized Generalized Linear Model estimator. Since it is not clear how a particular feasible-weighted nodewise regression may fit in an oracle inequality for penalized Generalized Linear Model, we need a second oracle inequality to get oracle bounds for the approximate inverse for the sample estimate of second-order partial derivative of Generalized Linear Model. Our contributions are fivefold: 1. We generalize the existing oracle inequality results in penalized Generalized Linear Models by proving the underlying conditions rather than assuming them. One of the key issues is the proof of a sample one-point margin condition and its use in an oracle inequality. 2. Our results cover even non sub-Gaussian errors and regressors. 3. We provide a feasible weighted nodewise regression proof which generalizes the results in the literature from a simple l_1 norm usage to norms generated from convex cones. 4. We realize that norms used in feasible nodewise regression proofs should be weaker or equal to the norms in penalized Generalized Linear Model loss. 5. We can debias the first step estimator via getting an approximate inverse of the singular-sample second order partial derivative of Generalized Linear Model loss.
Cites work
- A constrained \(\ell _{1}\) minimization approach to sparse precision matrix estimation
- A general theory of hypothesis tests and confidence regions for sparse high dimensional models
- A unified framework for high-dimensional analysis of M-estimators with decomposable regularizers
- Asymptotic Confidence Regions for High-Dimensional Structured Sparsity
- Asymptotically honest confidence regions for high dimensional parameters by the desparsified conservative Lasso
- Causal inference by quantile regression kink designs
- Central limit theorems and bootstrap in high dimensions
- Estimation and testing under sparsity. École d'Été de Probabilités de Saint-Flour XLV -- 2015
- High dimensional inverse covariance matrix estimation via linear programming
- High-dimensional statistics. A non-asymptotic viewpoint
- Honest confidence regions for nonparametric regression
- Inference on treatment effects after selection among high-dimensional controls
- Linear hypothesis testing for high dimensional generalized linear models
- Model Selection and Estimation in Regression with Grouped Variables
- On asymptotically optimal confidence regions and tests for high-dimensional models
- Oracle inequalities and optimal inference under group sparsity
- Oracle inequalities, variable selection and uniform inference in high-dimensional correlated random effects panel data models
- Robust inference on average treatment effects with possibly more covariates than observations
- The Group Lasso for Logistic Regression
- The benefit of group sparsity in group inference with de-biased scaled group Lasso
- Uniform inference in high-dimensional dynamic panel data models with approximately sparse fixed effects
- Weakly decomposable regularization penalties and structured sparsity
Cited in
(6)- Sparse variational analysis of linear mixed models for large data sets
- Should Humans Lie to Machines? The Incentive Compatibility of Lasso and GLM Structured Sparsity Estimators
- A Compressive Sensing Based Analysis of Anomalies in Generalized Linear Models
- Sparse Estimation of Generalized Linear Models (GLM) via Approximated Information Criteria
- Deep learning based residuals in non-linear factor models: precision matrix estimation of returns with low signal-to-noise ratio
- Matrix regression heterogeneity analysis
This page was built for publication: Generalized linear models with structured sparsity estimators
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6054394)
