Factor-Adjusted Regularized Model Selection

DOI10.48550/ARXIV.1612.08490zbMATH Open1456.62114arXiv1612.08490OpenAlexW3005119431WikidataQ91740386 ScholiaQ91740386MaRDI QIDQ150847FDOQ150847

Authors: Jianqing Fan, Yuan Ke, Kaizheng Wang, Yuan Ke, Kaizheng Wang, Jianqing Fan 

Publication date: 27 December 2016

Published in: Journal of Econometrics (Search for Journal in Brave)

Abstract: This paper studies model selection consistency for high dimensional sparse regression when data exhibits both cross-sectional and serial dependency. Most commonly-used model selection methods fail to consistently recover the true model when the covariates are highly correlated. Motivated by econometric studies, we consider the case where covariate dependence can be reduced through factor model, and propose a consistent strategy named Factor-Adjusted Regularized Model Selection (FarmSelect). By separating the latent factors from idiosyncratic components, we transform the problem from model selection with highly correlated covariates to that with weakly correlated variables. Model selection consistency as well as optimal rates of convergence are obtained under mild conditions. Numerical studies demonstrate the nice finite sample performance in terms of both model selection and out-of-sample prediction. Moreover, our method is flexible in a sense that it pays no price for weakly correlated and uncorrelated cases. Our method is applicable to a wide range of high dimensional sparse regression problems. An R-package FarmSelect is also provided for implementation.

Full work available at URL: https://arxiv.org/abs/1612.08490

Recommendations

• Model selection in factor-augmented regressions with estimated factors
• Model selection for generalized linear models with factor-augmented predictors
• Factor models and variable selection in high-dimensional regression analysis
• Variable selection by regularization methods for generalized mixed models
• Rank regularized estimation of approximate factor models
• Variable Selection in the Presence of Factors: A Model Selection Perspective
• Bayesian factor-adjusted sparse regression
• Regularization and variable selection in Heckman selection model
• Regularization and model selection with categorial effect modifiers
• Model selection in regression under structural constraints

zbMATH Keywords

factor modeltime seriesmodel selection consistencycorrelated covariatesregularized \(M\)-estimator

Mathematics Subject Classification ID

Computational methods for problems pertaining to statistics (62-08) Factor analysis and principal components; correspondence analysis (62H25) Linear regression; mixed models (62J05) Time series, auto-correlation, regression, etc. in statistics (GARCH) (62M10) Applications of statistics to economics (62P20) Ridge regression; shrinkage estimators (Lasso) (62J07)

Cites Work

• Sure independence screening in generalized linear models with NP-dimensionality
• Nearly unbiased variable selection under minimax concave penalty
• Estimating the dimension of a model
• Least angle regression. (With discussion)
• Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
• Forecasting Using Principal Components From a Large Number of Predictors
• Title not available (Why is that?)
• Title not available (Why is that?)
• Inferential Theory for Factor Models of Large Dimensions
• Determining the Number of Factors in Approximate Factor Models
• The Generalized Dynamic Factor Model
• Factor models and variable selection in high-dimensional regression analysis
• One-step sparse estimates in nonconcave penalized likelihood models
• Distributed optimization and statistical learning via the alternating direction method of multipliers
• Simultaneous analysis of Lasso and Dantzig selector
• The Dantzig selector: statistical estimation when \(p\) is much larger than \(n\). (With discussions and rejoinder).
• High-dimensional graphs and variable selection with the Lasso
• Eigenvalue ratio test for the number of factors
• Title not available (Why is that?)
• Factor modeling for high-dimensional time series: inference for the number of factors
• Sure Independence Screening for Ultrahigh Dimensional Feature Space
• Sharp Thresholds for High-Dimensional and Noisy Sparsity Recovery Using $\ell _{1}$-Constrained Quadratic Programming (Lasso)
• Determining the Number of Factors in the General Dynamic Factor Model
• Regularization and Variable Selection Via the Elastic Net
• Large Covariance Estimation by Thresholding Principal Orthogonal Complements
• Title not available (Why is that?)
• On consistency and sparsity for principal components analysis in high dimensions
• Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ ¹ minimization
• High dimensional stochastic regression with latent factors, endogeneity and nonlinearity
• Nonconcave penalized likelihood with a diverging number of parameters.
• The statistics and mathematics of high dimension low sample size asymptotics
• Title not available (Why is that?)
• Contour projected dimension reduction
• ARMA model identification
• Asymptotics of empirical eigenstructure for high dimensional spiked covariance
• On model selection consistency of regularized M-estimators
• Factor profiled sure independence screening
• Use of canonical analysis in time series model identification
• Forecasting the U.S. Unemployment Rate
• High dimensional ordinary least squares projection for screening variables
• A Bernstein type inequality and moderate deviations for weakly dependent sequences

Cited In (29)

• Model selection in factor-augmented regressions with estimated factors
• Are Latent Factor Regression and Sparse Regression Adequate?
• Ridge Regression Under Dense Factor Augmented Models
• A generalized knockoff procedure for FDR control in structural change detection
• Canonical thresholding for nonsparse high-dimensional linear regression
• Noisy matrix completion: understanding statistical guarantees for convex relaxation via nonconvex optimization
• Do We Exploit all Information for Counterfactual Analysis? Benefits of Factor Models and Idiosyncratic Correction
• Bayesian factor-adjusted sparse regression
• Bayesian MIDAS penalized regressions: estimation, selection, and prediction
• Learning Latent Factors From Diversified Projections and Its Applications to Over-Estimated and Weak Factors
• FNETS: Factor-Adjusted Network Estimation and Forecasting for High-Dimensional Time Series
• Comment on “A Tuning-Free Robust and Efficient Approach to High-Dimensional Regression”
• Factor Augmented Sparse Throughput Deep ReLU Neural Networks for High Dimensional Regression
• Nonparametric estimation of the random coefficients model: an elastic net approach
• Model selection for generalized linear models with factor-augmented predictors
• Bridging factor and sparse models
• Factor Augmented Inverse Regression and its Application to Microbiome Data Analysis
• High-Dimensional Time Series Segmentation via Factor-Adjusted Vector Autoregressive Modeling
• FarmSelect
• Efficient change point detection and estimation in high-dimensional correlation matrices
• F-test and z-test for high-dimensional regression models with a factor structure
• Semi-Standard Partial Covariance Variable Selection When Irrepresentable Conditions Fail
• Model-Free Feature Screening and FDR Control With Knockoff Features
• A Decorrelating and Debiasing Approach to Simultaneous Inference for High-Dimensional Confounded Models
• Factor models and variable selection in high-dimensional regression analysis
• Variable selection in high dimensional linear regressions with parameter instability
• Factor-adjusted tests for generalized linear models with multimodal data: an application to breast cancer data
• Mean tests for high-dimensional time series
• Title not available (Why is that?)

Uses Software

• R
• glmnet
• FarmSelect

This page was built for publication: Factor-Adjusted Regularized Model Selection

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q150847)