Delete or merge regressors for linear model selection

DOI10.1214/15-EJS1050zbMATH Open1323.62025arXiv1505.04008OpenAlexW3101307403MaRDI QIDQ78545FDOQ78545

Authors: Aleksandra Maj-Kańska, Piotr Pokarowski, Agnieszka Prochenka, Aleksandra Maj-Kańska, Piotr Pokarowski, Agnieszka Prochenka 

Publication date: 1 January 2015

Published in: Electronic Journal of Statistics (Search for Journal in Brave)

Abstract: We consider a problem of linear model selection in the presence of both continuous and categorical predictors. Feasible models consist of subsets of numerical variables and partitions of levels of factors. A new algorithm called delete or merge regressors (DMR) is presented which is a stepwise backward procedure involving ranking the predictors according to squared t-statistics and choosing the final model minimizing BIC. In the article we prove consistency of DMR when the number of predictors tends to infinity with the sample size and describe a simulation study using a pertaining R package. The results indicate significant advantage in time complexity and selection accuracy of our algorithm over Lasso-based methods described in the literature. Moreover, a version of DMR for generalized linear models is proposed.

Full work available at URL: https://arxiv.org/abs/1505.04008

Recommendations

• Regularization and model selection with categorial effect modifiers
• A stepwise regression method and consistent model selection for highdimensional sparse linear models
• Linear model selection -- towards a framework using a mixed integer linear programming approach
• Least angle regression. (With discussion)
• Model Selection and Estimation in Regression with Grouped Variables

zbMATH Keywords

variable selection\(t\)-statisticANOVABICconsistencymerging levels

Mathematics Subject Classification ID

Statistical ranking and selection procedures (62F07) Ridge regression; shrinkage estimators (Lasso) (62J07)

Cites Work

• Simultaneous Factor Selection and Collapsing Levels in ANOVA
• Least angle regression. (With discussion)
• Extended Bayesian information criteria for model selection with large model spaces
• Title not available (Why is that?)
• One-step sparse estimates in nonconcave penalized likelihood models
• Analysis of multi-stage convex relaxation for sparse regularization
• Sparsity and Smoothness Via the Fused Lasso
• A Cluster Analysis Method for Grouping Means in the Analysis of Variance
• Consistent Variable Selection in Linear Models
• Sparse modeling of categorial explanatory variables
• Combined \(\ell_1\) and greedy \(\ell_0\) penalized least squares for linear model selection
• Partitioning datasets based on equalities among parameters

Cited In (4)

• Pattern recovery and signal denoising by SLOPE when the design matrix is orthogonal
• Nonconvex fusion penalties for high-dimensional hierarchical categorical variables
• DMR
• DMRnet

Uses Software

• R
• lattice
• MASS (R)
• gvcm.cat
• CasANOVA

This page was built for publication: Delete or merge regressors for linear model selection

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q78545)