Delete or merge regressors for linear model selection

DOI10.1214/15-EJS1050MaRDI QIDQ78545zbMATH OpenOpenAlexFDO

Authors Aleksandra Maj-Kańska, Piotr Pokarowski, Agnieszka Prochenka, Aleksandra Maj-Kańska, Piotr Pokarowski, Agnieszka Prochenka

Publication date 1 January 2015

Published in Electronic Journal of Statistics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1505.04008, https://projecteuclid.org/euclid.ejs/1440507392

zbMATH Keywords

variable selection\(t\)-statisticANOVABICconsistencymerging levels

Mathematics Subject Classification ID

Statistical ranking and selection procedures (62F07) Ridge regression; shrinkage estimators (Lasso) (62J07)

Abstract: We consider a problem of linear model selection in the presence of both continuous and categorical predictors. Feasible models consist of subsets of numerical variables and partitions of levels of factors. A new algorithm called delete or merge regressors (DMR) is presented which is a stepwise backward procedure involving ranking the predictors according to squared t-statistics and choosing the final model minimizing BIC. In the article we prove consistency of DMR when the number of predictors tends to infinity with the sample size and describe a simulation study using a pertaining R package. The results indicate significant advantage in time complexity and selection accuracy of our algorithm over Lasso-based methods described in the literature. Moreover, a version of DMR for generalized linear models is proposed.

Recommendations

• Regularization and model selection with categorial effect modifiers
• A stepwise regression method and consistent model selection for highdimensional sparse linear models
• Linear model selection -- towards a framework using a mixed integer linear programming approach
• Least angle regression. (With discussion)
• Model Selection and Estimation in Regression with Grouped Variables

Cites work

• scientific article; zbMATH DE number 845714 (Why is no real title available?)
• A Cluster Analysis Method for Grouping Means in the Analysis of Variance
• Analysis of multi-stage convex relaxation for sparse regularization
• Combined \(\ell_1\) and greedy \(\ell_0\) penalized least squares for linear model selection
• Consistent Variable Selection in Linear Models
• Extended Bayesian information criteria for model selection with large model spaces
• Least angle regression. (With discussion)
• One-step sparse estimates in nonconcave penalized likelihood models
• Partitioning datasets based on equalities among parameters
• Simultaneous Factor Selection and Collapsing Levels in ANOVA
• Sparse modeling of categorial explanatory variables
• Sparsity and Smoothness Via the Fused Lasso

Cited in

(4)

• Pattern recovery and signal denoising by SLOPE when the design matrix is orthogonal
• Nonconvex fusion penalties for high-dimensional hierarchical categorical variables
• DMR
• DMRnet

Describes a project that uses

Uses Software

• CasANOVA
• gvcm.cat
• MASS (R)
• lattice
• R

This page was built for publication: Delete or merge regressors for linear model selection

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q78545)