Rare feature selection in high dimensions

DOI10.1080/01621459.2020.1796677zbMATH Open1464.62334arXiv1803.06675OpenAlexW3045744807MaRDI QIDQ149281FDOQ149281

Jacob Bien, Jacob Bien, Xiaohan Yan, Xiaohan Yan

Publication date: 18 March 2018

Published in: Journal of the American Statistical Association (Search for Journal in Brave)

Abstract: It is common in modern prediction problems for many predictor variables to be counts of rarely occurring events. This leads to design matrices in which many columns are highly sparse. The challenge posed by such "rare features" has received little attention despite its prevalence in diverse areas, ranging from natural language processing (e.g., rare words) to biology (e.g., rare species). We show, both theoretically and empirically, that not explicitly accounting for the rareness of features can greatly reduce the effectiveness of an analysis. We next propose a framework for aggregating rare features into denser features in a flexible manner that creates better predictors of the response. Our strategy leverages side information in the form of a tree that encodes feature similarity. We apply our method to data from TripAdvisor, in which we predict the numerical rating of a hotel based on the text of the associated review. Our method achieves high accuracy by making effective use of rare words; by contrast, the lasso is unable to identify highly predictive words if they are too rare. A companion R package, called rare, implements our new estimator, using the alternating direction method of multipliers.

Full work available at URL: https://arxiv.org/abs/1803.06675

Recommendations

• Feature selection when there are many influential features
• A framework for feature selection in clustering
• 10.1162/153244303322753616
• Targeted random projection for prediction from high-dimensional features
• Prediction weighted maximum frequency selection

zbMATH Keywords

ADMMfeature aggregationsparsity in datatext data

Mathematics Subject Classification ID

Classification and discrimination; cluster analysis (statistical aspects) (62H30) Ridge regression; shrinkage estimators (Lasso) (62J07) Computational methods for sparse matrices (65F50)

Cites Work

• Title not available (Why is that?)
• Title not available (Why is that?)
• Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers
• Regression analysis for microbiome compositional data
• A Logistic Normal Multinomial Regression Model for Microbiome Compositional Data Analysis
• Variable selection and regression analysis for graph-structured covariates with an application to genomics
• The solution path of the generalized lasso
• Tree-guided group lasso for multi-response regression with structured sparsity, with an application to eQTL mapping
• A Dirichlet-Tree Multinomial Regression Model for Associating Dietary Nutrients with Gut Microorganisms
• Variable selection in regression with compositional covariates
• Sparse regression with exact clustering
• Hypothesis testing for high-dimensional sparse binary regression
• Kernel-penalized regression for analysis of microbiome data
• Structured subcomposition selection in regression and its application to microbiome data analysis
• 10.1162/153244303322753670
• Homogeneity Pursuit

Cited In (16)

• Multiresolution categorical regression for interpretable cell-type annotation
• It's All Relative: Regression Analysis with Compositional Predictors
• LOL selection in high dimension
• The geometry of monotone operator splitting methods
• Sparse principal component regression via singular value decomposition approach
• Identifying Brain Hierarchical Structures Associated with Alzheimer's Disease Using a Regularized Regression Method with Tree Predictors
• Multivariate Monotone Inclusions in Saddle Form
• Sentiment analysis with covariate-assisted word embeddings
• Tree-Guided Rare Feature Selection and Logic Aggregation with Electronic Health Records Data
• Word embeddings as statistical estimators
• rare
• Feature selection for high-dimensional data
• Hierarchical Regularizers for Mixed-Frequency Vector Autoregressions
• Feature Screening for Interval-Valued Response with Application to Study Association between Posted Salary and Required Skills
• Projective splitting with forward steps
• Hedonic pricing modelling with unstructured predictors: an application to Italian fashion industry

Uses Software

• R
• WordNet
• tm
• glmnet
• Phyloseq
• mothur
• wordnet
• GloVe
• pplacer
• genlasso

This page was built for publication: Rare feature selection in high dimensions

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q149281)