Variable selection using MM algorithms

DOI10.1214/009053605000000200zbMATH Open1078.62028arXivmath/0508278OpenAlexW3101767848WikidataQ43138735 ScholiaQ43138735MaRDI QIDQ2583414FDOQ2583414

Authors: David R. Hunter, Runze Li 

Publication date: 16 January 2006

Published in: The Annals of Statistics (Search for Journal in Brave)

Abstract: Variable selection is fundamental to high-dimensional statistical modeling. Many variable selection techniques may be implemented by maximum penalized likelihood using various penalty functions. Optimizing the penalized likelihood function is often challenging because it may be nondifferentiable and/or nonconcave. This article proposes a new class of algorithms for finding a maximizer of the penalized likelihood for a broad class of penalty functions. These algorithms operate by perturbing the penalty function slightly to render it differentiable, then optimizing this differentiable function using a minorize-maximize (MM) algorithm. MM algorithms are useful extensions of the well-known class of EM algorithms, a fact that allows us to analyze the local and global convergence of the proposed algorithm using some of the techniques employed for EM algorithms. In particular, we prove that when our MM algorithms converge, they must converge to a desirable point; we also discuss conditions under which this convergence may be guaranteed. We exploit the Newton-Raphson-like aspect of these algorithms to propose a sandwich estimator for the standard errors of the estimators. Our method performs well in numerical tests.

Full work available at URL: https://arxiv.org/abs/math/0508278

Recommendations

• Variable selection in multivariate multiple regression
• MM algorithms for distance covariance based sufficient dimension reduction and sufficient variable selection
• The variable selection methods and algorithms in the multiple linear model
• The Mnet method for variable selection
• MM Algorithms for Variance Components Models
• Predictive Variable Selection for the Multivariate Linear Model
• Variable selection for multivariate generalized linear models
• MM ALGORITHMS FOR VARIANCE COMPONENT ESTIMATION AND SELECTION IN LOGISTIC LINEAR MIXED MODEL

zbMATH Keywords

SCADBICpenalized likelihoodAICLASSOoracle estimatorEM algorithmsMM algorithmsenvironment data

Mathematics Subject Classification ID

Point estimation (62F10) Parametric inference (62F99) Generalized linear models (logistic models) (62J12) Probabilistic models, generic numerical methods in probability and statistics (65C20)

Cites Work

• Title not available (Why is that?)
• Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
• Title not available (Why is that?)
• On the convergence properties of the EM algorithm
• Smoothing noisy data with spline functions: Estimating the correct degree of smoothing by the method of generalized cross-validation
• Partial likelihood
• Variable selection for Cox's proportional hazards model and frailty model
• Title not available (Why is that?)
• A Note on the Efficiency of Sandwich Covariance Matrix Estimation
• New Estimation and Model Selection Procedures for Semiparametric Modeling in Longitudinal Data Analysis
• Title not available (Why is that?)
• Nonconcave penalized likelihood with a diverging number of parameters.
• Regularization of Wavelet Approximations
• A Statistical View of Some Chemometrics Regression Tools
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Variable selection for multivariate failure time data
• Title not available (Why is that?)
• On the rate of convergence of the ECM algorithm

Cited In (only showing first 100 items - show all)

• Variable selection in linear measurement error models via penalized score functions
• Penalized factor mixture analysis for variable selection in clustered data
• Partial linear modelling with multi-functional covariates
• A globally convergent algorithm for Lasso-penalized mixture of linear regression models
• Variable selection of the quantile varying coefficient regression models
• Variable selection in robust semiparametric modeling for longitudinal data
• A focused information criterion for graphical models in fMRI connectivity with high-dimensional data
• Penalized MM regression estimation with \(L_\gamma\) penalty: a robust version of bridge regression
• Majorization-minimization algorithms for nonsmoothly penalized objective functions
• Sparse permutation invariant covariance estimation
• Variable selection in robust regression models for longitudinal data
• Simultaneous variable selection and de-coarsening in multi-path change-point models
• Robust structure identification and variable selection in partial linear varying coefficient models
• Robust truss topology optimization via semidefinite programming with complementarity constraints: a difference-of-convex programming approach
• The MM alternative to EM
• On optimality of Bayesian testimation in the normal means problem
• Network exploration via the adaptive LASSO and SCAD penalties
• Hidden Markov models with applications in cell adhesion experiments
• Global solutions to folded concave penalized nonconvex learning
• Another look at distance-weighted discrimination
• Penalized generalized estimating equations approach to longitudinal data with multinomial responses
• Endogeneity in high dimensions
• Sparse classification: a scalable discrete optimization perspective
• Statistical Software VASMM for Variable Selection in Multivariate Methods
• Relaxed sparse eigenvalue conditions for sparse estimation via non-convex regularized regression
• Model selection and structure specification in ultra-high dimensional generalised semi-varying coefficient models
• Penalized least-squares estimation for regression coefficients in high-dimensional partially linear models
• Estimation and variable selection for generalized additive partial linear models
• New efficient estimation and variable selection methods for semiparametric varying-coefficient partially linear models
• Selection and combination of biomarkers using ROC method for disease classification and prediction
• Asymptotic properties of lasso in high-dimensional partially linear models
• Variable selection for survival data with a class of adaptive elastic net techniques
• Variable selection in semiparametric regression modeling
• Asymptotic properties of bridge estimators in sparse high-dimensional regression models
• Robust penalized logistic regression with truncated loss functions
• Variable selection for panel count data via non-concave penalized estimating function
• Variable selection in linear mixed models using an extended class of penalties
• Penalized generalized estimating equations for high-dimensional longitudinal data analysis
• Thresholding-based iterative selection procedures for model selection and shrinkage
• Fixed and Random Effects Selection in Mixed Effects Models
• Pairwise Variable Selection for High-Dimensional Model-Based Clustering
• MM for penalized estimation
• Simultaneous variable selection and class fusion for high-dimensional linear discriminant analysis
• Adaptive bridge estimation for high-dimensional regression models
• Nearly unbiased variable selection under minimax concave penalty
• Sparse logistic principal components analysis for binary data
• Penalized least squares for single index models
• A unified approach to model selection and sparse recovery using regularized least squares
• Robust variable selection for the varying coefficient model based on composite \(L_1\)-\(L_2\) regression
• Sure Independence Screening for Ultrahigh Dimensional Feature Space
• Calibrating nonconvex penalized regression in ultra-high dimension
• Strong oracle optimality of folded concave penalized estimation
• Majorization minimization by coordinate descent for concave penalized generalized linear models
• Variable Selection Using a Smooth Information Criterion for Distributional Regression Models
• Variable selection in measurement error models
• New estimation and feature selection methods in mixture-of-experts models
• Oracle efficient variable selection in random and fixed effects panel data models
• High-dimensional generalized semiparametric model for longitudinal data
• On the estimation of variance parameters in non-standard generalised linear mixed models: application to penalised smoothing
• One-step sparse estimates in nonconcave penalized likelihood models
• A sure independence screening procedure for ultra-high dimensional partially linear additive models
• Penalized principal logistic regression for sparse sufficient dimension reduction
• Estimation in high-dimensional linear models with deterministic design matrices
• Penalized profiled semiparametric estimating functions
• A majorization-minimization approach to the sparse generalized eigenvalue problem
• Sharp quadratic majorization in one dimension
• Nonconcave penalized composite conditional likelihood estimation of sparse Ising models
• SCAD-penalized regression in high-dimensional partially linear models
• A Penalized Likelihood Method for Classification With Matrix-Valued Predictors
• Optimal computational and statistical rates of convergence for sparse nonconvex learning problems
• Latent variable selection in structural equation models
• Penalized joint generalized estimating equations for longitudinal binary data
• Weighted composite quantile regression estimation and variable selection for varying coefficient models with heteroscedasticity
• Regularization in statistics
• A general theory of concave regularization for high-dimensional sparse estimation problems
• Poststratification fusion learning in longitudinal data analysis
• Efficient estimation of the additive risks model for interval-censored data
• Sparse and smooth functional data clustering
• Bridge regression: adaptivity and group selection
• Penalized estimation equation for an extended single-index model
• High-performance statistical computing in the computing environments of the 2020s
• Heteroscedasticity identification and variable selection via multiple quantile regression
• Simultaneous variable selection and structural identification for time‐varying coefficient models
• Sparse identification of nonlinear dynamical systems via non-convex penalty least squares
• An ordinary differential equation-based solution path algorithm
• An iterative approach to distance correlation-based sure independence screening
• Variable selection for varying coefficient models via kernel based regularized rank regression
• Data mining for longitudinal data under multicollinearity and time dependence using penalized generalized estimating equations
• Smoothed rank correlation of the linear transformation regression model
• Doubly robust weighted composite quantile regression based on SCAD‐L₂
• MM algorithms for distance covariance based sufficient dimension reduction and sufficient variable selection
• Sparse recovery via nonconvex regularized \(M\)-estimators over \(\ell_q\)-balls
• A two-stage bridge estimator for regression models with endogeneity based on control function method
• Coordinate majorization descent algorithm for nonconvex penalized regression
• Variable selection in elliptical linear mixed model
• Modified SCAD penalty for constrained variable selection problems
• Robust and smoothing variable selection for quantile regression models with longitudinal data
• A split-and-conquer variable selection approach for high-dimensional general semiparametric models with massive data
• The use of random-effect models for high-dimensional variable selection problems
• Adaptive LASSO for selecting Fourier coefficients in a functional smooth time-varying cointegrating regression: an application to the Feldstein-Horioka puzzle

This page was built for publication: Variable selection using MM algorithms

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2583414)