MM for penalized estimation
From MaRDI portal
Publication:82924
classificationMM algorithmnonconvexquadratic majorizationregressionrobust estimationvariable selection
Computational methods for problems pertaining to statistics (62-08) Robustness and adaptive procedures (parametric inference) (62F35) Classification and discrimination; cluster analysis (statistical aspects) (62H30) Ridge regression; shrinkage estimators (Lasso) (62J07) General topics in computing methodologies (68U01)
Abstract: Penalized estimation can conduct variable selection and parameter estimation simultaneously. The general framework is to minimize a loss function subject to a penalty designed to generate sparse variable selection. The majorization-minimization (MM) algorithm is a computational scheme for stability and simplicity, and the MM algorithm has been widely applied in penalized estimation. Much of the previous work have focused on convex loss functions such as generalized linear models. When data are contaminated with outliers, robust loss functions can generate more reliable estimates. Recent literature has witnessed a growing impact of nonconvex loss-based methods, which can generate robust estimation for data contaminated with outliers. This article investigates MM algorithm for penalized estimation, provide innovative optimality conditions and establish convergence theory with both convex and nonconvex loss functions. With respect to applications, we focus on several nonconvex loss functions, which were formerly studied in machine learning for regression and classification problems. Performance of the proposed algorithms are evaluated on simulated and real data including healthcare costs and cancer clinical status. Efficient implementations of the algorithms are available in the R package mpath in CRAN.
Recommendations
- Penalized MM regression estimation with L_ penalty: a robust version of bridge regression
- Penalized maximum likelihood estimation and effective dimension
- M-estimation using penalties or sieves
- Penalized maximum likelihood estimation of a stochastic multivariate regression model
- Empirical likelihood-MM (EL-MM) estimation for the parameters of a linear regression model
- Estimates of MM type for the multivariate linear model
- Penalized Maximum Likelihood Estimator for Normal Mixtures
- The choice of the initial estimate for computing MM-estimates
- Nonconcave penalized M-estimation for the least absolute relative errors model
Cites work
- A unified convergence analysis of block successive minimization methods for nonsmooth optimization
- Auxiliary-function minimization algorithms
- Boosting in the presence of outliers: adaptive classification with nonconvex loss functions
- Coordinate descent algorithms for nonconvex penalized regression, with applications to biological feature selection
- (SOFTWARE) glmnet
- scientific article; zbMATH DE number 845714 (Why is no real title available?)
- Least angle regression. (With discussion)
- Majorization-minimization algorithms for nonsmoothly penalized objective functions
- MM optimization algorithms
- Nearly unbiased variable selection under minimax concave penalty
- One-step sparse estimates in nonconcave penalized likelihood models
- Piecewise linear regularized solution paths
- Quadratic Majorization for Nonconvex Loss with Applications to the Boosting Algorithm
- Robust boosting with truncated loss functions
- Robust Truncated Hinge Loss Support Vector Machines
- Sparse least trimmed squares regression for analyzing high-dimensional large data sets
- Variable selection using MM algorithms
- Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
Cited in
(9)- A forward and backward stagewise algorithm for nonconvex loss functions with adaptive Lasso
- Majorization-minimization algorithms for nonsmoothly penalized objective functions
- mpath
- Penalized MM regression estimation with L_ penalty: a robust version of bridge regression
- Composite difference-MAX programs for modern statistical estimation problems
- Unified robust estimation
- Active-set strategy based on a general modified Newton-Raphson algorithm for variable selection in highly ill-posed inverse problems
- Majorization minimization by coordinate descent for concave penalized generalized linear models
- The MM alternative to EM
This page was built for publication: MM for penalized estimation
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q82924)
