The MM alternative to EM

DOI10.1214/08-STS264zbMATH Open1329.62106arXiv1104.2203MaRDI QIDQ906520FDOQ906520

Authors: Tong Tong Wu, Kenneth Lange 

Publication date: 22 January 2016

Published in: Statistical Science (Search for Journal in Brave)

Abstract: The EM algorithm is a special case of a more general algorithm called the MM algorithm. Specific MM algorithms often have nothing to do with missing data. The first M step of an MM algorithm creates a surrogate function that is optimized in the second M step. In minimization, MM stands for majorize--minimize; in maximization, it stands for minorize--maximize. This two-step process always drives the objective function in the right direction. Construction of MM algorithms relies on recognizing and manipulating inequalities rather than calculating conditional expectations. This survey walks the reader through the construction of several specific MM algorithms. The potential of the MM algorithm in solving high-dimensional optimization and estimation problems is its most attractive feature. Our applications to random graph models, discriminant analysis and image restoration showcase this ability.

Full work available at URL: https://arxiv.org/abs/1104.2203

Recommendations

• EM vs MM: a case study
• Comparison of the EM algorithm and alternatives
• MM optimization algorithms
• Foundations of the MHM Method
• Electromagnetic cascades: an alternative solution
• MM for penalized estimation
• MMSE Dimension

zbMATH Keywords

maximum likelihooditerative majorizationpenalizationinequalities

Mathematics Subject Classification ID

Point estimation (62F10) Research exposition (monographs, survey articles) pertaining to statistics (62-02) History of mathematics in the 20th century (01A60) History of statistics (62-03)

Cites Work

• The elements of statistical learning. Data mining, inference, and prediction
• Pathwise coordinate optimization
• Title not available (Why is that?)
• Title not available (Why is that?)
• Linear Statistical Inference and its Applications
• On the convergence properties of the EM algorithm
• Title not available (Why is that?)
• Title not available (Why is that?)
• Robust Statistics
• Setting up alternating least squares and iterative majorization algorithms for solving various matrix optimization problems
• MM algorithms for generalized Bradley-Terry models.
• Variable selection using MM algorithms
• Simple and Globally Convergent Methods for Accelerating the Convergence of Any EM Algorithm
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Monotonicity of quadratic-approximation algorithms
• Nonmetric individual differences multidimensional scaling: An alternating least squares method with optimal scaling features
• Sharp quadratic majorization in one dimension
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• A curious likelihood identity for the multivariate t-distribution
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• THE NATURE OF POWER CORRECTIONS IN LARGE-β₀ APPROXIMATION
• Title not available (Why is that?)
• Inequalities: theory of majorization and its applications
• Minimization of a class of matrix trace functions by means of refined majorization
• Modern multidimensional scaling: theory and applications
• Log-concave probability and its applications
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Generalized convexity and inequalities
• Nonnegative matrix factorization for spectral data analysis
• Correspondence analysis with least absolute residuals
• Parameter convergence for EM and MM algorithms
• Title not available (Why is that?)
• An adaptive barrier method for convex programming
• Fitting longitudinal reduced-rank regression models by alternating least squares
• Computing estimates in the proportional odds model

Cited In (32)

• The appeals of quadratic majorization-minimization
• MM Algorithms for Variance Components Models
• Structured variable selection via prior-induced hierarchical penalty functions
• Another look at distance-weighted discrimination
• Inexact half-quadratic optimization for linear inverse problems
• Sparse minimum discrepancy approach to sufficient dimension reduction with simultaneous variable selection in ultrahigh dimension
• MM optimization algorithms
• Recent developments in expectation-maximization methods for analyzing complex data
• Penalized empirical likelihood for the sparse Cox regression model
• A coordinate majorization descent algorithm for \(\ell_1\) penalized learning
• Surrogate maximization/minimization algorithms and extensions
• Change-point computation for large graphical models: a scalable algorithm for Gaussian graphical models with change-points
• Locally Sparse Function-on-Function Regression
• Minorize–maximize algorithm for the generalized odds rate model for clustered current status data
• Empirical likelihood for censored linear regression and variable selection
• Maximum likelihood estimation of triangular and polygonal distributions
• A Legacy of EM Algorithms
• Graphics processing units and high-dimensional optimization
• Recursive nonlinear-system identification using latent variables
• Nested coordinate descent algorithms for empirical likelihood
• Is EM really necessary here? Examples where it seems simpler not to use EM
• An assembly and decomposition approach for constructing separable minorizing functions in a class of MM algorithms
• A fast unified algorithm for solving group-lasso penalize learning problems
• On the Bumpy Road to the Dominant Mode
• High-dimensional model-assisted inference for treatment effects with multi-valued treatments
• EM vs MM: a case study
• Quantile regression feature selection and estimation with grouped variables using Huber approximation
• Backfitting algorithms for total-variation and empirical-norm penalized additive modelling with high-dimensional data
• Flexible Expectile Regression in Reproducing Kernel Hilbert Spaces
• Distance majorization and its applications
• Efficient estimation of the additive risks model for interval-censored data
• High-performance statistical computing in the computing environments of the 2020s

Uses Software

• UCI-ml
• ElemStatLearn

This page was built for publication: The MM alternative to EM

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q906520)