Distance majorization and its applications
From MaRDI portal
Abstract: The problem of minimizing a continuously differentiable convex function over an intersection of closed convex sets is ubiquitous in applied mathematics. It is particularly interesting when it is easy to project onto each separate set, but nontrivial to project onto their intersection. Algorithms based on Newton's method such as the interior point method are viable for small to medium-scale problems. However, modern applications in statistics, engineering, and machine learning are posing problems with potentially tens of thousands of parameters or more. We revisit this convex programming problem and propose an algorithm that scales well with dimensionality. Our proposal is an instance of a sequential unconstrained minimization technique and revolves around three ideas: the majorization-minimization (MM) principle, the classical penalty method for constrained optimization, and quasi-Newton acceleration of fixed-point algorithms. The performance of our distance majorization algorithms is illustrated in several applications.
Recommendations
- Proximal distance algorithms: theory and practice
- The proximal distance algorithm
- Sharp quadratic majorization in one dimension
- Majorization-minimization algorithms for nonsmoothly penalized objective functions
- Incremental majorization-minimization optimization with application to large-scale machine learning
Cites work
- scientific article; zbMATH DE number 1667417 (Why is no real title available?)
- scientific article; zbMATH DE number 46305 (Why is no real title available?)
- scientific article; zbMATH DE number 193111 (Why is no real title available?)
- scientific article; zbMATH DE number 2107836 (Why is no real title available?)
- scientific article; zbMATH DE number 2121575 (Why is no real title available?)
- scientific article; zbMATH DE number 1391397 (Why is no real title available?)
- scientific article; zbMATH DE number 5060482 (Why is no real title available?)
- scientific article; zbMATH DE number 3381785 (Why is no real title available?)
- scientific article; zbMATH DE number 3390139 (Why is no real title available?)
- scientific article; zbMATH DE number 961607 (Why is no real title available?)
- A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems
- A Look at the Generalized Heron Problem through the Lens of Majorization-Minimization
- A finite algorithm for finding the projection of a point onto the canonical simplex of \({\mathbb R}^ n\)
- A quasi-Newton acceleration for high-dimensional optimization algorithms
- Alternating minimization as sequential unconstrained minimization: a survey
- An Algorithm for Restricted Least Squares Regression
- Applications of variational analysis to a generalized Fermat-Torricelli problem
- Applications of variational analysis to a generalized heron problem
- Applied iterative methods.
- Constrained Statistical Inference
- Convex analysis and nonlinear optimization. Theory and examples
- Convex optimization theory.
- Equivalent Subgradient Versions of Hamiltonian and Euler–Lagrange Equations in Variational Analysis
- How good are projection methods for convex feasibility problems?
- Monotonicity of quadratic-approximation algorithms
- Nonlinear optimization.
- Numerical analysis for statisticians
- On the effectiveness of projection methods for convex feasibility problems with linear inequality constraints
- Optimization
- Projected Newton Methods for Optimization Problems with Simple Constraints
- Proximal splitting methods in signal processing
- Proximity function minimization using multiple Bregman projections, with applications to split feasibility and Kullback--Leibler distance minimization
- Robust Statistics
- Sequential unconstrained minimization algorithms for constrained optimization
- Signal Recovery by Proximal Forward-Backward Splitting
- Solving a generalized Heron problem by means of convex analysis
- Sufficient conditions for the convergence of monotonic mathematical programming algorithms
- Tackling box-constrained optimization via a new projected quasi-Newton approach
- The Gradient Projection Method for Nonlinear Programming. Part I. Linear Constraints
- The MM alternative to EM
Cited in
(13)- High-performance statistical computing in the computing environments of the 2020s
- An MM Algorithm for Split Feasibility Problems
- Asymptotic distance and its application
- The log-exponential smoothing technique and Nesterov's accelerated gradient method for generalized Sylvester problems
- Algorithms for Sparse Support Vector Machines
- Proximal distance algorithms: theory and practice
- A Legacy of EM Algorithms
- The stochastic proximal distance algorithm
- Calculation of the Prokhorov distance by optimal quantization and maximum flow
- Clustering and multifacility location with constraints via distance function penalty methods and DC programming
- Distance measures with heavy aggregation operators
- The proximal distance algorithm
- A Sharper Computational Tool for Regression
This page was built for publication: Distance majorization and its applications
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q403660)
