An Algorithm for Unconstrained Quadratically Penalized Convex Optimization
From MaRDI portal
Abstract: A descent algorithm, "Quasi-Quadratic Minimization with Memory" (QQMM), is proposed for unconstrained minimization of the sum, , of a non-negative convex function, , and a quadratic form. Such problems come up in regularized estimation in machine learning and statistics. In addition to values of , QQMM requires the (sub)gradient of . Two features of QQMM help keep low the number of evaluations of the objective function it needs. First, QQMM provides good control over stopping the iterative search. This feature makes QQMM well adapted to statistical problems because in such problems the objective function is based on random data and therefore stopping early is sensible. Secondly, QQMM uses a complex method for determining trial minimizers of . After a description of the problem and algorithm a simulation study comparing QQMM to the popular BFGS optimization algorithm is described. The simulation study and other experiments suggest that QQMM is generally substantially faster than BFGS in the problem domain for which it was designed. A QQMM-BFGS hybrid is also generally substantially faster than BFGS but does better than QQMM when QQMM is very slow.
Recommendations
- An unconstrained convex programming approach to solving convex quadratic programming problems
- A q-conjugate gradient algorithm for unconstrained optimization problems
- A globally and quadratically convergent algorithm with efficient implementation for unconstrained optimization
- An efficient algorithm for convex quadratic semi-definite optimization
- Algorithms for quasiconvex minimization
- A new penalty function algorithm for convex quadratic programming
- An algorithm for indefinite quadratic programming with convex constraints
- A class of exponential quadratically convergent iterative formulae for unconstrained optimization
- Penalized semidefinite programming for quadratically-constrained quadratic optimization
- A new algorithm for concave quadratic programming
Cites work
- A new approach to variable metric algorithms
- An Algorithm for Unconstrained Quadratically Penalized Convex Optimization
- Convex analysis and nonlinear optimization. Theory and examples.
- scientific article; zbMATH DE number 3678917 (Why is no real title available?)
- scientific article; zbMATH DE number 44982 (Why is no real title available?)
- scientific article; zbMATH DE number 45848 (Why is no real title available?)
- scientific article; zbMATH DE number 47310 (Why is no real title available?)
- scientific article; zbMATH DE number 193093 (Why is no real title available?)
- scientific article; zbMATH DE number 1313113 (Why is no real title available?)
- scientific article; zbMATH DE number 708500 (Why is no real title available?)
- Introductory lectures on convex optimization. A basic course.
- Regularization networks and support vector machines
- Statistical behavior and consistency of classification methods based on convex risk minimization.
- The Convergence of a Class of Double-rank Minimization Algorithms 1. General Considerations
- The elements of statistical learning. Data mining, inference, and prediction
Cited in
(7)- A modified local quadratic approximation algorithm for penalized optimization problems
- UOBYQA: unconstrained optimization by quadratic approximation
- An Algorithm for Unconstrained Quadratically Penalized Convex Optimization
- An unconstrained convex programming approach to solving convex quadratic programming problems
- scientific article; zbMATH DE number 1546513 (Why is no real title available?)
- Quadratic programming and penalized regression
- scientific article; zbMATH DE number 5263160 (Why is no real title available?)
This page was built for publication: An Algorithm for Unconstrained Quadratically Penalized Convex Optimization
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3087581)
