Modified spectral projected subgradient method: convergence analysis and momentum parameter heuristics

MaRDI QIDQ1653958zbMATH OpenFDO

Authors Samantha Clapp, Charles Cratty, Breeanna Page, Milagros C. Loreto

Publication date 7 August 2018

Published in Bulletin of Computational Applied Mathematics (Search for Journal in Brave)

Full work available at URL http://drive.google.com/open?id=0B5GyVVQ6O030VU5fVFZ4VXpFZVk

zbMATH Keywords

momentum termspectral projected gradientsubgradient methods

Mathematics Subject Classification ID

Numerical optimization and variational techniques (65K10) Derivative-free methods and methods using generalized derivatives (90C56)

Review text

Summary: The modified spectral projected subgradient (MSPS) was proposed to solve Langrangen dual problems, and its convergence was shown when the momentum term was zero. The MSPS uses a momentum term in order to speed up its convergence. The momentum term is built on the multiplication of a momentum parameter and the direction of the previous iterate. In this work, we show convergence when the momentum parameter is a non-zero constant. We also propose heuristics to choose the momentum parameter intended to avoid the zigzagging phenomenon of kind I. This phenomenon is present in the MSPS when at an iterate the subgradient forms an obtuse angle with the previous direction. We identify and diminish the zigzagging phenomenon of kind I on setcovering problems, and compare our numerical results to those of the original MSPS algorithm.

Recommendations

• Convergence analysis for the modified spectral projected subgradient method
• Spectral projected subgradient with a momentum term for the Lagrangean dual approach
• Convergence properties of nonmonotone spectral projected gradient methods
• A multivariate spectral projected gradient method for bound constrained optimization
• On a modified subgradient algorithm for dual problems via sharp augmented Lagrangian

Cites work

• A Heuristic Method for the Set Covering Problem
• A Nonmonotone Line Search Technique for Newton’s Method
• Algorithm 813
• An Incremental Gradient(-Projection) Method with Momentum Term and Adaptive Stepsize Rule
• Convergence analysis for the modified spectral projected subgradient method
• scientific article; zbMATH DE number 3282977 (Why is no real title available?)
• Nonmonotone Spectral Projected Gradient Methods on Convex Sets
• Spectral projected subgradient with a momentum term for the Lagrangean dual approach
• Spectral residual method without gradient information for solving large-scale nonlinear systems of equations
• The Barzilai and Borwein Gradient Method for the Large Scale Unconstrained Minimization Problem

Cited in

(4)

• A numerical study of applying spectral-step subgradient method for solving nonsmooth unconstrained optimization problems
• Convergence analysis for the modified spectral projected subgradient method
• A new spectral conjugate subgradient method with application in computed tomography image reconstruction
• Spectral projected subgradient with a momentum term for the Lagrangean dual approach

Describes a project that uses

Uses Software

• SPG
• OR-Library

This page was built for publication: Modified spectral projected subgradient method: convergence analysis and momentum parameter heuristics

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1653958)