A Riemannian Proximal Newton Method

DOI10.1137/23M1565097WikidataQ128162580 ScholiaQ128162580MaRDI QIDQ6202763FDOQ6202763

Authors P.-A. Absil, Wen Huang, Rujun Jiang, Simon Vary

Publication date 27 February 2024

Published in SIAM Journal on Optimization (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2304.04032

zbMATH Keywords

superlinearRiemannian optimizationmanifold optimizationproximal Newton methodembedded submanifold

Mathematics Subject Classification ID

Nonconvex programming, global optimization (90C26) Nonlinear programming (90C30) Computational methods for problems pertaining to operations research and mathematical programming (90-08) Mathematical programming (90Cxx)

Abstract: In recent years, the proximal gradient method and its variants have been generalized to Riemannian manifolds for solving optimization problems with an additively separable structure, i.e., $f+h$, where $f$ is continuously differentiable, and $h$ may be nonsmooth but convex with computationally reasonable proximal mapping. In this paper, we generalize the proximal Newton method to embedded submanifolds for solving the type of problem with $h(x)=mu|x{|}_1$. The generalization relies on the Weingarten and semismooth analysis. It is shown that the Riemannian proximal Newton method has a local superlinear convergence rate under certain reasonable assumptions. Moreover, a hybrid version is given by concatenating a Riemannian proximal gradient method and the Riemannian proximal Newton method. It is shown that if the objective function satisfies the Riemannian KL property and the switch parameter is chosen appropriately, then the hybrid method converges globally and also has a local superlinear convergence rate. Numerical experiments on random and synthetic data are used to demonstrate the performance of the proposed methods.

Recommendations

• Riemannian proximal gradient methods
• An inexact Riemannian proximal gradient method
• Proximal gradient algorithm with trust region scheme on Riemannian manifold
• Newton acceleration on manifolds identified by proximal gradient methods
• A proximal bundle algorithm for nonsmooth optimization on Riemannian manifolds

Cites work

• scientific article; zbMATH DE number 46303 (Why is no real title available?)
• scientific article; zbMATH DE number 1765103 (Why is no real title available?)
• scientific article; zbMATH DE number 845714 (Why is no real title available?)
• scientific article; zbMATH DE number 5937962 (Why is no real title available?)
• scientific article; zbMATH DE number 5223994 (Why is no real title available?)
• A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems
• A Riemannian gradient sampling algorithm for nonsmooth optimization on manifolds
• A further result on an implicit function theorem for locally Lipschitz functions
• A globally convergent proximal Newton-type method in nonsmooth convex optimization
• A nonsmooth version of Newton's method
• A proximal Newton-type method for equilibrium problems
• A proximal bundle algorithm for nonsmooth optimization on Riemannian manifolds
• A regularized semi-smooth Newton method with projection steps for composite convex programs
• A semismooth Newton based augmented Lagrangian method for nonsmooth optimization on matrix manifolds
• Accelerated and inexact forward-backward algorithms
• An Extrinsic Look at the Riemannian Hessian
• An Introduction to Optimization on Smooth Manifolds
• An extension of fast iterative shrinkage‐thresholding algorithm to Riemannian optimization for sparse principal component analysis
• An inexact Riemannian proximal gradient method
• Compressed modes for variational problems in mathematics and physics
• Convex Sparse Spectral Clustering: Single-View to Multi-View
• Fast Gradient-Based Algorithms for Constrained Total Variation Image Denoising and Deblurring Problems
• First-order methods in optimization
• Global rates of convergence for nonconvex optimization on manifolds
• IMRO: A proximal quasi-Newton method for solving \(\ell_1\)-regularized least squares problems
• Implicit Functions and Solution Mappings
• Intrinsic representation of tangent vectors and vector transports on matrix manifolds
• Introduction to Riemannian Manifolds
• Inverse and implicit function theorems forH-differentiable and semismooth functions
• Large-scale sparse inverse covariance matrix estimation
• Lectures on convex optimization
• On efficiently solving the subproblems of a level-set method for fused lasso problems
• Proximal Newton-type methods for minimizing composite functions
• Proximal gradient method for nonsmooth optimization over the Stiefel manifold
• Proximal quasi-Newton method for composite optimization over the Stiefel manifold
• Riemannian proximal gradient methods
• Semismooth Homeomorphisms and Strong Stability of Semidefinite and Lorentz Complementarity Problems
• The rate of convergence of Nesterov's accelerated forward-backward method is actually faster than \(1/k^2\)
• Weakly convex optimization over Stiefel manifold using Riemannian subgradient-type methods
• \(\varepsilon\)-subgradient algorithms for locally Lipschitz functions on Riemannian manifolds

Cited in

(3)

• An adaptive regularized proximal Newton-type methods for composite optimization over the Stiefel manifold
• Nonsmooth optimization over the Stiefel manifold and beyond: proximal gradient method and recent variants
• Proximal gradient algorithm with trust region scheme on Riemannian manifold

This page was built for publication: A Riemannian Proximal Newton Method

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6202763)