A Riemannian Proximal Newton Method

DOI10.1137/23M1565097arXiv2304.04032WikidataQ128162580 ScholiaQ128162580MaRDI QIDQ6202763FDOQ6202763

Authors: P.-A. Absil, Wen Huang, Rujun Jiang, Simon Vary 

Publication date: 27 February 2024

Published in: SIAM Journal on Optimization (Search for Journal in Brave)

Abstract: In recent years, the proximal gradient method and its variants have been generalized to Riemannian manifolds for solving optimization problems with an additively separable structure, i.e., $f+h$, where $f$ is continuously differentiable, and $h$ may be nonsmooth but convex with computationally reasonable proximal mapping. In this paper, we generalize the proximal Newton method to embedded submanifolds for solving the type of problem with $h(x)=mu|x{|}_1$. The generalization relies on the Weingarten and semismooth analysis. It is shown that the Riemannian proximal Newton method has a local superlinear convergence rate under certain reasonable assumptions. Moreover, a hybrid version is given by concatenating a Riemannian proximal gradient method and the Riemannian proximal Newton method. It is shown that if the objective function satisfies the Riemannian KL property and the switch parameter is chosen appropriately, then the hybrid method converges globally and also has a local superlinear convergence rate. Numerical experiments on random and synthetic data are used to demonstrate the performance of the proposed methods.

Full work available at URL: https://arxiv.org/abs/2304.04032

Recommendations

• Riemannian proximal gradient methods
• An inexact Riemannian proximal gradient method
• Proximal gradient algorithm with trust region scheme on Riemannian manifold
• Newton acceleration on manifolds identified by proximal gradient methods
• A proximal bundle algorithm for nonsmooth optimization on Riemannian manifolds

zbMATH Keywords

superlinearRiemannian optimizationmanifold optimizationproximal Newton methodembedded submanifold

Mathematics Subject Classification ID

Nonconvex programming, global optimization (90C26) Nonlinear programming (90C30) Computational methods for problems pertaining to operations research and mathematical programming (90-08) Mathematical programming (90Cxx)

Cites Work

• A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems
• IMRO: A proximal quasi-Newton method for solving \(\ell_1\)-regularized least squares problems
• Title not available (Why is that?)
• First-order methods in optimization
• Title not available (Why is that?)
• Title not available (Why is that?)
• Compressed modes for variational problems in mathematics and physics
• Fast Gradient-Based Algorithms for Constrained Total Variation Image Denoising and Deblurring Problems
• A nonsmooth version of Newton's method
• Accelerated and inexact forward-backward algorithms
• Implicit Functions and Solution Mappings
• Title not available (Why is that?)
• Title not available (Why is that?)
• \(\varepsilon\)-subgradient algorithms for locally Lipschitz functions on Riemannian manifolds
• An Extrinsic Look at the Riemannian Hessian
• Semismooth Homeomorphisms and Strong Stability of Semidefinite and Lorentz Complementarity Problems
• A further result on an implicit function theorem for locally Lipschitz functions
• Inverse and implicit function theorems forH-differentiable and semismooth functions
• Proximal Newton-type methods for minimizing composite functions
• Intrinsic representation of tangent vectors and vector transports on matrix manifolds
• The rate of convergence of Nesterov's accelerated forward-backward method is actually faster than \(1/k^2\)
• A proximal Newton-type method for equilibrium problems
• Lectures on convex optimization
• Weakly convex optimization over Stiefel manifold using Riemannian subgradient-type methods
• Introduction to Riemannian Manifolds
• A globally convergent proximal Newton-type method in nonsmooth convex optimization
• Riemannian proximal gradient methods
• Global rates of convergence for nonconvex optimization on manifolds
• A regularized semi-smooth Newton method with projection steps for composite convex programs
• Large-scale sparse inverse covariance matrix estimation
• A Riemannian gradient sampling algorithm for nonsmooth optimization on manifolds
• On efficiently solving the subproblems of a level-set method for fused lasso problems
• Convex Sparse Spectral Clustering: Single-View to Multi-View
• Proximal gradient method for nonsmooth optimization over the Stiefel manifold
• A semismooth Newton based augmented Lagrangian method for nonsmooth optimization on matrix manifolds
• An inexact Riemannian proximal gradient method
• An Introduction to Optimization on Smooth Manifolds
• An extension of fast iterative shrinkage‐thresholding algorithm to Riemannian optimization for sparse principal component analysis
• A proximal bundle algorithm for nonsmooth optimization on Riemannian manifolds
• Proximal quasi-Newton method for composite optimization over the Stiefel manifold

Cited In (3)

• An adaptive regularized proximal Newton-type methods for composite optimization over the Stiefel manifold
• Nonsmooth optimization over the Stiefel manifold and beyond: proximal gradient method and recent variants
• Proximal gradient algorithm with trust region scheme on Riemannian manifold

This page was built for publication: A Riemannian Proximal Newton Method

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6202763)