A geometric integration approach to nonsmooth, nonconvex optimisation
From MaRDI portal
Publication:2088134
DOI10.1007/S10208-020-09489-2zbMATH Open1500.65033arXiv1807.07554OpenAlexW3187980583MaRDI QIDQ2088134FDOQ2088134
Authors: Erlend S. Riis, Matthias J. Ehrhardt, Carola-Bibiane Schönlieb, G. R. W. Quispel 
Publication date: 21 October 2022
Published in: Foundations of Computational Mathematics (Search for Journal in Brave)
Abstract: The optimisation of nonsmooth, nonconvex functions without access to gradients is a particularly challenging problem that is frequently encountered, for example in model parameter optimisation problems. Bilevel optimisation of parameters is a standard setting in areas such as variational regularisation problems and supervised machine learning. We present efficient and robust derivative-free methods called randomised Itoh--Abe methods. These are generalisations of the Itoh--Abe discrete gradient method, a well-known scheme from geometric integration, which has previously only been considered in the smooth setting. We demonstrate that the method and its favourable energy dissipation properties are well-defined in the nonsmooth setting. Furthermore, we prove that whenever the objective function is locally Lipschitz continuous, the iterates almost surely converge to a connected set of Clarke stationary points. We present an implementation of the methods, and apply it to various test problems. The numerical results indicate that the randomised Itoh--Abe methods are superior to state-of-the-art derivative-free optimisation methods in solving nonsmooth problems while remaining competitive in terms of efficiency.
Full work available at URL: https://arxiv.org/abs/1807.07554
Recommendations
- Limited memory discrete gradient bundle method for nonsmooth derivative-free optimization
- Discrete gradient method: Derivative-free method for nonsmooth optimization
- Manifold sampling for optimizing nonsmooth nonconvex compositions
- A Stochastic Subgradient Method for Nonsmooth Nonconvex Multilevel Composition Optimization
- Aggregate subgradient smoothing methods for large scale nonsmooth nonconvex optimisation and applications
Clarke subdifferentialgeometric numerical integrationdiscrete gradient methodsbilevel optimisationderivative-free optimisationnonsmooth optimisationnonconvex optimisation
Numerical mathematical programming methods (65K05) Nonconvex programming, global optimization (90C26) Stochastic programming (90C15) Image processing (compression, reconstruction, etc.) in information and communication theory (94A08) Derivative-free methods and methods using generalized derivatives (90C56)
Cites Work
- Discrete gradient method: Derivative-free method for nonsmooth optimization
- Nonlinear total variation based noise removal algorithms
- Parallelized hybrid optimization methods for nonsmooth problems using NOMAD and linesearch
- A Linesearch-Based Derivative-Free Approach for Nonsmooth Constrained Optimization
- Title not available (Why is that?)
- A Simplex Method for Function Minimization
- Title not available (Why is that?)
- Title not available (Why is that?)
- Proximal Alternating Minimization and Projection Methods for Nonconvex Problems: An Approach Based on the Kurdyka-Łojasiewicz Inequality
- A first-order primal-dual algorithm for convex problems with applications to imaging
- Methods of descent for nondifferentiable optimization
- Bilevel optimization for calibrating point spread functions in blind deconvolution
- A bilevel optimization approach for parameter learning in variational models
- Evaluating Derivatives
- Strongly Regular Generalized Equations
- Title not available (Why is that?)
- Geometric Numerical Integration
- Total generalized variation
- Random gradient-free minimization of convex functions
- Algorithm 909: NOMAD: nonlinear optimization with the MADS algorithm
- Nonsmooth optimization via quasi-Newton methods
- Mesh Adaptive Direct Search Algorithms for Constrained Optimization
- The NEWUOA software for unconstrained optimization without derivatives
- A proximal bundle method based on approximate subgradients
- Geometric integration using discrete gradients
- Bilevel parameter learning for higher-order total variation regularisation models
- On Nesterov's nonsmooth Chebyshev-Rosenbrock functions
- Subdifferential properties of quasiconvex and pseudoconvex functions: Unified approach
- Time integration and discrete Hamiltonian systems
- Optimizing an empirical scoring function for transmembrane protein structure determination
- Hamiltonian-conserving discrete canonical equations based on variational difference quotients
- Six lectures on the geometric integration of ODEs
- Title not available (Why is that?)
- A nonderivative version of the gradient sampling algorithm for nonsmooth nonconvex optimization
- Level set and PDE based reconstruction methods in imaging. Lecture notes given at the CIME summer school, Cetraro, Italy, September 2008
- Discrete gradient methods for solving ODEs numerically while preserving a first integral
- An adaptive gradient sampling algorithm for non-smooth optimization
- Generalized convexity of functions and generalized monotonicity of set-valued maps
- A survey of subdifferential calculus with applications
- A hybrid extended pattern search/genetic algorithm for multi-stage wind farm optimization
- Title not available (Why is that?)
- Derivative-free and blackbox optimization
- Derivative-free optimization methods
- A quasi-Newton algorithm for nonconvex, nonsmooth optimization with global convergence guarantees
- A TGV-based framework for variational image decompression, zooming, and reconstruction. I: Analytics
- On the equivalence between SOR-type methods for linear systems and the discrete gradient methods for gradient systems
- Discrete gradient methods for solving variational image regularisation models
- Bilevel Optimization with Nonsmooth Lower Level Problems
- Modern regularization methods for inverse problems
- Dissipative numerical schemes on Riemannian manifolds with applications to gradient flows
- Inexact derivative-free optimization for bilevel learning
- Improving the Flexibility and Robustness of Model-based Derivative-free Optimization Solvers
- Variational image regularization with Euler's elastica using a discrete gradient scheme
- Bregman Itoh-Abe methods for sparse optimisation
- Discrete gradients for computational Bayesian inference
- Title not available (Why is that?)
Cited In (5)
- Manifold Sampling for Optimizing Nonsmooth Nonconvex Compositions
- Existence results on Lagrange multiplier approach for gradient flows and application to optimization
- Variational image regularization with Euler's elastica using a discrete gradient scheme
- Title not available (Why is that?)
- Discrete gradients in short-range molecular dynamics simulations
Uses Software
This page was built for publication: A geometric integration approach to nonsmooth, nonconvex optimisation
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2088134)
