An Improved Unconstrained Approach for Bilevel Optimization
From MaRDI portal
Abstract: In this paper, we focus on the nonconvex-strongly-convex bilevel optimization problem (BLO). In this BLO, the objective function of the upper-level problem is nonconvex and possibly nonsmooth, and the lower-level problem is smooth and strongly convex with respect to the underlying variable . We show that the feasible region of BLO is a Riemannian manifold. Then we transform BLO to its corresponding unconstrained constraint dissolving problem (CDB), whose objective function is explicitly formulated from the objective functions in BLO. We prove that BLO is equivalent to the unconstrained optimization problem CDB. Therefore, various efficient unconstrained approaches, together with their theoretical results, can be directly applied to BLO through CDB. We propose a unified framework for developing subgradient-based methods for CDB. Remarkably, we show that several existing efficient algorithms can fit the unified framework and be interpreted as descent algorithms for CDB. These examples further demonstrate the great potential of our proposed approach.
Recommendations
- scientific article; zbMATH DE number 5176444
- Techniques for gradient-based bilevel optimization with non-smooth lower level problems
- Methodology and first-order algorithms for solving nonsmooth and non-strongly convex bilevel optimization problems
- First-order penalty methods for bilevel optimization
- Bilevel optimization: reformulation and first optimality conditions
Cites work
- A brief introduction to manifold optimization
- A Lagrange multiplier expression method for bilevel polynomial optimization
- A regularized smoothing method for fully parameterized convex problems with applications to convex and nonconvex two-stage stochastic programming
- A Robust Gradient Sampling Algorithm for Nonsmooth, Nonconvex Optimization
- A sequential quadratic programming algorithm for nonconvex, nonsmooth constrained optimization
- A Two-Timescale Stochastic Algorithm Framework for Bilevel Optimization: Complexity Analysis and Application to Actor-Critic
- An inertial Newton algorithm for deep learning
- An Introduction to Optimization on Smooth Manifolds
- An overview of bilevel optimization
- Approximating Subdifferentials by Random Sampling of Gradients
- Conservative set valued fields, automatic differentiation, stochastic gradient methods and deep learning
- Constraint Qualifications and KKT Conditions for Bilevel Programming Problems
- Convergence of constant step stochastic gradient descent for non-smooth non-convex functions
- Decomposition algorithms for some deterministic and two-stage stochastic single-leader multi-follower games
- Geometric categories and o-minimal structures
- scientific article; zbMATH DE number 46303 (Why is no real title available?)
- scientific article; zbMATH DE number 5223994 (Why is no real title available?)
- Model completeness results for expansions of the ordered field of real numbers by restricted Pfaffian functions and the exponential function
- New Branch-and-Bound Rules for Linear Bilevel Programming
- On stochastic gradient and subgradient methods with adaptive steplength sequences
- OnActor-Critic Algorithms
- Random gradient-free minimization of convex functions
- Randomized smoothing for stochastic optimization
- Sample complexity of sample average approximation for conditional stochastic optimization
- Smoothing SQP Methods for Solving Degenerate Nonsmooth Constrained Optimization Problems with Applications to Bilevel Programs
- Solving Stochastic Compositional Optimization is Nearly as Easy as Solving Stochastic Optimization
- Stochastic subgradient method converges on tame functions
Cited in
(8)- A one-step worst-case optimal algorithm for bi-objective univariate optimization
- A New Predictor Corrector Variant for Unconstrained Bi-objective Optimization Problems
- Convergence properties of stochastic proximal subgradient method in solving a class of composite optimization problems with cardinality regularizer
- First-order penalty methods for bilevel optimization
- A gentle introduction to algorithms for bilevel optimization from machine learning
- CDOpt: a Python package for a class of Riemannian optimization
- A constraint dissolving approach for nonsmooth optimization over the Stiefel manifold
- A minimization approach for minimax optimization with coupled constraints
This page was built for publication: An Improved Unconstrained Approach for Bilevel Optimization
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6076870)
