The effect of smooth parametrizations on nonconvex optimization landscapes
From MaRDI portal
Publication:6506211
DOI10.1007/S10107-024-02058-3arXiv2207.03512MaRDI QIDQ6506211FDOQ6506211
Authors: Eitan Levin, Joe Kileel, Nicolas Boumal 
Abstract: We develop new tools to study landscapes in nonconvex optimization. Given one optimization problem, we pair it with another by smoothly parametrizing the domain. This is either for practical purposes (e.g., to use smooth optimization algorithms with good guarantees) or for theoretical purposes (e.g., to reveal that the landscape satisfies a strict saddle property). In both cases, the central question is: how do the landscapes of the two problems relate? More precisely: how do desirable points such as local minima and critical points in one problem relate to those in the other problem? A key finding in this paper is that these relations are often determined by the parametrization itself, and are almost entirely independent of the cost function. Accordingly, we introduce a general framework to study parametrizations by their effect on landscapes. The framework enables us to obtain new guarantees for an array of problems, some of which were previously treated on a case-by-case basis in the literature. Applications include: optimization over low-rank matrices and tensors by optimizing over a factorization; the Burer--Monteiro approach to semidefinite programs; training neural networks by optimizing over their weights and biases; and quotienting out symmetries.
Optimality conditions and duality in mathematical programming (90C46) Nonconvex programming, global optimization (90C26) Set-valued and variational analysis (49J53) Applications of differential geometry to data and computer science (53Z50)
This page was built for publication: The effect of smooth parametrizations on nonconvex optimization landscapes
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6506211)
