RedEx: beyond fixed representation methods via convex optimization
Optimizing Neural networks is a difficult task which is still not well understood. On the other hand, fixed representation methods such as kernels and random features have provable optimization guarantees but inferior performance due to their inherent inability to learn the representations. In this paper, the authors aim at bridging this gap by presenting a novel architecture called RedEx (ReducedExpander Extractor) that is as expressive as neural networks and can also be trained in a layer-wise fashion via a convex program with semi-definite constraints and optimization guarantees. They also show that RedEx provably surpasses fixed representation methods, in the sense that it can efficiently learn a family of target functions which fixed representation methods cannot.\N\NFor the entire collection see [Zbl 1551.68029].
- 10.1162/153244303321897681
- A faster interior point method for semidefinite programming
- A primal-dual interior-point algorithm for nonsymmetric exponential-cone optimization
- Convex optimization: algorithms and complexity
- Cryptographic limitations on learning Boolean formulae and finite automata
- scientific article; zbMATH DE number 48956 (Why is no real title available?)
- Interior Gradient and Proximal Methods for Convex and Conic Optimization
- Interior-point methods
- Linearized two-layers neural networks in high dimension
- Phase retrieval via Wirtinger flow: theory and algorithms
- Phaselift: exact and stable signal recovery from magnitude measurements via convex programming
- Understanding machine learning. From theory to algorithms
This page was built for publication: RedEx: beyond fixed representation methods via convex optimization
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q7017102)
