Classification and Regression Using an Outer Approximation Projection-Gradient Method

DOI10.1109/TSP.2017.2709262MaRDI QIDQ4621845zbMATH OpenOpenAlexDBLPWikidataFDO

Authors Michel Barlaud, Wafa Belhajali, Patrick L. Combettes, Lionel Fillatre

Publication date 8 February 2019

Published in IEEE Transactions on Signal Processing (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1506.02196

Mathematics Subject Classification ID

Sensitivity, stability, parametric optimization (90C31) Signal theory (characterization, reconstruction, filtering, etc.) (94A12)

Abstract: This paper deals with sparse feature selection and grouping for classification and regression. The classification or regression problems under consideration consists in minimizing a convex empirical risk function subject to an $el{l}^1$ constraint, a pairwise $el{l}^{i}nfty$ constraint, or a pairwise $el{l}^1$ constraint. Existing work, such as the Lasso formulation, has focused mainly on Lagrangian penalty approximations, which often require ad hoc or computationally expensive procedures to determine the penalization parameter. We depart from this approach and address the constrained problem directly via a splitting method. The structure of the method is that of the classical gradient-projection algorithm, which alternates a gradient step on the objective and a projection step onto the lower level set modeling the constraint. The novelty of our approach is that the projection step is implemented via an outer approximation scheme in which the constraint set is approximated by a sequence of simple convex sets consisting of the intersection of two half-spaces. Convergence of the iterates generated by the algorithm is established for a general smooth convex minimization problem with inequality constraints. Experiments on both synthetic and biological data show that our method outperforms penalty methods.

Recommendations

• An Outer-Product-of-Gradient Approach to Dimension Reduction and its Application to Classification in High Dimensional Space
• Algorithmic Learning Theory
• A version of the regularized gradient projection method
• Regularized gradient projection method based on set extension
• scientific article; zbMATH DE number 4028654
• Estimation of gradients and coordinate covariation in classification
• Classification and Regression via Integer Optimization
• Gradient projection methods for quadratic programs and applications in training support vector machines

Cited in

(4)

• A filtered bucket-clustering method for projection onto the simplex and the \(\ell_1\) ball
• A random block-coordinate Douglas-Rachford splitting method with low computational complexity for binary logistic regression
• The application of an oblique-projected Landweber method to a model of supervised learning
• Outer perturbations of a projection method and two approximation methods for the split equality problem

This page was built for publication: Classification and Regression Using an Outer Approximation Projection-Gradient Method

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4621845)