Homotopy continuation approaches for robust SV classification and regression

DOI10.1007/S10994-017-5627-7zbMATH Open1455.68186arXiv1507.03229OpenAlexW2964283741MaRDI QIDQ1697922FDOQ1697922

Authors: Shinya Suzumura, Kohei Ogawa, Masashi Sugiyama, Masayuki Karasuyama, Ichiro Takeuchi 

Publication date: 20 February 2018

Published in: Machine Learning (Search for Journal in Brave)

Abstract: In support vector machine (SVM) applications with unreliable data that contains a portion of outliers, non-robustness of SVMs often causes considerable performance deterioration. Although many approaches for improving the robustness of SVMs have been studied, two major challenges remain in robust SVM learning. First, robust learning algorithms are essentially formulated as non-convex optimization problems. It is thus important to develop a non-convex optimization method for robust SVM that can find a good local optimal solution. The second practical issue is how one can tune the hyperparameter that controls the balance between robustness and efficiency. Unfortunately, due to the non-convexity, robust SVM solutions with slightly different hyper-parameter values can be significantly different, which makes model selection highly unstable. In this paper, we address these two issues simultaneously by introducing a novel homotopy approach to non-convex robust SVM learning. Our basic idea is to introduce parametrized formulations of robust SVM which bridge the standard SVM and fully robust SVM via the parameter that represents the influence of outliers. We characterize the necessary and sufficient conditions of the local optimal solutions of robust SVM, and develop an algorithm that can trace a path of local optimal solutions when the influence of outliers is gradually decreased. An advantage of our homotopy approach is that it can be interpreted as simulated annealing, a common approach for finding a good local optimal solution in non-convex optimization problems. In addition, our homotopy method allows stable and efficient model selection based on the path of local optimal solutions. Empirical performances of the proposed approach are demonstrated through intensive numerical experiments both on robust classification and regression problems.

Full work available at URL: https://arxiv.org/abs/1507.03229

Recommendations

• Convergence rate of SVM for kernel-based robust regression
• Robust support vector regression in the primal
• Robust support vector machines for classification and computational issues
• Training robust support vector regression with smooth non-convex loss function
• Robust support vector machines for classification with nonconvex and smooth losses

zbMATH Keywords

classificationregressionnon-convex optimizationparametric programmingrobust support vector machine

Mathematics Subject Classification ID

Nonparametric robustness (62G35) Classification and discrimination; cluster analysis (statistical aspects) (62H30) General nonlinear regression (62J02) Learning and adaptive systems in artificial intelligence (68T05) Applications of mathematical programming (90C90) Nonconvex programming, global optimization (90C26)

Cites Work

• Nearly unbiased variable selection under minimax concave penalty
• SparseNet: coordinate descent with nonconvex penalties
• Title not available (Why is that?)
• Support-vector networks
• The Concave-Convex Procedure
• Robust Truncated Hinge Loss Support Vector Machines
• Title not available (Why is that?)
• Title not available (Why is that?)
• On ψ-Learning
• 10.1162/153244303321897690
• Multicategory ψ-Learning
• Approximating parameterized convex optimization problems
• Title not available (Why is that?)
• Title not available (Why is that?)
• Algorithmics for hard problems.
• Title not available (Why is that?)

Cited In (3)

• Convergence rate of SVM for kernel-based robust regression
• Learning rates of kernel-based robust classification
• A robust outlier control framework for classification designed with family of homotopy loss function

Uses Software

• sparsenet

This page was built for publication: Homotopy continuation approaches for robust SV classification and regression

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1697922)