Principal support vector machines for linear and nonlinear sufficient dimension reduction

DOI10.1214/11-AOS932zbMATH Open1246.62153arXiv1203.2790OpenAlexW2035140935WikidataQ57434300 ScholiaQ57434300MaRDI QIDQ449992FDOQ449992

Authors: Andreas Artemiou, Bing Li, Lexin Li 

Publication date: 3 September 2012

Published in: The Annals of Statistics (Search for Journal in Brave)

Abstract: We introduce a principal support vector machine (PSVM) approach that can be used for both linear and nonlinear sufficient dimension reduction. The basic idea is to divide the response variables into slices and use a modified form of support vector machine to find the optimal hyperplanes that separate them. These optimal hyperplanes are then aligned by the principal components of their normal vectors. It is proved that the aligned normal vectors provide an unbiased, $sqrtn$-consistent, and asymptotically normal estimator of the sufficient dimension reduction space. The method is then generalized to nonlinear sufficient dimension reduction using the reproducing kernel Hilbert space. In that context, the aligned normal vectors become functions and it is proved that they are unbiased in the sense that they are functions of the true nonlinear sufficient predictors. We compare PSVM with other sufficient dimension reduction methods by simulation and in real data analysis, and through both comparisons firmly establish its practical advantages.

Full work available at URL: https://arxiv.org/abs/1203.2790

Recommendations

• Sufficient dimension reduction via principal L\(q\) support vector machine
• Principal weighted support vector machines for sufficient dimension reduction in binary classification
• A cost based reweighted scheme of principal support vector machine
• A general theory for nonlinear sufficient dimension reduction: formulation and estimation
• High-dimensional sufficient dimension reduction through principal projections

zbMATH Keywords

principal componentsreproducing kernel Hilbert spaceinverse regressioncontour regressioninvariant kernel

Mathematics Subject Classification ID

Nonparametric regression and quantile regression (62G08) Factor analysis and principal components; correspondence analysis (62H25) Learning and adaptive systems in artificial intelligence (68T05) Estimation in multivariate analysis (62H12) Applications of functional analysis in probability theory and statistics (46N30) Graphical methods in statistics (62A09)

Cites Work

• Estimating the dimension of a model
• Asymptotic Statistics
• Sliced Inverse Regression for Dimension Reduction
• On Sliced Inverse Regression With High-Dimensional Covariates
• Theory of Reproducing Kernels
• Comment
• On almost linearity of low dimensional projections from high dimensional data
• Title not available (Why is that?)
• Title not available (Why is that?)
• An Adaptive Estimation of Dimension Reduction Space
• Algorithmic Learning Theory
• Dimension reduction for nonelliptically distributed predictors
• Regression analysis under link violation
• Contour regression: a general approach to dimension reduction
• On Directional Regression for Dimension Reduction
• Title not available (Why is that?)
• Title not available (Why is that?)
• Kernel dimension reduction in regression
• Title not available (Why is that?)
• Fisher lecture: Dimension reduction in regression
• Sufficient Dimension Reduction via Inverse Regression
• Principal fitted components for dimension reduction in regression
• Dimension reduction for conditional mean in regression
• On Principal Hessian Directions for Data Visualization and Dimension Reduction: Another Application of Stein's Lemma
• Title not available (Why is that?)
• Dimension Reduction for the Conditionalkth Moment in Regression
• Asymptotics for sliced average variance estimation
• The commutation matrix: Some properties and applications
• Title not available (Why is that?)
• Title not available (Why is that?)
• Successive direction extraction for estimating the central subspace in a multiple-index regres\-sion
• On the distribution of the left singular vectors of a random matrix and its applications
• Graphics for Regressions With a Binary Response
• A characterization of spherical distributions
• An RKHS formulation of the inverse regression dimension-reduction problem
• Convex functional analysis
• Title not available (Why is that?)
• A nonlinear multi-dimensional variable selection method for high dimensional data: sparse MAVE
• Nonparametric estimating equations based on a penalized information criterion
• On Quasi Likelihood Equations with Non-parametric Weights

Cited In (45)

• Predictive power of principal components for single-index model and sufficient dimension reduction
• A brief review of linear sufficient dimension reduction through optimization
• On an Additive Semigraphoid Model for Statistical Networks With Application to Pathway Analysis
• Dimension reduction techniques for conditional expectiles
• Using adaptively weighted large margin classifiers for robust sufficient dimension reduction
• Using DAGs to identify the sufficient dimension reduction in the principal fitted components model
• Dimension reduction in binary response regression: a joint modeling approach
• On expectile-assisted inverse regression estimation for sufficient dimension reduction
• Sliced inverse median difference regression
• Sufficient dimension reduction via principal L\(q\) support vector machine
• Dimension reduction for functional data based on weak conditional moments
• Principal quantile regression for sufficient dimension reduction with heteroscedasticity
• Principal weighted logistic regression for sufficient dimension reduction in binary classification
• A robust proposal of estimation for the sufficient dimension reduction problem
• Principal minimax support vector machine for sufficient dimension reduction with contaminated data
• Quantile-slicing estimation for dimension reduction in regression
• On sufficient dimension reduction for functional data: inverse moment-based methods
• A new reproducing kernel‐based nonlinear dimension reduction method for survival data
• High-dimensional sufficient dimension reduction through principal projections
• A comparative study of the use of large margin classifiers on seismic data
• A general theory for nonlinear sufficient dimension reduction: formulation and estimation
• Deep nonlinear sufficient dimension reduction
• Graph informed sliced inverse regression
• On post dimension reduction statistical inference
• Inverse regression-based uncertainty quantification algorithms for high-dimensional models: theory and practice
• On a nonlinear extension of the principal fitted component model
• Statistical learning on emerging economies
• On the predictive potential of kernel principal components
• Nonlinear predictive directions in clinical trials
• On sufficient dimension reduction via principal asymmetric least squares
• Using sliced inverse mean difference for sufficient dimension reduction
• Dimension reduction-based adaptive-to-model semi-supervised classification
• Gradient-based approach to sufficient dimension reduction with functional or longitudinal covariates
• Sliced Inverse Regression in Metric Spaces
• An ensemble of inverse moment estimators for sufficient dimension reduction
• A study on imbalance support vector machine algorithms for sufficient dimension reduction
• A Cost Based Reweighted Scheme of Principal Support Vector Machine
• Gradient-Based Kernel Dimension Reduction for Regression
• On the conditional distributions of low-dimensional projections from high-dimensional data
• Nonlinear and additive principal component analysis for functional data
• Penalized principal logistic regression for sparse sufficient dimension reduction
• On efficient dimension reduction with respect to a statistical functional of interest
• Principal weighted support vector machines for sufficient dimension reduction in binary classification
• Title not available (Why is that?)
• Nonlinear dimension reduction for conditional quantiles

Uses Software

• Kernlab

This page was built for publication: Principal support vector machines for linear and nonlinear sufficient dimension reduction

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q449992)