Wasserstein discriminant analysis

DOI10.1007/S10994-018-5717-1MaRDI QIDQ1722729zbMATH OpenOpenAlexFDO

Authors Rémi Flamary, Marco Cuturi, Nicolas Courty, Alain Rakotomamonjy

Publication date 18 February 2019

Published in Machine Learning (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1608.08063

zbMATH Keywords

linear discriminant analysisoptimal transportWasserstein distance

Mathematics Subject Classification ID

Classification and discrimination; cluster analysis (statistical aspects) (62H30) Probabilistic measure theory (60A10)

Abstract: Wasserstein Discriminant Analysis (WDA) is a new supervised method that can improve classification of high-dimensional data by computing a suitable linear map onto a lower dimensional subspace. Following the blueprint of classical Linear Discriminant Analysis (LDA), WDA selects the projection matrix that maximizes the ratio of two quantities: the dispersion of projected points coming from different classes, divided by the dispersion of projected points coming from the same class. To quantify dispersion, WDA uses regularized Wasserstein distances, rather than cross-variance measures which have been usually considered, notably in LDA. Thanks to the the underlying principles of optimal transport, WDA is able to capture both global (at distribution scale) and local (at samples scale) interactions between classes. Regularized Wasserstein distances can be computed using the Sinkhorn matrix scaling algorithm; We show that the optimization of WDA can be tackled using automatic differentiation of Sinkhorn iterations. Numerical experiments show promising results both in terms of prediction and visualization on toy examples and real life datasets such as MNIST and on deep features obtained from a subset of the Caltech dataset.

Recommendations

• Statistical data analysis in the Wasserstein space
• Hybrid Wasserstein distance and fast distribution clustering
• Inference for empirical Wasserstein distances on finite spaces
• Geodesic PCA versus Log-PCA of Histograms in the Wasserstein Space
• Maxi-Min discriminant analysis via online learning

Cites work

• scientific article; zbMATH DE number 5223994 (Why is no real title available?)
• An overview of bilevel optimization
• Choosing multiple parameters for support vector machines
• Computational optimal transport. With applications to data sciences
• Dimension reduction: a guided tour
• Dimensionality reduction of multimodal labeled data by local Fisher discriminant analysis
• Direct estimation of the derivative of quadratic mutual information with application in supervised dimension reduction
• Distance metric learning for large margin nearest neighbor classification
• Iterative Bregman projections for regularized transportation problems
• Learning deep architectures for AI
• Manopt, a Matlab toolbox for optimization on manifolds
• Optimal Transport
• Optimization Problems with Perturbations: A Guided Tour
• Pymanopt: a Python toolbox for optimization on manifolds using automatic differentiation
• Sufficient dimension reduction via squared-loss mutual information estimation
• The Sinkhorn–Knopp Algorithm: Convergence and Applications
• The elements of statistical learning. Data mining, inference, and prediction
• Two-stage image denoising by principal component analysis with local pixel grouping
• Visualizing data using t-SNE

Cited in

(18)

• Wasserstein Regression
• A sparse control approach to optimal sensor placement in PDE-constrained parameter estimation problems
• scientific article; zbMATH DE number 7650079 (Why is no real title available?)
• Semi-discrete optimal transport: hardness, regularization and numerical solution
• Projection-based techniques for high-dimensional optimal transport problems
• An Improved Central Limit Theorem and Fast Convergence Rates for Entropic Transportation Costs
• The Wasserstein Distance as a Dissimilarity Measure for Mass Spectra with Application to Spectral Deconvolution
• Minimax estimation of smooth optimal transport maps
• Minimax estimation of smooth densities in Wasserstein distance
• Semi-discrete optimal transport: a solution procedure for the unsquared Euclidean distance case
• Statistical inference for Bures-Wasserstein barycenters
• The transport map computed by iterated function system
• Coupling matrix manifolds assisted optimization for optimal transport problems
• Estimation of Wasserstein distances in the spiked transport model
• Linearized Wasserstein dimensionality reduction with approximation guarantees
• CytOpT: Optimal Transport with Domain Adaptation for Interpreting Flow Cytometry data
• On quantum optimal transport
• Differentiation and regularity of semi-discrete optimal transport with respect to the parameters of the discrete measure

Describes a project that uses

Uses Software

• POT
• Caltech-256
• LMNN
• t-SNE
• DeCAF
• Pymanopt
• Manopt
• UCI

This page was built for publication: Wasserstein discriminant analysis

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1722729)