Self-supervised Metric Learning in Multi-View Data: A Downstream Task Perspective
From MaRDI portal
Abstract: Self-supervised metric learning has been a successful approach for learning a distance from an unlabeled dataset. The resulting distance is broadly useful for improving various distance-based downstream tasks, even when no information from downstream tasks is utilized in the metric learning stage. To gain insights into this approach, we develop a statistical framework to theoretically study how self-supervised metric learning can benefit downstream tasks in the context of multi-view data. Under this framework, we show that the target distance of metric learning satisfies several desired properties for the downstream tasks. On the other hand, our investigation suggests the target distance can be further improved by moderating each direction's weights. In addition, our analysis precisely characterizes the improvement by self-supervised metric learning on four commonly used downstream tasks: sample identification, two-sample testing, -means clustering, and -nearest neighbor classification. When the distance is estimated from an unlabeled dataset, we establish the upper bound on distance estimation's accuracy and the number of samples sufficient for downstream task improvement. Finally, numerical experiments are presented to support the theoretical results in the paper.
Cites work
- A kernel two-sample test
- A new test for multivariate normality
- A two-sample test for high-dimensional data with applications to gene-set testing
- Central limit theorem for integrated square error of multivariate nonparametric density estimators
- Constructing summary statistics for approximate Bayesian computation: semi-automatic approximate Bayesian computation. With discussion and authors' reply
- Discriminatory Analysis. Nonparametric Discrimination: Consistency Properties
- Distance metric learning for large margin nearest neighbor classification
- Equivalence of distance-based and RKHS-based statistics in hypothesis testing
- Fast generalization rates for distance metric learning. Improved theoretical analysis for smooth strongly convex distance metric learning
- Fast learning rates for plug-in classifiers
- Generalization bounds for metric and similarity learning
- Hypothesis testing for phylogenetic composition: a minimum-cost flow perspective
- Large sample covariance matrices and high-dimensional data analysis
- Large scale online learning of image similarity through ranking
- Lectures on the nearest neighbor method
- Metric Learning
- Metric learning: a survey
- Multivariate generalizations of the Wald-Wolfowitz and Smirnov two-sample tests
- On the optimality of sliced inverse regression in high dimensions
- Optimal weighted nearest neighbour classifiers
- Query-Augmented Active Metric Learning
- Sliced Inverse Regression for Dimension Reduction
- Spectral Methods for Data Science: A Statistical Perspective
- Statistical foundations of data science
- Sufficient dimension reduction and prediction in regression
Cited in
(2)
This page was built for publication: Self-supervised Metric Learning in Multi-View Data: A Downstream Task Perspective
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6144761)
