Adaptive transfer learning
From MaRDI portal
Abstract: In transfer learning, we wish to make inference about a target population when we have access to data both from the distribution itself, and from a different but related source distribution. We introduce a flexible framework for transfer learning in the context of binary classification, allowing for covariate-dependent relationships between the source and target distributions that are not required to preserve the Bayes decision boundary. Our main contributions are to derive the minimax optimal rates of convergence (up to poly-logarithmic factors) in this problem, and show that the optimal rate can be achieved by an algorithm that adapts to key aspects of the unknown transfer relationship, as well as the smoothness and tail parameters of our distributional classes. This optimal rate turns out to have several regimes, depending on the interplay between the relative sample sizes and the strength of the transfer relationship, and our algorithm achieves optimality by careful, decision tree-based calibration of local nearest-neighbour procedures.
Recommendations
Cites work
- A theory of learning from different domains
- Adaptation based on generalized discrepancy
- Cellular tree classifiers
- Choice of neighbor order in nearest-neighbor classification
- Classification in general finite dimensional spaces with the \(k\)-nearest neighbor rule
- Classification with asymmetric label noise: consistency and maximal denoising
- Classification with imperfect training labels
- Consistent nonparametric regression. Discussion
- Density ratio estimation in machine learning. Foreword by Thomas G. Dietterich
- Domain generalization by marginal transfer learning
- Fast learning rates for plug-in classifiers
- scientific article; zbMATH DE number 3860199 (Why is no real title available?)
- scientific article; zbMATH DE number 1246230 (Why is no real title available?)
- scientific article; zbMATH DE number 6999890 (Why is no real title available?)
- Introduction to nonparametric estimation
- Learning from binary labels with instance-dependent noise
- Lectures on the nearest neighbor method
- Local nearest neighbour classification with applications to semi-supervised learning
- Measure theory and fine properties of functions
- Measuring mass concentrations and estimating density contour clusters -- An excess mass approach
- Minimax-optimal classification with dyadic decision trees
- Nearest neighbor pattern classification
- New analysis and algorithm for learning with drifting distributions
- Obtaining minimax lower bounds: a review
- Optimal aggregation of classifiers in statistical learning.
- Optimal weighted nearest neighbour classifiers
- Rates of convergence of nearest neighbor estimation under arbitrary sampling
- Risk bounds for the majority vote: from a PAC-Bayesian analysis to a learning algorithm
- Smooth discrimination analysis
- The benefit of multitask representation learning
- The metric entropy of diffeomorphisms. I: Characterization of measures satisfying Pesin's entropy formula
- Transfer learning for nonparametric classification: minimax rate and adaptive classifier
Cited in
(45)- On universal transfer learning
- Transfer Learning under High-dimensional Generalized Linear Models
- TNN: a transfer learning classifier based on weighted nearest neighbors
- Statistical learning from biased training samples
- A no-free-lunch theorem for multitask learning
- Binary classification with corrupted labels
- Improved linear regression prediction by transfer learning
- Transfer learning for nonparametric classification: minimax rate and adaptive classifier
- Transfer of semiparametric single index model in binary classification
- On Universal Transfer Learning
- Online transfer learning
- Optimal Bayesian Transfer Learning
- Matching component analysis for transfer learning
- Robust transfer learning of high-dimensional generalized linear model
- Some thoughts about transfer learning. What role for the source domain?
- Optimally tackling covariate shift in RKHS-based nonparametric regression
- Optimal subgroup selection
- Transfer learning for contextual multi-armed bandits
- Ensemble of Adapters for Transfer Learning Based on Evidence Theory
- Transfer learning for functional mean estimation: phase transition and adaptive algorithms
- A fast trans-lasso algorithm with penalized weighted score function
- Local convergence rates of the nonparametric least squares estimator with applications to transfer learning
- Reprint of: Some thoughts about transfer learning. What role for the source domain?
- Turning the information-sharing dial: efficient inference from different data sources
- Nonparametric classification with missing data
- Weighted likelihood transfer learning for high-dimensional generalized linear models
- Double debiased transfer learning for adaptive Huber regression
- Pseudo-labeling for kernel ridge regression under covariate shift
- Transfer learning for high-dimensional transelliptical graphical models
- Transfer learning for high-dimensional data with heavy-tailed noise: a sparse convoluted rank regression method
- Robust transfer learning for high-dimensional GLM using -divergence with applications to cancer genomics
- An adaptive transfer learning perspective on classification in nonstationary environments
- Editorial: Special issue on ``Learning across distribution shifts
- Robust transfer learning with unreliable source data
- Transfer learning for piecewise-constant mean estimation: optimality, _1 and _0 penalization
- Being Bayesian in the 2020s: opportunities and challenges in the practice of modern applied Bayesian statistics
- Group transfer learning for high-dimensional linear regression
- Supervised learning with evolving tasks and performance guarantees
- A joint estimation approach for monotonic regression functions in general dimensions
- Debiased transfer learning estimation and inference for multinomial regression
- Efficient risk assessment of time-to-event targets with adaptive information transfer
- Regularization using synthetic data for high-dimensional inference
- Minimax and adaptive transfer learning for nonparametric classification under distributed differential privacy constraints
- Recent developments on statistical transfer learning
- Estimation beyond missing (completely) at random
This page was built for publication: Adaptive transfer learning
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2073726)
