Transfer learning for nonparametric classification: minimax rate and adaptive classifier

DOI10.1214/20-AOS1949zbMATH Open1466.62351arXiv1906.02903MaRDI QIDQ2656589FDOQ2656589

Authors: Yanyan Li 

Publication date: 11 March 2021

Published in: The Annals of Statistics (Search for Journal in Brave)

Abstract: Human learners have the natural ability to use knowledge gained in one setting for learning in a different but related setting. This ability to transfer knowledge from one task to another is essential for effective learning. In this paper, we study transfer learning in the context of nonparametric classification based on observations from different distributions under the posterior drift model, which is a general framework and arises in many practical problems. We first establish the minimax rate of convergence and construct a rate-optimal two-sample weighted $K$-NN classifier. The results characterize precisely the contribution of the observations from the source distribution to the classification task under the target distribution. A data-driven adaptive classifier is then proposed and is shown to simultaneously attain within a logarithmic factor of the optimal rate over a large collection of parameter spaces. Simulation studies and real data applications are carried out where the numerical results further illustrate the theoretical analysis. Extensions to the case of multiple source distributions are also considered.

Full work available at URL: https://arxiv.org/abs/1906.02903

Recommendations

• Adaptive transfer learning
• TNN: a transfer learning classifier based on weighted nearest neighbors
• Marginal singularity and the benefits of labels in covariate-shift
• Minimax nonparametric classification. II. Model selection for adaptation
• Optimal aggregation of classifiers in statistical learning.

zbMATH Keywords

classificationtransfer learningminimax rateadaptivitydomain adaptation

Mathematics Subject Classification ID

Nonparametric hypothesis testing (62G10) Classification and discrimination; cluster analysis (statistical aspects) (62H30) Applications of statistics to social sciences (62P25) Minimax procedures in statistical decision theory (62C20)

Cites Work

• Nearest neighbor pattern classification
• Classification in general finite dimensional spaces with the \(k\)-nearest neighbor rule
• Optimal weighted nearest neighbour classifiers
• Improving predictive inference under covariate shift by weighting the log-likelihood function
• A survey on concept drift adaptation
• Asymptotically Minimax Adaptive Estimation. I: Upper Bounds. Optimally Adaptive Estimates
• Optimal aggregation of classifiers in statistical learning.
• Fast learning rates for plug-in classifiers
• On the rate of convergence of nearest neighbor rules (Corresp.)
• On a Problem of Adaptive Estimation in Gaussian White Noise
• Optimal pointwise adaptive methods in nonparametric estimation
• Asymptotically Minimax Adaptive Estimation. II. Schemes without Optimal Adaptation: Adaptive Estimators
• Title not available (Why is that?)
• Title not available (Why is that?)

Cited In (24)

• Targeting underrepresented populations in precision medicine: a federated transfer learning approach
• Local convergence rates of the nonparametric least squares estimator with applications to transfer learning
• Adaptive transfer learning
• A no-free-lunch theorem for multitask learning
• Deep neural network classifier for multidimensional functional data
• Turning the information-sharing dial: efficient inference from different data sources
• Distributed adaptive nearest neighbor classifier: algorithm and theory
• Nonparametric classification with missing data
• Statistical learning from biased training samples
• Weighted likelihood transfer learning for high-dimensional generalized linear models
• Transfer learning for contextual multi-armed bandits
• Efficient and multiply robust risk estimation under general forms of dataset shift
• Tnn: a transfer learning classifier based on weighted nearest neighbors
• Transfer learning via random forests: a one-shot federated approach
• Double debiased transfer learning for adaptive Huber regression
• Optimally tackling covariate shift in RKHS-based nonparametric regression
• Marginal singularity and the benefits of labels in covariate-shift
• Transfer Learning under High-dimensional Generalized Linear Models
• The art of transfer learning: an adaptive and robust pipeline
• Transfer learning for functional mean estimation: phase transition and adaptive algorithms
• Robust transfer learning of high-dimensional generalized linear model
• Estimation and Inference for High-Dimensional Generalized Linear Models with Knowledge Transfer
• Accommodating Time-Varying Heterogeneity in Risk Estimation under the Cox Model: A Transfer Learning Approach
• Transfer Learning in Large-Scale Gaussian Graphical Models with False Discovery Rate Control

Uses Software

• OpenStreetMap

This page was built for publication: Transfer learning for nonparametric classification: minimax rate and adaptive classifier

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2656589)