Fast rates by transferring from auxiliary hypotheses

DOI10.1007/S10994-016-5594-4MaRDI QIDQ2361574zbMATH OpenOpenAlexFDO

Authors Ilja Kuzborskij, Francesco Orabona

Publication date 30 June 2017

Published in Machine Learning (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1412.1619

zbMATH Keywords

domain adaptationfast-rate generalization boundsRademacher complexitysmooth loss functionsstrongly-convex regularizerstransfer learning

Mathematics Subject Classification ID

General nonlinear regression (62J02) Learning and adaptive systems in artificial intelligence (68T05)

Abstract: In this work we consider the learning setting where, in addition to the training set, the learner receives a collection of auxiliary hypotheses originating from other tasks. We focus on a broad class of ERM-based linear algorithms that can be instantiated with any non-negative smooth loss function and any strongly convex regularizer. We establish generalization and excess risk bounds, showing that, if the algorithm is fed with a good combination of source hypotheses, generalization happens at the fast rate $mathcalO(1/m)$ instead of the usual $mathcalO(1/sqrtm)$. On the other hand, if the source hypotheses combination is a misfit for the target task, we recover the usual learning rate. As a byproduct of our study, we also prove a new bound on the Rademacher complexity of the smooth loss class under weaker assumptions compared to previous works.

Recommendations

• Fast rates in statistical and online learning
• Fast learning rates in statistical inference through aggregation
• Fast learning from \(\alpha\)-mixing observations
• FAST RATES FOR ESTIMATION ERROR AND ORACLE INEQUALITIES FOR MODEL SELECTION
• Fast rates for empirical vector quantization
• Fast classification rates without standard margin assumptions
• scientific article; zbMATH DE number 2089364
• Learning Theory
• Fast Bayesian approach for parameter estimation
• The speed of sequential asymptotic learning

Cites work

• 10.1162/153244302760200704
• 10.1162/153244303321897690
• A new learning paradigm: learning using privileged information
• A theory of learning from different domains
• Domain adaptation and sample bias correction theory and algorithm for regression
• Foundations of machine learning
• scientific article; zbMATH DE number 1804115 (Why is no real title available?)
• scientific article; zbMATH DE number 1113195 (Why is no real title available?)
• Inequalities on the Lambert W function and hyperpower function
• Local Rademacher complexities
• Model selection for regularized least-squares algorithm in learning theory
• On the Hardness of Domain Adaptation and the Utility of Unlabeled Target Samples
• Regularization techniques for learning with matrices
• Transfer learning for reinforcement learning domains: a survey
• Understanding machine learning. From theory to algorithms

Cited in

(5)

• Handling concept drift via model reuse
• Risk bound of transfer learning using parametric feature mapping and its application to sparse coding
• scientific article; zbMATH DE number 7750672 (Why is no real title available?)
• Learning from similar linear representations: adaptivity, minimaxity, and robustness
• Source-free domain adaptation requires penalized diversity

This page was built for publication: Fast rates by transferring from auxiliary hypotheses

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2361574)