Interpolation consistency training for semi-supervised learning

DOI10.1016/J.NEUNET.2021.10.008MaRDI QIDQ6055165zbMATH OpenFDO

Authors Vikas Verma, Kenji Kawaguchi, A. Lamb, Juho Kannala, Arno Solin, Yoshua Bengio, David Lopez-Paz

Publication date 28 September 2023

Published in Neural Networks (Search for Journal in Brave)

Copyright license Creative Commons Attribution 4.0 International

Full work available at URL https://arxiv.org/abs/1903.03825

zbMATH Keywords

consistency regularizationdeep neural networksmixupsemi-supervised learning

Mathematics Subject Classification ID

Artificial neural networks and deep learning (68T07)

Abstract: We introduce Interpolation Consistency Training (ICT), a simple and computation efficient algorithm for training Deep Neural Networks in the semi-supervised learning paradigm. ICT encourages the prediction at an interpolation of unlabeled points to be consistent with the interpolation of the predictions at those points. In classification problems, ICT moves the decision boundary to low-density regions of the data distribution. Our experiments show that ICT achieves state-of-the-art performance when applied to standard neural network architectures on the CIFAR-10 and SVHN benchmark datasets. Our theoretical analysis shows that ICT corresponds to a certain type of data-adaptive regularization with unlabeled points which reduces overfitting to labeled points under high confidence values.

Recommendations

• Semi-supervised interpolation in an anticausal learning scenario
• On semi-supervised learning
• Semi-supervised learning using multiple one-dimensional embedding based adaptive interpolation
• Unbiased generative semi-supervised learning
• Label propagation via local geometry preserving for deep semi-supervised image recognition

Cites work

• 10.1162/153244303321897690
• Foundations of machine learning

Cited in

(7)

• Semi-supervised learning using multiple one-dimensional embedding based adaptive interpolation
• Graph interpolating activation improves both natural and robust accuracies in data-efficient deep learning
• Semi-supervised interpolation in an anticausal learning scenario
• Manifold adversarial training for supervised and semi-supervised learning
• A holistic global-local stochastic configuration network modeling framework with antinoise awareness for efficient semi-supervised regression
• Diverse and consistent multi-view networks for semi-supervised regression
• A review of pseudo-labeling for computer vision

This page was built for publication: Interpolation consistency training for semi-supervised learning

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6055165)