Robust unsupervised domain adaptation for neural networks via moment alignment

DOI10.1016/J.INS.2019.01.025zbMATH Open1448.68383arXiv1711.06114OpenAlexW2770501762MaRDI QIDQ2212554FDOQ2212554

Authors: Werner Zellinger, Bernhard A. Moser, Thomas Grubinger, Edwin Lughofer, Thomas Natschläger, Susanne Saminger-Platz 

Publication date: 24 November 2020

Published in: Information Sciences (Search for Journal in Brave)

Abstract: A novel approach for unsupervised domain adaptation for neural networks is proposed. It relies on metric-based regularization of the learning process. The metric-based regularization aims at domain-invariant latent feature representations by means of maximizing the similarity between domain-specific activation distributions. The proposed metric results from modifying an integral probability metric such that it becomes less translation-sensitive on a polynomial function space. The metric has an intuitive interpretation in the dual space as the sum of differences of higher order central moments of the corresponding activation distributions. Under appropriate assumptions on the input distributions, error minimization is proven for the continuous case. As demonstrated by an analysis of standard benchmark experiments for sentiment analysis, object recognition and digit recognition, the outlined approach is robust regarding parameter changes and achieves higher classification accuracies than comparable approaches. The source code is available at https://github.com/wzell/mann.

Full work available at URL: https://arxiv.org/abs/1711.06114

Recommendations

• Domain-adversarial training of neural networks
• Distribution-matching embedding for visual domain adaptation
• Learning kernels for unsupervised domain adaptation with applications to visual object recognition
• Asymmetric and category invariant feature transformations for domain adaptation
• Learning smooth representations with generalized softmax for unsupervised domain adaptation

zbMATH Keywords

transfer learningneural networksdomain adaptationmoment distanceintegral probability metric

Mathematics Subject Classification ID

Classification and discrimination; cluster analysis (statistical aspects) (62H30) Learning and adaptive systems in artificial intelligence (68T05) Pattern recognition, speech recognition (68T10)

Cites Work

• Adaptive subgradient methods for online learning and stochastic optimization
• Integral Probability Metrics and Their Generating Classes of Functions
• Real Analysis and Probability
• Bounds on the Expectation of a Convex Function of a Multivariate Random Variable
• The methods of distances in the theory of probability and statistics.
• Title not available (Why is that?)
• A theory of learning from different domains
• Domain adaptation and sample bias correction theory and algorithm for regression
• MMD GAN
• Title not available (Why is that?)

Cited In (12)

• Convergent Newton method and neural network for the electric energy usage prediction
• Finding robust transfer features for unsupervised domain adaptation
• Unsupervised domain adaptation in the wild via disentangling representation learning
• A deep multi-task representation learning method for time series classification and retrieval
• Domain-adversarial training of neural networks
• On generalization in moment-based domain adaptation
• Learning smooth representations with generalized softmax for unsupervised domain adaptation
• Asymmetric and category invariant feature transformations for domain adaptation
• Distribution-matching embedding for visual domain adaptation
• Subdomain adaptation capsule network for unsupervised mechanical fault diagnosis
• A two-branch symmetric domain adaptation neural network based on Ulam stability theory
• MapFlow: latent transition via normalizing flow for unsupervised domain adaptation

Uses Software

• ImageNet
• AdaGrad
• Adam
• i-RevNet
• AlexNet
• MMD GAN

This page was built for publication: Robust unsupervised domain adaptation for neural networks via moment alignment

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2212554)