Multiview Alignment and Generation in CCA via Consistent Latent Encoding

DOI10.1162/NECO_A_01309zbMATH Open1455.62115arXiv2005.11716OpenAlexW3049425243WikidataQ98469095 ScholiaQ98469095MaRDI QIDQ3386409FDOQ3386409

Donna Xu, Yaxin Shi, Yuangang Pan, Ivor W. Tsang

Publication date: 4 January 2021

Published in: Neural Computation (Search for Journal in Brave)

Abstract: Multi-view alignment, achieving one-to-one correspondence of multi-view inputs, is critical in many real-world multi-view applications, especially for cross-view data analysis problems. Recently, an increasing number of works study this alignment problem with Canonical Correlation Analysis (CCA). However, existing CCA models are prone to misalign the multiple views due to either the neglect of uncertainty or the inconsistent encoding of the multiple views. To tackle these two issues, this paper studies multi-view alignment from the Bayesian perspective. Delving into the impairments of inconsistent encodings, we propose to recover correspondence of the multi-view inputs by matching the marginalization of the joint distribution of multi-view random variables under different forms of factorization. To realize our design, we present Adversarial CCA (ACCA) which achieves consistent latent encodings by matching the marginalized latent encodings through the adversarial training paradigm. Our analysis based on conditional mutual information reveals that ACCA is flexible for handling implicit distributions. Extensive experiments on correlation analysis and cross-view generation under noisy input settings demonstrate the superiority of our model.

Full work available at URL: https://arxiv.org/abs/2005.11716

Recommendations

• An efficient dictionary-based multi-view learning method
• Grassmannian Regularized Structured Multi-View Embedding for Image Classification

zbMATH Keywords

canonical correlation analysis (CCA)

Mathematics Subject Classification ID

Statistical aspects of big data and data science (62R07) Measures of association (correlation, canonical correlation, etc.) (62H20)

Cites Work

• Algorithmic Learning Theory
• Title not available (Why is that?)
• Title not available (Why is that?)
• RELATIONS BETWEEN TWO SETS OF VARIATES
• Domain-adversarial training of neural networks
• Probabilistic Principal Component Analysis
• Lectures on algebraic statistics
• Advanced Lectures on Machine Learning
• Cross-Modal Localization via Sparsity

Uses Software

• CycleGAN
• pix2pix
• U-Net
• MGAN

This page was built for publication: Multiview Alignment and Generation in CCA via Consistent Latent Encoding

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3386409)