Tensor Networks for Dimensionality Reduction and Large-scale Optimization: Part 2 Applications and Future Perspectives

DOI10.1561/2200000067zbMATH Open1376.68123arXiv1708.09165OpenAlexW2617994470MaRDI QIDQ4593746FDOQ4593746

Authors: Anh-Huy Phan, Qibin Zhao, Namgil Lee, Ivan Oseledets, Masashi Sugiyama, Andrzej Cichocki, Danilo P. Mandic 

Publication date: 22 November 2017

Published in: Foundations and Trends® in Machine Learning (Search for Journal in Brave)

Abstract: Part 2 of this monograph builds on the introduction to tensor networks and their operations presented in Part 1. It focuses on tensor network models for super-compressed higher-order representation of data/parameters and related cost functions, while providing an outline of their applications in machine learning and data analytics. A particular emphasis is on the tensor train (TT) and Hierarchical Tucker (HT) decompositions, and their physically meaningful interpretations which reflect the scalability of the tensor network approach. Through a graphical approach, we also elucidate how, by virtue of the underlying low-rank tensor approximations and sophisticated contractions of core tensors, tensor networks have the ability to perform distributed computations on otherwise prohibitively large volumes of data/parameters, thereby alleviating or even eliminating the curse of dimensionality. The usefulness of this concept is illustrated over a number of applied areas, including generalized regression and classification (support tensor machines, canonical correlation analysis, higher order partial least squares), generalized eigenvalue decomposition, Riemannian optimization, and in the optimization of deep neural networks. Part 1 and Part 2 of this work can be used either as stand-alone separate texts, or indeed as a conjoint comprehensive review of the exciting field of low-rank tensor networks and tensor decompositions.

Full work available at URL: https://arxiv.org/abs/1708.09165

Recommendations

• Tensor networks for dimensionality reduction and large-scale optimization. I: Low-rank tensor decompositions
• Tensor networks in machine learning
• Tensor networks and hierarchical tensors for the solution of high-dimensional partial differential equations
• Tensor neural network models for tensor singular value decompositions
• scientific article; zbMATH DE number 5302704
• On the optimization landscape of tensor decompositions
• Learning with tensors: a framework based on convex optimization and spectral regularization
• Tensor Methods for Large, Sparse Unconstrained Optimization
• Tensor principal component analysis via convex optimization

zbMATH Keywords

dimensionality reductiontensor decompositionlarge-scale optimizationtensor networks

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05) Research exposition (monographs, survey articles) pertaining to computer science (68-02)

Cited In (26)

• Towards compact neural networks via end-to-end training: a Bayesian tensor approach with automatic rank determination
• An efficient algorithm for computing the approximate t-URV and its applications
• Perturbations of the \textsc{Tcur} decomposition for tensor valued data in the Tucker format
• A randomized algorithm for tensor singular value decomposition using an arbitrary number of passes
• A Low-Rank Matrix Equation Method for Solving PDE-Constrained Optimization Problems
• Gauging tensor networks with belief propagation
• Computing low-rank approximations of large-scale matrices with the tensor network randomized SVD
• Approximation theory of tree tensor networks: tensorized univariate functions
• MERACLE: constructive layer-wise conversion of a tensor train into a MERA
• Tensor networks in machine learning
• Geometric Methods on Low-Rank Matrix and Tensor Manifolds
• Hybrid tensor decomposition in neural network compression
• Black Box Approximation in the Tensor Train Format Initialized by ANOVA Decomposition
• Fundamental tensor operations for large-scale data analysis using tensor network formats
• Optimization problems in contracted tensor networks
• Low-rank nonnegative tensor approximation via alternating projections and sketching
• Correlation length in random MPS and PEPS
• An efficient randomized fixed-precision algorithm for tensor singular value decomposition
• Tensor networks for dimensionality reduction and large-scale optimization. I: Low-rank tensor decompositions
• Geometry of tree-based tensor formats in tensor Banach spaces
• Learning with tree tensor networks: complexity estimates and model selection
• Alternating Linear Scheme in a Bayesian Framework for Low-Rank Tensor Approximation
• Frobenius norm inequalities of commutators based on different products
• Title not available (Why is that?)
• Time-homogeneous top-\(K\) ranking using tensor decompositions
• A TT-based hierarchical framework for decomposing high-order tensors

This page was built for publication: Tensor Networks for Dimensionality Reduction and Large-scale Optimization: Part 2 Applications and Future Perspectives

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4593746)