Active Learning of Tree Tensor Networks using Optimal Least Squares

DOI10.1137/21M1415911MaRDI QIDQ6177928zbMATH OpenFDO

Authors Cécile Haberstich, A. Nouy, G. Perrin

Publication date 31 August 2023

Published in SIAM/ASA Journal on Uncertainty Quantification (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2104.13436

zbMATH Keywords

active learning with weighted least squaresadaptive strategieshigh-dimensional approximationprincipal component analysistree-based tensor formats

Mathematics Subject Classification ID

Numerical analysis (65-XX) Artificial intelligence (68Txx)

Abstract: In this paper, we propose new learning algorithms for approximating high-dimensional functions using tree tensor networks in a least-squares setting. Given a dimension tree or architecture of the tensor network, we provide an algorithm that generates a sequence of nested tensor subspaces based on a generalization of principal component analysis for multivariate functions. An optimal least-squares method is used for computing projections onto the generated tensor subspaces, using samples generated from a distribution depending on the previously generated subspaces. We provide an error bound in expectation for the obtained approximation. Practical strategies are proposed for adapting the feature spaces and ranks to achieve a prescribed error. Also, we propose an algorithm that progressively constructs the dimension tree by suitable pairings of variables, that allows to further reduce the number of samples necessary to reach that error. Numerical examples illustrate the performance of the proposed algorithms and show that stable approximations are obtained with a number of samples close to the number of free parameters of the estimated tensor networks.

Cites work

• A continuous analogue of the tensor-train decomposition
• A general multipurpose interpolation procedure: The magic points
• A Multilinear Singular Value Decomposition
• A new method for reconstruction of cross-sections using Tucker decomposition
• Analysis of tensor approximation schemes for continuous functions
• Approximation of boundary element matrices
• Approximation rates for the hierarchical tensor format in periodic Sobolev spaces
• Approximation theory of tree tensor networks: tensorized univariate functions
• Black box approximation of tensors in hierarchical Tucker format
• Boosted optimal weighted least-squares
• Hierarchical Singular Value Decomposition of Tensors
• High-probability bounds for the reconstruction error of PCA
• Higher-order principal component analysis for the approximation of tensors in tree-based low-rank formats
• scientific article; zbMATH DE number 3954145 (Why is no real title available?)
• LEARNING HIGH-DIMENSIONAL PROBABILITY DISTRIBUTIONS USING TREE TENSOR NETWORKS
• Learning with tree tensor networks: complexity estimates and model selection
• Low-rank tensor methods with subspace correction for symmetric eigenvalue problems
• Spectral tensor-train decomposition
• Tensor networks and hierarchical tensors for the solution of high-dimensional partial differential equations
• Tensor networks for dimensionality reduction and large-scale optimization. I: Low-rank tensor decompositions
• Tensor Spaces and Numerical Tensor Calculus
• Tree adaptive approximation in the hierarchical tensor format
• Tree-based tensor formats
• TT-cross approximation for multidimensional arrays

Cited in

(2)

• Tensor decision trees for continual learning from drifting data streams
• Approximation in the extended functional tensor train format

This page was built for publication: Active Learning of Tree Tensor Networks using Optimal Least Squares

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6177928)