Universal Features for High-Dimensional Learning and Inference

DOI10.1561/0100000107arXiv1911.09105WikidataQ128619486 ScholiaQ128619486MaRDI QIDQ6125994

Lizhong Zheng, Gregory W. Wornell, Anuran Makur, Shao-Lun Huang

Publication date: 9 April 2024

Published in: Foundations and Trends® in Communications and Information Theory (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/1911.09105

zbMATH Keywords

clusteringstatistical signal processingdimensionality reductionspectral methodsdetection and estimationclassification and predictioninformation theory and computer sciencestatistical/machine learninginformation theory and statisticspattern recognition and learning

Mathematics Subject Classification ID

Factor analysis and principal components; correspondence analysis (62H25) Estimation in multivariate analysis (62H12) Classification and discrimination; cluster analysis (statistical aspects) (62H30) Artificial neural networks and deep learning (68T07) Numerical computation of eigenvalues and eigenvectors of matrices (65F15) Learning and adaptive systems in artificial intelligence (68T05) Computing methodologies for image processing (68U10) Signal theory (characterization, reconstruction, filtering, etc.) (94A12) Image processing (compression, reconstruction, etc.) in information and communication theory (94A08) Eigenvalues, singular values, and eigenvectors (15A18) Information theory (general) (94A15)

Cites Work

• Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions
• A well-conditioned estimator for large-dimensional covariance matrices
• Estimation of high-dimensional low-rank matrices
• Estimation of (near) low-rank matrices with noise and high-dimensional scaling
• Bernstein polynomials and learning theory
• User-friendly tail bounds for sums of random matrices
• Comparison of contraction coefficients for \(f\)-divergences
• Geometric bounds for eigenvalues of Markov chains
• Information geometry and its applications
• Nonlinear measures of association with kernel canonical correlation analysis and applications
• Maximum likelihood methods for linear and log-linear models in categorical data
• Ramanujan graphs
• A simplified neuron model as a principal component analyzer
• Approximation of density functions by sequences of exponential families
• Reduced-rank regression for the multivariate linear model
• Spreading of sets in product spaces and hypercontraction of the Markov operator
• Monotone dependence
• The Gifi system of descriptive multivariate analysis.
• A unifying information-theoretic framework for independent component analysis
• Multilayer feedforward networks are universal approximators
• Estimation of distributions using orthogonal expansions
• Principal component analysis.
• Generalized neural networks for spectral analysis: dynamics and Liapunov functions
• Support-vector networks
• Strong data-processing inequalities for channels and Bayesian networks
• Estimation of a multivariate density
• A class of measures of informativity of observation channels
• Remarks on functional canonical variates, alternating least squares methods and ACE
• Exact matrix completion via convex optimization
• Strong Data Processing Inequalities and <inline-formula> <tex-math notation="LaTeX">$\Phi $ </tex-math> </inline-formula>-Sobolev Inequalities for Discrete Channels
• Generalized Independent Component Analysis Over Finite Alphabets
• Numerical Methods for Large Eigenvalue Problems
• 10.1162/153244303768966085
• On Sequences of Pairs of Dependent Random Variables
• Remarks on Some Nonparametric Estimates of a Density Function
• Some properties of the bivariate normal distribution considered in the form of a contingency table
• The Structure of Bivariate Distributions
• On measures of dependence
• Information Theoretical Analysis of Multivariate Correlation
• The general theory of canonical correlation and its relation to functional analysis
• Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization
• Non-linear canonical correlation†
• Estimating Optimal Transformations for Multiple Regression and Correlation
• Elliptically Symmetric Distributions: A Review and Bibliography
• The common information of two dependent random variables
• Approximation theory of output statistics
• Universal approximation bounds for superpositions of a sigmoidal function
• On the Early History of the Singular Value Decomposition
• I.—PROBABILITY: THE DEDUCTIVE AND INDUCTIVE PROBLEMS
• Polynomial Singular Value Decompositions of a Family of Source-Channel Models
• Geometric diffusions as a tool for harmonic analysis and structure definition of data: Diffusion maps
• Canonical Correlation Analysis: An Overview with Application to Learning Methods
• On Choosing and Bounding Probability Metrics
• On the Sample Complexity of HGR Maximal Correlation Functions for Large Datasets
• On the Robustness of Information-Theoretic Privacy Measures and Mechanisms
• Privacy With Estimation Guarantees
• A Coordinate System for Gaussian Networks
• A Probabilistic and RIPless Theory of Compressed Sensing
• A New Data Processing Inequality and Its Applications in Distributed Source and Channel Coding
• Matrix Completion From a Few Entries
• The Power of Convex Relaxation: Near-Optimal Matrix Completion
• Probability of error of some adaptive pattern-recognition machines
• Principal Inertia Components and Applications
• Elements of Information Theory
• Appendix: On Common Information and Related Characteristics of Correlated Information Sources
• Maximum Entropy for Hypothesis Formulation, Especially for Multidimensional Contingency Tables
• New version of the probabilistic generalization of the large sieve
• Contingency tables with given marginals
• Transmission of noisy information to a noisy receiver with minimum distortion
• Information transmission with additional noise
• On the Symmetrized Kronecker Power of a Matrix and Extensions of Mehler’s Formula for Hermite Polynomials
• On Estimation of a Probability Density Function and Mode
• RELATIONS BETWEEN TWO SETS OF VARIATES
• THE POPULATION FREQUENCIES OF SPECIES AND THE ESTIMATION OF POPULATION PARAMETERS
• Information Theory
• Information Theory and Statistics: A Tutorial
• Approximation by superpositions of a sigmoidal function
• Remarks on the maximum correlation coefficient
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item