Conditional mean embedding and optimal feature selection via positive definite kernels

DOI10.7494/OPMATH.2024.44.1.79MaRDI QIDQ6091085zbMATH OpenFDO

Authors Palle Jorgensen, Myung-Sin Song, James Tian

Publication date 23 November 2023

Published in Opuscula Mathematica (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2305.08100

zbMATH Keywords

embedding problemsframesmachine learningoptimizationpositive-definite kernelsreproducing kernel Hilbert spacestochastic processes

Mathematics Subject Classification ID

Positive definite functions in one variable harmonic analysis (42A82) General harmonic expansions, frames (42C15) Linear operators and ill-posed problems, regularization (47A52) Linear operators in reproducing-kernel Hilbert spaces (including de Branges, de Branges-Rovnyak, and other structured spaces) (47B32) Applications of operator theory in optimization, convex analysis, mathematical programming, economics (47N10) Estimation in multivariate analysis (62H12) Ridge regression; shrinkage estimators (Lasso) (62J07) Numerical solutions of ill-posed problems in abstract spaces; regularization (65J20) Artificial neural networks and deep learning (68T07) Quadratic programming (90C20)

Abstract: Motivated by applications, we consider here new operator theoretic approaches to Conditional mean embeddings (CME). Our present results combine a spectral analysis-based optimization scheme with the use of kernels, stochastic processes, and constructive learning algorithms. For initially given non-linear data, we consider optimization-based feature selections. This entails the use of convex sets of positive definite (p.d.) kernels in a construction of optimal feature selection via regression algorithms from learning models. Thus, with initial inputs of training data (for a suitable learning algorithm,) each choice of p.d. kernel $K$ in turn yields a variety of Hilbert spaces and realizations of features. A novel idea here is that we shall allow an optimization over selected sets of kernels $K$ from a convex set $C$ of positive definite kernels $K$. Hence our extquotedblleft optimal extquotedblright{} choices of feature representations will depend on a secondary optimization over p.d. kernels $K$ within a specified convex set $C$.

Cites work

• A kernel-based measure for conditional mean dependence
• A rigorous theory of conditional mean embeddings
• A topological view of unsupervised learning from noisy data
• Decomposition of Gaussian processes, and factorization of positive definite kernels
• Error bounds for learning the kernel
• Geometry on probability spaces
• High-dimensional variable screening through kernel-based conditional mean dependence
• scientific article; zbMATH DE number 193012 (Why is no real title available?)
• scientific article; zbMATH DE number 7626711 (Why is no real title available?)
• Infinite-dimensional analysis. Operators in Hilbert space; stochastic calculus via representations, and duality theory
• Kernel smoothing, mean shift, and their learning theory with directional data
• Minimax Rate Optimal Adaptive Nearest Neighbor Classification and Regression
• Model-based feature selection and clustering of RNA-seq data for unsupervised subtype discovery
• Non-commutative analysis
• On multivariate discrete least squares
• Online learning algorithms
• Realizations and factorizations of positive definite kernels
• Reproducing kernels and choices of associated feature spaces, in the form of \(L^2\)-spaces
• Reproducing kernels: harmonic analysis and some of their applications
• Sampling with positive definite kernels and an associated dichotomy
• Solving a class of feature selection problems via fractional 0--1 programming
• Testing independence of functional variables by angle covariance
• Theory of Reproducing Kernels

Cited in

(1)

• Stochastic bundles, new classes of Gaussian processes and white noise-space analysis indexed by measures

This page was built for publication: Conditional mean embedding and optimal feature selection via positive definite kernels

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6091085)