Supervised learning with probabilistic morphisms and kernel mean embeddings

MaRDI QIDQ6436166zbMATH OpenFDO

Authors Hông Vân Lê

Publication date 10 May 2023

Mathematics Subject Classification ID

Convergence of probability measures (60B10) Nonparametric estimation (62G05) Applications of functional analysis in probability theory and statistics (46N30) Higher categories and homotopical algebra (18N99)

Abstract: In this paper I propose a concept of a correct loss function in a generative model of supervised learning for an input space $mathcalX$ and a label space $mathcalY$, both of which are measurable spaces. A correct loss function in a generative model of supervised learning must accurately measure the discrepancy between elements of a hypothesis space $mathcalH$ of possible predictors and the supervisor operator, even when the supervisor operator does not belong to $mathcalH$. To define correct loss functions, I propose a characterization of a regular conditional probability measure $m{u}_{mathcalY|mathcalX}$ for a probability measure $mu$ on $mathcalXimesmathcalY$ relative to the projection $P{i}_{mathcalX}:mathcalXimesmathcalYomathcalX$ as a solution of a linear operator equation. If $mathcalY$ is a separable metrizable topological space with the Borel $sigma$-algebra $mathcalB(mathcalY)$, I propose an additional characterization of a regular conditional probability measure $m{u}_{mathcalY|mathcalX}$ as a minimizer of mean square error on the space of Markov kernels, referred to as probabilistic morphisms, from $mathcalX$ to $mathcalY$. This characterization utilizes kernel mean embeddings. Building upon these results and employing inner measure to quantify the generalizability of a learning algorithm, I extend a result due to Cucker-Smale, which addresses the learnability of a regression model, to the setting of a conditional probability estimation problem. Additionally, I present a variant of Vapnik's regularization method for solving stochastic ill-posed problems, incorporating inner measure, and showcase its applications.

This page was built for publication: Supervised learning with probabilistic morphisms and kernel mean embeddings

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6436166)