Robust Kernel-based Distribution Regression

DOI10.1088/1361-6420/AC23C3MaRDI QIDQ6365835arXivFDO

Authors Zhan Yu, Daniel W. C. Ho, Zhongjie Shi, Ding-Xuan Zhou

Publication date 21 April 2021

Published in arXiv

Full work available at URL https://doi.org/10.1088/1361-6420/AC23C3

Mathematics Subject Classification ID

General nonlinear regression (62J02) Learning and adaptive systems in artificial intelligence (68T05) Ridge regression; shrinkage estimators (Lasso) (62J07)

arXiv classification

cs.LGstat.MLmath.FA

Abstract: Regularization schemes for regression have been widely studied in learning theory and inverse problems. In this paper, we study distribution regression (DR) which involves two stages of sampling, and aims at regressing from probability measures to real-valued responses over a reproducing kernel Hilbert space (RKHS). Recently, theoretical analysis on DR has been carried out via kernel ridge regression and several learning behaviors have been observed. However, the topic has not been explored and understood beyond the least square based DR. By introducing a robust loss function $l_{sigma}$ for two-stage sampling problems, we present a novel robust distribution regression (RDR) scheme. With a windowing function $V$ and a scaling parameter $sigma$ which can be appropriately chosen, $l_{sigma}$ can include a wide range of popular used loss functions that enrich the theme of DR. Moreover, the loss $l_{sigma}$ is not necessarily convex, hence largely improving the former regression class (least square) in the literature of DR. The learning rates under different regularity ranges of the regression function $f_{ho}$ are comprehensively studied and derived via integral operator techniques. The scaling parameter $sigma$ is shown to be crucial in providing robustness and satisfactory learning rates of RDR.

This page was built for publication: Robust Kernel-based Distribution Regression

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6365835)