Online regularized learning algorithm for functional data

DOI10.1016/J.JCO.2024.101825MaRDI QIDQ6193950zbMATH OpenOpenAlexFDO

Authors Yuan Mao, Zheng-Chu Guo

Publication date 19 March 2024

Published in Journal of Complexity (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2211.13549

zbMATH Keywords

regularizationfunctional linear modelreproducing kernel Hilbert spacesonline learning

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05) Computational aspects of data analysis and big data (68T09) Computational learning theory (68Q32) Hilbert spaces with reproducing kernels (= (proper) functional Hilbert spaces, including de Branges-Rovnyak and other structured spaces) (46E22)

Abstract: In recent years, functional linear models have attracted growing attention in statistics and machine learning, with the aim of recovering the slope function or its functional predictor. This paper considers online regularized learning algorithm for functional linear models in reproducing kernel Hilbert spaces. Convergence analysis of excess prediction error and estimation error are provided with polynomially decaying step-size and constant step-size, respectively. Fast convergence rates can be derived via a capacity dependent analysis. By introducing an explicit regularization term, we uplift the saturation boundary of unregularized online learning algorithms when the step-size decays polynomially, and establish fast convergence rates of estimation error without capacity assumption. However, it remains an open problem to obtain capacity independent convergence rates for the estimation error of the unregularized online learning algorithm with decaying step-size. It also shows that convergence rates of both prediction error and estimation error with constant step-size are competitive with those in the literature.

Recommendations

• Capacity dependent analysis for functional online learning algorithms
• Learning rates of regularized regression for functional data
• Fast and strong convergence of online learning algorithms
• Convergence analysis of online algorithms
• Online Regularized Classification Algorithms

Cites work

• A reproducing kernel Hilbert space approach to functional linear regression
• An RKHS approach to estimate individualized treatment rules based on functional predictors
• Distributed least squares prediction for functional linear regression*
• Fast and strong convergence of online learning algorithms
• Functional data analysis.
• Learning Theory
• Learning theory of distributed spectral algorithms
• Methodology and convergence rates for functional linear regression
• Minimax and adaptive prediction for functional linear regression
• Nonparametric stochastic approximation with large step-sizes
• On regularization algorithms in learning theory
• Online Learning as Stochastic Approximation of Regularization Paths: Optimality and Almost-Sure Convergence
• Online Regularized Classification Algorithms
• Online gradient descent algorithms for functional data learning
• Online gradient descent learning algorithms
• Online regularized learning with pairwise loss functions
• Optimal rates for coefficient-based regularized regression
• Optimal rates for multi-pass stochastic gradient methods
• Optimal rates for the regularized least-squares algorithm
• Prediction in functional linear regression
• Rates of convergence of randomized Kaczmarz algorithms in Hilbert spaces
• Support Vector Machines

Cited in

(4)

• On the convergence of gradient descent for robust functional linear regression
• Nyström subsampling for functional linear regression
• Spectral algorithms for functional linear regression
• Capacity dependent analysis for functional online learning algorithms

This page was built for publication: Online regularized learning algorithm for functional data

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6193950)