Online Nonlinear Estimation via Iterative <inline-formula> <tex-math notation="LaTeX">L²</tex-math> </inline-formula>-Space Projections: Reproducing Kernel of Subspace

DOI10.1109/TSP.2018.2846271MaRDI QIDQ4622301zbMATH OpenFDO

Authors Motoya Ohnishi, Masahiro Yukawa

Publication date 12 February 2019

Published in IEEE Transactions on Signal Processing (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1712.04573

Mathematics Subject Classification ID

Nonparametric estimation (62G05) Learning and adaptive systems in artificial intelligence (68T05) Signal theory (characterization, reconstruction, filtering, etc.) (94A12)

Abstract: We propose a novel online learning paradigm for nonlinear-function estimation tasks based on the iterative projections in the L2 space with probability measure reflecting the stochastic property of input signals. The proposed learning algorithm exploits the reproducing kernel of the so-called dictionary subspace, based on the fact that any finite-dimensional space of functions has a reproducing kernel characterized by the Gram matrix. The L2-space geometry provides the best decorrelation property in principle. The proposed learning paradigm is significantly different from the conventional kernel-based learning paradigm in two senses: (i) the whole space is not a reproducing kernel Hilbert space and (ii) the minimum mean squared error estimator gives the best approximation of the desired nonlinear function in the dictionary subspace. It preserves efficiency in computing the inner product as well as in updating the Gram matrix when the dictionary grows. Monotone approximation, asymptotic optimality, and convergence of the proposed algorithm are analyzed based on the variable-metric version of adaptive projected subgradient method. Numerical examples show the efficacy of the proposed algorithm for real data over a variety of methods including the extended Kalman filter and many batch machine-learning methods such as the multilayer perceptron.

Recommendations

• An Online Projection Estimator for Nonparametric Regression in Reproducing Kernel Hilbert Spaces
• Nearly optimal stochastic approximation for online principal subspace estimation
• Adaptive Nonlinear Estimation Based on Parallel Projection Along Affine Subspaces in Reproducing Kernel Hilbert Space
• Theory and Applications of Models of Computation
• A Stochastic Majorize-Minimize Subspace Algorithm for Online Penalized Least Squares Estimation
• scientific article; zbMATH DE number 5811685
• Parsimonious online learning with kernels via sparse projections in function space
• Near-optimal stochastic approximation for online principal component estimation
• Online kernel estimation of stationary stochastic diffusion models
• Online Dictionary Learning for Kernel LMS

Cited in

(2)

• scientific article; zbMATH DE number 5811685 (Why is no real title available?)
• A Stochastic Majorize-Minimize Subspace Algorithm for Online Penalized Least Squares Estimation

This page was built for publication: Online Nonlinear Estimation via Iterative <inline-formula> <tex-math notation="LaTeX">$L^2$</tex-math> </inline-formula>-Space Projections: Reproducing Kernel of Subspace

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4622301)