Kernel-based linear system identification: when does the representer theorem hold?

DOI10.1016/J.AUTOMATICA.2023.111347MaRDI QIDQ6537302zbMATH OpenFDO

Authors Gianluigi Pillonetto, Mauro Bisiacco

Publication date 14 May 2024

Published in Automatica (Search for Journal in Brave)

Full work available at URL https://doi.org/10.1016/J.AUTOMATICA.2023.111347

zbMATH Keywords

kernel-based linear system identification, reproducing kernel Hilber spaceregularized system identificationrepresenter theorem

Mathematics Subject Classification ID

System identification (93B30) Linear systems in control theory (93C05) Control/observation systems in abstract spaces (93C25)

Review text

Kernel-based identification is an increasingly popular paradigm in learning dynamic systems from data. It is centred on estimation of an unknown function (an impulse response) from a finite set of linearly related noisy measurements. Mathematically, this function is assumed to belong to a reproducing kernel Hilbert space (RKHS), a particular space of functions induced by positive definite kernels. The corresponding estimate minimizes an objective functional which trades-off measurements fit and the RKHS norm which acts as regularizer.\N\NIn this setting, the representer theorem (RT) constitutes a main result in kernel-based linear system identification, which states that regularized estimates of impulse responses are sums of a finite number of basis functions obtained by convolving the kernel with the system input. For its applicability, however, the structure of the kernel-based estimator should satisfy some technical conditions regarding the continuity of the functionals which map the impulse responses into the system output.\N\NThe key result of the paper in question regards the family of stable RKHSs induced by discrete-time kernels or continuous-time Mercer (continuous) kernels. The authors show that this family coincides with the class of RKHSs where all the convolutions induced by physical (bounded) inputs are continuous functionals. In turn, this proves a fundamental connection between RT and BIBO stability, the latter meaning that the output of a dynamic system fed with a bounded input remains bounded. Specifically, RT is valid given any physical (bounded) input to the system if and only if the kernel is stable. In other words, RKHS stability is the necessary and sufficient condition to make kernel based linear system identification well-posed. As a by-product, a new stability test is obtained and a the relationship between uniform continuity of convolutions and stable RKHSs is derived. Overall, the paper is an extremely good and valuable piece of work on a subject that attracts considerable attention in statistics, system identification and machine learning.

Recommendations

• System identification using kernel-based regularization: new insights on stability and consistency issues
• On the mathematical foundations of stable RKHSs
• The existence and uniqueness of solutions for kernel-based system identification
• On the stability of reproducing kernel Hilbert spaces of discrete-time impulse responses
• Mercer kernel absolute integrability is only sufficient for RKHS stability

Cites work

• A new kernel-based approach for linear system identification
• A new kernel-based approach to system identification with quantized output data
• A representer theorem for deep kernel learning
• Applied functional analysis. Functional analysis, Sobolev spaces and elliptic differential equations
• Applied functional analysis. Main principles and their applications
• Continuous-Time DC Kernel—A Stable Generalized First-Order Spline Kernel
• Estimating parameters and stochastic functions of one variable using nonlinear measurement models
• scientific article; zbMATH DE number 1804115 (Why is no real title available?)
• scientific article; zbMATH DE number 45848 (Why is no real title available?)
• scientific article; zbMATH DE number 1022658 (Why is no real title available?)
• Kernel absolute summability is sufficient but not necessary for RKHS stability
• Kernel methods in system identification, machine learning and function estimation: a survey
• Kernels for linear time invariant system identification
• Maximum Entropy Kernels for System Identification
• On the estimation of transfer functions, regularizations and Gaussian processes-revisited
• On the mathematical foundations of learning
• On the mathematical foundations of stable RKHSs
• On the stability of reproducing kernel Hilbert spaces of discrete-time impulse responses
• Regularization networks and support vector machines
• Regularized system identification. Learning dynamic models from data
• Solutions of nonlinear control and estimation problems in reproducing kernel Hilbert spaces: existence and numerical determination
• Support-vector networks
• System Identification Via Sparse Multiple Kernel-Based Regularization Using Sequential Convex Optimization Techniques
• System identification. A survey
• The elements of statistical learning. Data mining, inference, and prediction
• The existence and uniqueness of solutions for kernel-based system identification
• Theory of Reproducing Kernels

Cited in

(2)

• A refinement of the stability test for reproducing kernel Hilbert spaces
• Mercer kernel absolute integrability is only sufficient for RKHS stability

This page was built for publication: Kernel-based linear system identification: when does the representer theorem hold?

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6537302)