Regularized linear system identification using atomic, nuclear and kernel-based norms: the role of the stability constraint

DOI10.1016/J.AUTOMATICA.2016.02.012MaRDI QIDQ286265zbMATH OpenOpenAlexFDO

Authors Gianluigi Pillonetto, Tianshi Chen, Alessandro Chiuso, Lennart Ljung, Giuseppe De Nicolao

Publication date 20 May 2016

Published in Automatica (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1507.00564

zbMATH Keywords

atomic and nuclear normsBayesian interpretation of regularizationGaussian processesHankel operatorkernel-based regularizationLASSOlinear system identificationreproducing kernel Hilbert spaces

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05) System identification (93B30) Linear systems in control theory (93C05)

Abstract: Inspired by ideas taken from the machine learning literature, new regularization techniques have been recently introduced in linear system identification. In particular, all the adopted estimators solve a regularized least squares problem, differing in the nature of the penalty term assigned to the impulse response. Popular choices include atomic and nuclear norms (applied to Hankel matrices) as well as norms induced by the so called stable spline kernels. In this paper, a comparative study of estimators based on these different types of regularizers is reported. Our findings reveal that stable spline kernels outperform approaches based on atomic and nuclear norms since they suitably embed information on impulse response stability and smoothness. This point is illustrated using the Bayesian interpretation of regularization. We also design a new class of regularizers defined by "integral" versions of stable spline/TC kernels. Under quite realistic experimental conditions, the new estimators outperform classical prediction error methods also when the latter are equipped with an oracle for model order selection.

Recommendations

• Generalized system identification with stable spline kernels
• A new kernel-based approach for linear system identification
• Tuning complexity in regularized kernel-based regression and linear system identification: the robustness of the marginal likelihood estimator
• Maximum entropy vector kernels for MIMO system identification
• Kernel methods in system identification, machine learning and function estimation: a survey

Cites work

• A new kernel-based approach for linear system identification
• A new look at the statistical model identification
• Compressed sensing
• Convex vs non-convex estimators for regression and sparse estimation: the mean squared error properties of ARD and GLasso
• Estimation with quadratic loss.
• Exact matrix completion via convex optimization
• For most large underdetermined systems of linear equations the minimal 𝓁₁‐norm solution is also the sparsest solution
• Frequency Domain Subspace Identification Using Nuclear Norm Minimization and Hankel Matrix Realizations
• Graph implementations for nonsmooth convex programs
• scientific article; zbMATH DE number 45848 (Why is no real title available?)
• scientific article; zbMATH DE number 45871 (Why is no real title available?)
• scientific article; zbMATH DE number 842531 (Why is no real title available?)
• scientific article; zbMATH DE number 840151 (Why is no real title available?)
• scientific article; zbMATH DE number 845714 (Why is no real title available?)
• scientific article; zbMATH DE number 2208228 (Why is no real title available?)
• Interior-point method for nuclear norm approximation with application to system identification
• Kernel methods in system identification, machine learning and function estimation: a survey
• Model predictive control. With a foreword by M. J. Grimble and M. A. Johnson
• On covariance function tests used in system identification
• On the estimation of transfer functions, regularizations and Gaussian processes-revisited
• On the mathematical foundations of learning
• Prediction error identification of linear systems: a nonparametric Gaussian regression approach
• Sparse Estimation of Polynomial and Rational Dynamical Models
• System identification using Laguerre models
• System Identification Via Sparse Multiple Kernel-Based Regularization Using Sequential Convex Optimization Techniques
• Tensors in Image Processing and Computer Vision
• The Adaptive Lasso and Its Oracle Properties
• The convex geometry of linear inverse problems
• The elements of statistical learning. Data mining, inference, and prediction
• The Estimation of Prediction Error
• Theory of Reproducing Kernels
• Tuning complexity in regularized kernel-based regression and linear system identification: the robustness of the marginal likelihood estimator
• Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties

Cited in

(32)

• The generalized cross validation filter
• On input design for regularized LTI system identification: power-constrained input
• Subspace-based spectrum estimation in innovation models by mixed norm minimization
• System identification using kernel-based regularization: new insights on stability and consistency issues
• The harmonic analysis of kernel functions
• On asymptotic properties of hyperparameter estimators for kernel-based regularization methods
• Revisiting total model errors and model validation
• Sparse estimation in linear dynamic networks using the stable spline horseshoe prior
• On embeddings and inverse embeddings of input design for regularized system identification
• Fractional order neural networks for system identification
• Linear system identification using the sequential stabilizing spline algorithm
• Kernel-based system identification with manifold regularization: a Bayesian perspective
• Regularized spectrum estimation using stable spline kernels
• Maximum entropy vector kernels for MIMO system identification
• Nuclear norm system identification with missing inputs and outputs
• Maximum entropy properties of discrete-time first-order stable spline kernel
• Control-oriented regularization for linear system identification
• The existence and uniqueness of solutions for kernel-based system identification
• A new kernel-based approach to hybrid system identification
• Kernel absolute summability is sufficient but not necessary for RKHS stability
• Generalized system identification with stable spline kernels
• Stability analysis of distributed-order nonlinear dynamic systems
• A shift in paradigm for system identification
• On the regularization and optimization in quantum detector tomography
• Stable spline identification of linear systems under missing data
• When cannot regularization improve the least squares estimate in the kernel-based regularized system identification
• Kernel-based identification with frequency domain side-information
• Sequential stabilizing spline algorithm for linear systems: eigenvalue approximation and polishing
• Deep networks for system identification: a survey
• A graph learning approach for kernel-based system identification with manifold regularization
• Tuning complexity in regularized kernel-based regression and linear system identification: the robustness of the marginal likelihood estimator
• A new kernel-based approach for linear system identification

Describes a project that uses

Uses Software

• LASSO
• CVX
• glmnet
• ElemStatLearn

This page was built for publication: Regularized linear system identification using atomic, nuclear and kernel-based norms: the role of the stability constraint

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q286265)