Regularized linear system identification using atomic, nuclear and kernel-based norms: the role of the stability constraint

DOI10.1016/J.AUTOMATICA.2016.02.012zbMATH Open1338.93121arXiv1507.00564OpenAlexW809676393MaRDI QIDQ286265FDOQ286265

Authors: Gianluigi Pillonetto, Tianshi Chen, Alessandro Chiuso, Lennart Ljung, Giuseppe De Nicolao 

Publication date: 20 May 2016

Published in: Automatica (Search for Journal in Brave)

Abstract: Inspired by ideas taken from the machine learning literature, new regularization techniques have been recently introduced in linear system identification. In particular, all the adopted estimators solve a regularized least squares problem, differing in the nature of the penalty term assigned to the impulse response. Popular choices include atomic and nuclear norms (applied to Hankel matrices) as well as norms induced by the so called stable spline kernels. In this paper, a comparative study of estimators based on these different types of regularizers is reported. Our findings reveal that stable spline kernels outperform approaches based on atomic and nuclear norms since they suitably embed information on impulse response stability and smoothness. This point is illustrated using the Bayesian interpretation of regularization. We also design a new class of regularizers defined by "integral" versions of stable spline/TC kernels. Under quite realistic experimental conditions, the new estimators outperform classical prediction error methods also when the latter are equipped with an oracle for model order selection.

Full work available at URL: https://arxiv.org/abs/1507.00564

Recommendations

• Generalized system identification with stable spline kernels
• A new kernel-based approach for linear system identification
• Tuning complexity in regularized kernel-based regression and linear system identification: the robustness of the marginal likelihood estimator
• Maximum entropy vector kernels for MIMO system identification
• Kernel methods in system identification, machine learning and function estimation: a survey

zbMATH Keywords

Gaussian processesreproducing kernel Hilbert spacesLASSOHankel operatoratomic and nuclear normsBayesian interpretation of regularizationkernel-based regularizationlinear system identification

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05) System identification (93B30) Linear systems in control theory (93C05)

Cites Work

• The elements of statistical learning. Data mining, inference, and prediction
• The Adaptive Lasso and Its Oracle Properties
• Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
• Title not available (Why is that?)
• A new look at the statistical model identification
• Theory of Reproducing Kernels
• Title not available (Why is that?)
• Title not available (Why is that?)
• Exact matrix completion via convex optimization
• On the mathematical foundations of learning
• Title not available (Why is that?)
• Estimation with quadratic loss.
• For most large underdetermined systems of linear equations the minimal 𝓁₁‐norm solution is also the sparsest solution
• Compressed sensing
• On the estimation of transfer functions, regularizations and Gaussian processes-revisited
• System Identification Via Sparse Multiple Kernel-Based Regularization Using Sequential Convex Optimization Techniques
• Graph implementations for nonsmooth convex programs
• Kernel methods in system identification, machine learning and function estimation: a survey
• The Estimation of Prediction Error
• Prediction error identification of linear systems: a nonparametric Gaussian regression approach
• On covariance function tests used in system identification
• A new kernel-based approach for linear system identification
• Title not available (Why is that?)
• Title not available (Why is that?)
• Frequency Domain Subspace Identification Using Nuclear Norm Minimization and Hankel Matrix Realizations
• The convex geometry of linear inverse problems
• Model predictive control. With a foreword by M. J. Grimble and M. A. Johnson
• Title not available (Why is that?)
• Sparse Estimation of Polynomial and Rational Dynamical Models
• Interior-point method for nuclear norm approximation with application to system identification
• Tensors in Image Processing and Computer Vision
• System identification using Laguerre models
• Tuning complexity in regularized kernel-based regression and linear system identification: the robustness of the marginal likelihood estimator

Cited In (24)

• The harmonic analysis of kernel functions
• Sparse estimation in linear dynamic networks using the stable spline horseshoe prior
• Fractional order neural networks for system identification
• On embeddings and inverse embeddings of input design for regularized system identification
• Subspace-based spectrum estimation in innovation models by mixed norm minimization
• When cannot regularization improve the least squares estimate in the kernel-based regularized system identification
• Maximum entropy vector kernels for MIMO system identification
• A new kernel-based approach for linear system identification
• Linear system identification using the sequential stabilizing spline algorithm
• Kernel-based identification with frequency domain side-information
• Kernel Absolute Summability Is Sufficient but Not Necessary for RKHS Stability
• The generalized cross validation filter
• A shift in paradigm for system identification
• System identification using kernel-based regularization: new insights on stability and consistency issues
• Stability analysis of distributed-order nonlinear dynamic systems
• On input design for regularized LTI system identification: power-constrained input
• On asymptotic properties of hyperparameter estimators for kernel-based regularization methods
• Kernel-based system identification with manifold regularization: a Bayesian perspective
• The existence and uniqueness of solutions for kernel-based system identification
• Revisiting total model errors and model validation
• Deep networks for system identification: a survey
• On the regularization and optimization in quantum detector tomography
• Sequential stabilizing spline algorithm for linear systems: eigenvalue approximation and polishing
• Nuclear norm system identification with missing inputs and outputs

Uses Software

• LASSO
• CVX
• glmnet
• ElemStatLearn

This page was built for publication: Regularized linear system identification using atomic, nuclear and kernel-based norms: the role of the stability constraint

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q286265)