Model Order Selection Based on Information Theoretic Criteria: Design of the Penalty
From MaRDI portal
Publication:4580591
DOI10.1109/TSP.2015.2414900zbMATH Open1394.94803arXiv1910.03980OpenAlexW2056601790MaRDI QIDQ4580591FDOQ4580591
Authors: Andrea Mariani, Andrea Giorgetti, Marco Chiani 
Publication date: 22 August 2018
Published in: IEEE Transactions on Signal Processing (Search for Journal in Brave)
Abstract: Information theoretic criteria (ITC) have been widely adopted in engineering and statistics for selecting, among an ordered set of candidate models, the one that better fits the observed sample data. The selected model minimizes a penalized likelihood metric, where the penalty is determined by the criterion adopted. While rules for choosing a penalty that guarantees a consistent estimate of the model order are known, theoretical tools for its design with finite samples have never been provided in a general setting. In this paper, we study model order selection for finite samples under a design perspective, focusing on the generalized information criterion (GIC), which embraces the most common ITC. The theory is general, and as case studies we consider: a) the problem of estimating the number of signals embedded in additive white Gaussian noise (AWGN) by using multiple sensors; b) model selection for the general linear model (GLM), which includes e.g. the problem of estimating the number of sinusoids in AWGN. The analysis reveals a trade-off between the probabilities of overestimating and underestimating the order of the model. We then propose to design the GIC penalty to minimize underestimation while keeping the overestimation probability below a specified level. For the considered problems, this method leads to analytical derivation of the optimal penalty for a given sample size. A performance comparison between the penalty optimized GIC and common AIC and BIC is provided, demonstrating the effectiveness of the proposed design strategy.
Full work available at URL: https://arxiv.org/abs/1910.03980
Signal theory (characterization, reconstruction, filtering, etc.) (94A12) Information theory (general) (94A15)
Cited In (1)
This page was built for publication: Model Order Selection Based on Information Theoretic Criteria: Design of the Penalty
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4580591)
