Finite-Sample Risk Bounds for Maximum Likelihood Estimation With Arbitrary Penalties

DOI10.1109/TIT.2017.2789214MaRDI QIDQ4569216zbMATH OpenOpenAlexFDO

Authors W. D. Brinda, Jason M. Klusowski

Publication date 27 June 2018

Published in IEEE Transactions on Information Theory (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1712.10087

Mathematics Subject Classification ID

Statistical aspects of information-theoretic topics (62B10) Point estimation (62F10) Ridge regression; shrinkage estimators (Lasso) (62J07) Bounds on codes (94B65)

Abstract: The MDL two-part coding $extitindexofresolvability$ provides a finite-sample upper bound on the statistical risk of penalized likelihood estimators over countable models. However, the bound does not apply to unpenalized maximum likelihood estimation or procedures with exceedingly small penalties. In this paper, we point out a more general inequality that holds for arbitrary penalties. In addition, this approach makes it possible to derive exact risk bounds of order $1/n$ for iid parametric models, which improves on the order $(logn)/n$ resolvability bounds. We conclude by discussing implications for adaptive estimation.

Cited in

(2)

• Minimum description length revisited
• Risk bounds for statistical learning

This page was built for publication: Finite-Sample Risk Bounds for Maximum Likelihood Estimation With Arbitrary Penalties

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4569216)