Model selection with the loss rank principle

DOI10.1016/J.CSDA.2009.11.015MaRDI QIDQ962384zbMATH OpenOpenAlexDBLPWikidataFDO

Authors Marcus Hutter, Minh-Ngoc Tran

Publication date 6 April 2010

Published in Computational Statistics and Data Analysis (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1003.0516

Mathematics Subject Classification ID

Computational methods for problems pertaining to statistics (62-08) Linear regression; mixed models (62J05) Learning and adaptive systems in artificial intelligence (68T05)

Abstract: A key issue in statistics and machine learning is to automatically select the "right" model complexity, e.g., the number of neighbors to be averaged over in k nearest neighbor (kNN) regression or the polynomial degree in regression with polynomials. We suggest a novel principle - the Loss Rank Principle (LoRP) - for model selection in regression and classification. It is based on the loss rank, which counts how many other (fictitious) data would be fitted better. LoRP selects the model that has minimal loss rank. Unlike most penalized maximum likelihood variants (AIC, BIC, MDL), LoRP depends only on the regression functions and the loss function. It works without a stochastic noise model, and is directly applicable to any non-parametric regressor, like kNN.

Recommendations

• The Loss Rank Principle for Model Selection
• Data-Driven Model Choice in Multivariate Nonparametric Regression
• Model Selection Using the Minimum Description Length Principle
• Model Selection and the Principle of Minimum Description Length
• Adaptive model selection using empirical complexities

Cites work

• Approximation of the determinant of large sparse symmetric positive definite matrices
• Asymptotic mean efficiency of a selection of regression variables
• Can the strengths of AIC and BIC be shared? A conflict between model indentification and regression estimation
• Estimating the dimension of a model
• scientific article; zbMATH DE number 708500 (Why is no real title available?)
• scientific article; zbMATH DE number 1034037 (Why is no real title available?)
• scientific article; zbMATH DE number 3444596 (Why is no real title available?)
• scientific article; zbMATH DE number 1906319 (Why is no real title available?)
• scientific article; zbMATH DE number 1453026 (Why is no real title available?)
• scientific article; zbMATH DE number 7646022 (Why is no real title available?)
• Model selection and error estimation
• Model selection based on minimum description length
• Modeling by shortest data description
• Rademacher penalties and structural risk minimization
• Regression and time series model selection in small samples
• Smoothing noisy data with spline functions: Estimating the correct degree of smoothing by the method of generalized cross-validation
• Testing the order of a model
• The elements of statistical learning. Data mining, inference, and prediction
• The Loss Rank Principle for Model Selection
• The Relationship between Variable Selection and Data Agumentation and a Method for Prediction
• Tuning parameter selectors for the smoothly clipped absolute deviation method

Cited in

(5)

• Rank-based methods for modeling dependence between loss triangles
• The loss rank criterion for variable selection in linear regression analysis
• scientific article; zbMATH DE number 6500982 (Why is no real title available?)
• A loss function approach to model specification testing and its relative efficiency
• The Loss Rank Principle for Model Selection

Describes a project that uses

Uses Software

• bootstrap

This page was built for publication: Model selection with the loss rank principle

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q962384)