A note on model selection for small sample regression

DOI10.1007/S10994-017-5645-5MaRDI QIDQ682390zbMATH OpenOpenAlexFDO

Authors Masanori Kawakita, Jun-ichi Takeuchi

Publication date 2 February 2018

Published in Machine Learning (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1610.03938

zbMATH Keywords

finite samplemodel selectionregressionunlabeled data

Mathematics Subject Classification ID

Statistical ranking and selection procedures (62F07) General nonlinear regression (62J02)

Abstract: The risk estimator called "Direct Eigenvalue Estimator" (DEE) is studied. DEE was developed for small sample regression. In contrast to many existing model selection criteria, derivation of DEE requires neither any asymptotic assumption nor any prior knowledge about the noise variance and the noise distribution. It was reported that DEE performed well in small sample cases but DEE performed a little worse than the state-of-the-art ADJ. This seems somewhat counter-intuitive because DEE was developed for specifically regression problem by exploiting available information exhaustively, while ADJ was developed for general setting. In this paper, we point out that the derivation of DEE includes an inappropriate part in spite that the resultant form of DEE is valid in a sense. As its result, DEE cannot derive its potential. We introduce a class of valid risk estimators based on the idea of DEE and show that better risk estimators (mDEE) can be found in the class. By numerical experiments, we verify that mDEE often performs better than or at least equally the original DEE and ADJ.

Recommendations

• Model selection for small sample regression
• Linear model selection based on risk estimation
• The risk inflation criterion for multiple regression
• Generalized ridge estimator and model selection criteria in multivariate linear regression
• scientific article; zbMATH DE number 1784954

Cites work

• Estimating the dimension of a model
• Further analysis of the data by Akaike's information criterion and the finite corrections
• Generalised information criteria in model selection
• scientific article; zbMATH DE number 3444596 (Why is no real title available?)
• Matrix Analysis
• Model selection for small sample regression
• Statistical predictor identification
• The elements of statistical learning. Data mining, inference, and prediction

Cited in

(4)

• A small-sample correction for the Schwarz SIC model selection criterion.
• A robust model structure selection method for small sample size and multiple datasets problems
• Regression and time series model selection in small samples
• A Small-Sample Estimator for the Sample-Selection Model

Describes a project that uses

Uses Software

• UCI
• ElemStatLearn

This page was built for publication: A note on model selection for small sample regression

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q682390)