On-line predictive linear regression

DOI10.1214/08-AOS622zbMATH Open1160.62065arXivmath/0511522OpenAlexW3104560798WikidataQ62046709 ScholiaQ62046709MaRDI QIDQ1018652FDOQ1018652

Alexander Gammerman, Ilia Nouretdinov, Vladimir Vovk

Publication date: 20 May 2009

Published in: The Annals of Statistics (Search for Journal in Brave)

Abstract: We consider the on-line predictive version of the standard problem of linear regression; the goal is to predict each consecutive response given the corresponding explanatory variables and all the previous observations. We are mainly interested in prediction intervals rather than point predictions. The standard treatment of prediction intervals in linear regression analysis has two drawbacks: (1) the classical prediction intervals guarantee that the probability of error is equal to the nominal significance level epsilon, but this property per se does not imply that the long-run frequency of error is close to epsilon; (2) it is not suitable for prediction of complex systems as it assumes that the number of observations exceeds the number of parameters. We state a general result showing that in the on-line protocol the frequency of error for the classical prediction intervals does equal the nominal significance level, up to statistical fluctuations. We also describe alternative regression models in which informative prediction intervals can be found before the number of observations exceeds the number of parameters. One of these models, which only assumes that the observations are independent and identically distributed, is popular in machine learning but greatly underused in the statistical theory of regression.

Full work available at URL: https://arxiv.org/abs/0906.3123

Recommendations

• Prediction intervals for regression models
• Theory and Applications of Models of Computation
• Adaptive and optimal online linear regression on \(\ell ^{1}\)-balls
• Adaptive and optimal online linear regression on \(\ell^1\)-balls
• Leading strategies in competitive on-line prediction

zbMATH Keywords

multivariate analysisGauss linear modelprequential statisticsindependent identically distributed observationson-line protocol

Mathematics Subject Classification ID

Nonparametric regression and quantile regression (62G08) Linear regression; mixed models (62J05) Learning and adaptive systems in artificial intelligence (68T05) Inference from stochastic processes and prediction (62M20)

Cites Work

• Title not available (Why is that?)
• Title not available (Why is that?)
• Prediction, Learning, and Games
• Present Position and Potential Developments: Some Personal Views: Statistical Theory: The Prequential Approach
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Extremal families and systems of sufficient statistics
• An ancillarity paradox which appears in multiple linear regression
• Determination of Sample Sizes for Setting Tolerance Limits
• Characterizations of prediction sufficiency (adequacy) in terms of risk functions
• On the Completeness of Order Statistics
• A Tale of Two Regressions
• Sequential Probability Forecasts and the Probability Integral Transform

Cited In (17)

• Distribution-free conditional median inference
• Conformal Sensitivity Analysis for Individual Treatment Effects
• An Exact and Robust Conformal Inference Method for Counterfactual and Synthetic Controls
• Conformal prediction: a unified review of theory and new challenges
• Unsupervised streaming anomaly detection for instrumented infrastructure
• Gaussian copula function-on-scalar regression in reproducing kernel Hilbert space
• PredictiveRegression
• Random Forest Prediction Intervals
• Online Forecasting Matrix Factorization
• Distribution-Free Prediction Sets
• Distribution-Free Predictive Inference For Regression
• Prediction intervals with controlled length in the heteroscedastic Gaussian regression
• Discretized conformal prediction for efficient distribution-free inference
• A study of two on-line prediction algorithms
• Nonparametric predictive distributions based on conformal prediction
• Conformal Predictive Distributions with Kernels
• Conditional predictive inference for stable algorithms

This page was built for publication: On-line predictive linear regression

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1018652)