Making and evaluating point forecasts
From MaRDI portal
Publication:91134
DOI10.1198/JASA.2011.R10138zbMATH Open1232.62028arXiv0912.0902OpenAlexW2075965721MaRDI QIDQ91134FDOQ91134
Authors: Tilmann Gneiting, Tilmann Gneiting 
Publication date: June 2011
Published in: Journal of the American Statistical Association (Search for Journal in Brave)
Abstract: Typically, point forecasting methods are compared and assessed by means of an error measure or scoring function, such as the absolute error or the squared error. The individual scores are then averaged over forecast cases, to result in a summary measure of the predictive performance, such as the mean absolute error or the (root) mean squared error. I demonstrate that this common practice can lead to grossly misguided inferences, unless the scoring function and the forecasting task are carefully matched. Effective point forecasting requires that the scoring function be specified ex ante, or that the forecaster receives a directive in the form of a statistical functional, such as the mean or a quantile of the predictive distribution. If the scoring function is specified ex ante, the forecaster can issue the optimal point forecast, namely, the Bayes rule. If the forecaster receives a directive in the form of a functional, it is critical that the scoring function be consistent for it, in the sense that the expected score is minimized when following the directive. A functional is elicitable if there exists a scoring function that is strictly consistent for it. Expectations, ratios of expectations and quantiles are elicitable. For example, a scoring function is consistent for the mean functional if and only if it is a Bregman function. It is consistent for a quantile if and only if it is generalized piecewise linear. Similar characterizations apply to ratios of expectations and to expectiles. Weighted scoring functions are consistent for functionals that adapt to the weighting in peculiar ways. Not all functionals are elicitable; for instance, conditional value-at-risk is not, despite its popularity in quantitative finance.
Full work available at URL: https://arxiv.org/abs/0912.0902
Recommendations
- Of quantiles and expectiles: consistent scoring functions, Choquet representations and forecast rankings. With discussion and authors' reply
- Order-sensitivity and equivariance of scoring functions
- Higher order elicitability and Osband's principle
- Point forecasting and forecast evaluation with generalized Huber loss
- Forecast evaluation of quantiles, prediction intervals, and other set-valued functionals
Bayes ruleBregman functionconditional value-at-risk (CVaR)decision theoryelicitabilityexpectilemeanmedianmodeproper scoring rulequantilesstatistical functionals
Decision theory (91B06) Bayesian problems; characterization of Bayes procedures (62C10) Statistical decision theory (62C99)
Cited In (only showing first 100 items - show all)
- Comparative evaluation of point process forecasts
- A coupled statistical and deterministic model for forecasting climate-driven dengue incidence in Selangor, Malaysia
- Range-based risk measures and their applications
- Regression diagnostics meets forecast evaluation: conditional calibration, reliability diagrams, and coefficient of determination
- Better the devil you know: improved forecasts from imperfect models
- Testing for auto-calibration with Lorenz and concentration curves
- Generic Conditions for Forecast Dominance
- Comparing Possibly Misspecified Forecasts
- Predicting the Global Minimum Variance Portfolio
- Proper Scoring Rules for Evaluating Density Forecasts with Asymmetric Loss Functions
- Retire: robust expectile regression in high dimensions
- Volatility prediction comparison via robust volatility proxies: an empirical deviation perspective
- Estimation of complier expected shortfall treatment effects with a binary instrumental variable
- Diversification quotients based on VaR and ES
- Parametric quantile autoregressive moving average models with exogenous terms
- Bias-reduced and variance-corrected asymptotic Gaussian inference about extreme expectiles
- Calibrating Distribution Models from PELVE
- Bayesian Forecasting of Many Count-Valued Time Series
- Title not available (Why is that?)
- Distributionally robust reinsurance with expectile
- Forecasting overdispersed INAR(1) count time series with negative binomial marginal
- Convex and Lorenz orders under balance correction in nonlife insurance pricing: review and new developments
- Robust estimation in regression and classification methods for large dimensional data
- Using proxies to improve forecast evaluation
- Reconstruction of late Holocene climate based on tree growth and mechanistic hierarchical models
- Model selection with Gini indices under auto-calibration
- Lasso regularization within the LocalGLMnet architecture
- Multiple measures realized GARCH models
- Safe machine learning
- Backtestability and the ridge backtest
- On Testing Equal Conditional Predictive Ability Under Measurement Error
- Forecasting with Economic News
- Consistent Estimation of Distribution Functions under Increasing Concave and Convex Stochastic Ordering
- Comparing Predictive Accuracy in the Presence of a Loss Function Shape Parameter
- Realized Quantiles*
- Combined Density Nowcasting in an Uncertain Economic Environment
- Backtesting Systemic Risk Forecasts Using Multi-Objective Elicitability
- Powerful Backtests for Historical Simulation Expected Shortfall Models
- Pinball boosting of regression quantiles
- Truncated generalized extreme value distribution-based ensemble model output statistics model for calibration of wind speed ensemble forecasts
- Model selection with Pearson's correlation, concentration and Lorenz curves under autocalibration
- Sequential testing for elicitable functionals via supermartingales
- One Axiom to Rule Them All: A Minimalist Axiomatization of Quantiles
- Flexible Expectile Regression in Reproducing Kernel Hilbert Spaces
- Uncertainty Comparison Between Value-at-Risk and Expected Shortfall
- Inter-order relations between equivalence for \(L_p\)-quantiles of the Student's \(t\) distribution
- Risk quantization by magnitude and propensity
- An expectile computation cookbook
- Inference for joint quantile and expected shortfall regression
- Forecasting Value at Risk and Expected Shortfall Using a Semiparametric Approach Based on the Asymmetric Laplace Distribution
- Isotonic recalibration under a low signal-to-noise ratio
- Conditionally Elicitable Dynamic Risk Measures for Deep Reinforcement Learning
- Marked self-exciting point process modelling of information diffusion on twitter
- Uniform calibration tests for forecasting systems with small lead time
- Scoring predictions at extreme quantiles
- A parsimonious parametric model for generating margin requirements for futures
- Tail asymptotics of generalized deflated risks with insurance applications
- Semi-parametric Bayesian tail risk forecasting incorporating realized measures of volatility
- Quantile Evaluation, Sensitivity to Bracketing, and Sharing Business Payoffs
- Optimal estimation of the supremum and occupation times of a self-similar Lévy process
- Optimal Trading Policies for Wind Energy Producer
- Dual representation of expectile-based expected shortfall and its properties
- Semiparametric empirical best prediction for small area estimation of unemployment indicators
- Grouped multivariate and functional time series forecasting: an application to annuity pricing
- Point forecasting and forecast evaluation with generalized Huber loss
- Joint inference on extreme expectiles for multivariate heavy-tailed distributions
- Risks in emerging markets equities: time-varying versus spatial risk analysis
- On the indirect elicitability of the mode and modal interval
- Bayesian spline method for assessing extreme loads on wind turbines
- Estimating and backtesting risk under heavy tails
- Probabilistic sensitivity measures as information value
- Distributional Transforms, Probability Distortions, and Their Applications
- Joint generalized quantile and conditional tail expectation regression for insurance risk analysis
- Backtesting VaR and expectiles with realized scores
- Forecasting intra-individual changes of affective states taking into account inter-individual differences using intensive longitudinal data from a university student dropout study in math
- Functional prediction of intraday cumulative returns
- A Theory for Measures of Tail Risk
- Scoring interval forecasts: equal-tailed, shortest, and modal interval
- Forecast evaluation of quantiles, prediction intervals, and other set-valued functionals
- Econometric modeling of risk measures: a selective review of the recent literature
- Isotonic regression for elicitable functionals and their Bayes risk
- Why scoring functions cannot assess tail properties
- Measuring and adjusting for overconfidence
- Testing the reliability of forecasting systems
- Bayes risk, elicitability, and the Expected Shortfall
- On automatic bias reduction for extreme expectile estimation
- On the elicitability of range value at risk
- General truthfulness characterizations via convex analysis
- Density forecast of financial returns using decomposition and maximum entropy
- Random distributions via sequential quantile array
- On the properties of the Lambda value at risk: robustness, elicitability and consistency
- Properization: constructing proper scoring rules via Bayes acts
- Robust VIF regression with application to variable selection in large data sets
- Deep quantile and deep composite triplet regression
- Characterizing the optimal solutions to the isotonic regression problem for identifiable functionals
- Virtual historical simulation for estimating the conditional VaR of large portfolios
- Inventory -- forecasting: mind the gap
- Feature extraction for functional time series: theory and application to NIR spectroscopy data
- Estimating value-at-risk and expected shortfall using the intraday low and range data
- Spatio-temporal short-term wind forecast: a calibrated regime-switching method
This page was built for publication: Making and evaluating point forecasts
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q91134)
