Making and evaluating point forecasts
From MaRDI portal
Abstract: Typically, point forecasting methods are compared and assessed by means of an error measure or scoring function, such as the absolute error or the squared error. The individual scores are then averaged over forecast cases, to result in a summary measure of the predictive performance, such as the mean absolute error or the (root) mean squared error. I demonstrate that this common practice can lead to grossly misguided inferences, unless the scoring function and the forecasting task are carefully matched. Effective point forecasting requires that the scoring function be specified ex ante, or that the forecaster receives a directive in the form of a statistical functional, such as the mean or a quantile of the predictive distribution. If the scoring function is specified ex ante, the forecaster can issue the optimal point forecast, namely, the Bayes rule. If the forecaster receives a directive in the form of a functional, it is critical that the scoring function be consistent for it, in the sense that the expected score is minimized when following the directive. A functional is elicitable if there exists a scoring function that is strictly consistent for it. Expectations, ratios of expectations and quantiles are elicitable. For example, a scoring function is consistent for the mean functional if and only if it is a Bregman function. It is consistent for a quantile if and only if it is generalized piecewise linear. Similar characterizations apply to ratios of expectations and to expectiles. Weighted scoring functions are consistent for functionals that adapt to the weighting in peculiar ways. Not all functionals are elicitable; for instance, conditional value-at-risk is not, despite its popularity in quantitative finance.
Recommendations
- Of quantiles and expectiles: consistent scoring functions, Choquet representations and forecast rankings. With discussion and authors' reply
- Order-sensitivity and equivariance of scoring functions
- Higher order elicitability and Osband's principle
- Point forecasting and forecast evaluation with generalized Huber loss
- Forecast evaluation of quantiles, prediction intervals, and other set-valued functionals
Cited in
(only showing first 100 items - show all)- Testing the reliability of forecasting systems
- Grouped multivariate and functional time series forecasting: an application to annuity pricing
- Robust VIF regression with application to variable selection in large data sets
- Bayes risk, elicitability, and the Expected Shortfall
- Conditionally Elicitable Dynamic Risk Measures for Deep Reinforcement Learning
- On the elicitability of range value at risk
- Estimating value-at-risk and expected shortfall using the intraday low and range data
- Estimating and backtesting risk under heavy tails
- Tail asymptotics of generalized deflated risks with insurance applications
- Optimal insurance design in the presence of exclusion clauses
- Scoring interval forecasts: equal-tailed, shortest, and modal interval
- A relative error-based approach for variable selection
- Isotonic regression for elicitable functionals and their Bayes risk
- Using the Bayesian Shtarkov solution for predictions
- Forecast evaluation of quantiles, prediction intervals, and other set-valued functionals
- Spatio-temporal short-term wind forecast: a calibrated regime-switching method
- Joint generalized quantile and conditional tail expectation regression for insurance risk analysis
- Optimal operational service levels in vendor managed inventory contracts -- an exact approach
- Measurability of functionals and of ideal point forecasts
- The consistency and asymptotic normality of the kernel type expectile regression estimator for functional data
- Backtesting extreme value theory models of expected shortfall
- A parsimonious parametric model for generating margin requirements for futures
- Density forecast of financial returns using decomposition and maximum entropy
- Characterizing the optimal solutions to the isotonic regression problem for identifiable functionals
- Optimal estimation of the supremum and occupation times of a self-similar Lévy process
- Probabilistic sensitivity measures as information value
- Marked self-exciting point process modelling of information diffusion on twitter
- Distributional transforms, probability distortions, and their applications
- Inventory -- forecasting: mind the gap
- Feature extraction for functional time series: theory and application to NIR spectroscopy data
- Optimal trading policies for wind energy producer
- Point forecasting and forecast evaluation with generalized Huber loss
- Joint inference on extreme expectiles for multivariate heavy-tailed distributions
- Forecasting intra-individual changes of affective states taking into account inter-individual differences using intensive longitudinal data from a university student dropout study in math
- Risks in emerging markets equities: time-varying versus spatial risk analysis
- Focusing on regions of interest in forecast evaluation
- On the indirect elicitability of the mode and modal interval
- Functional prediction of intraday cumulative returns
- Bayesian spline method for assessing extreme loads on wind turbines
- Why scoring functions cannot assess tail properties
- Semi-parametric Bayesian tail risk forecasting incorporating realized measures of volatility
- Backtesting expected shortfall and beyond
- Econometric modeling of risk measures: a selective review of the recent literature
- Quantile evaluation, sensitivity to bracketing, and sharing business payoffs
- Uniform calibration tests for forecasting systems with small lead time
- Scoring predictions at extreme quantiles
- Measuring and adjusting for overconfidence
- Semiparametric empirical best prediction for small area estimation of unemployment indicators
- A theory for measures of tail risk
- On the properties of the lambda value at risk: robustness, elicitability and consistency
- Backtesting VaR and expectiles with realized scores
- Order-sensitivity and equivariance of scoring functions
- Comparing Predictive Accuracy in the Presence of a Loss Function Shape Parameter
- Distributionally robust reinsurance with expectile
- Realized Quantiles*
- Estimation combining unbiased and possibly biased estimators
- Combined Density Nowcasting in an Uncertain Economic Environment
- Predicting the Global Minimum Variance Portfolio
- Proper Scoring Rules for Evaluating Density Forecasts with Asymmetric Loss Functions
- Multistep quantile forecasts for supply chain and logistics operations: bootstrapping, the GARCH model and quantile regression based approaches
- Retire: robust expectile regression in high dimensions
- Volatility prediction comparison via robust volatility proxies: an empirical deviation perspective
- Backtesting Systemic Risk Forecasts Using Multi-Objective Elicitability
- Estimation of complier expected shortfall treatment effects with a binary instrumental variable
- Diversification quotients based on VaR and ES
- Powerful Backtests for Historical Simulation Expected Shortfall Models
- Pinball boosting of regression quantiles
- Truncated generalized extreme value distribution-based ensemble model output statistics model for calibration of wind speed ensemble forecasts
- The dependence structure between equity and foreign exchange markets and tail risk forecasts of foreign investments
- Mean-variance-VaR portfolios: MIQP formulation and performance analysis
- Lasso regularization within the LocalGLMnet architecture
- Flexible Expectile Regression in Reproducing Kernel Hilbert Spaces
- Inter-order relations between equivalence for \(L_p\)-quantiles of the Student's \(t\) distribution
- Risk quantization by magnitude and propensity
- An expectile computation cookbook
- Inference for joint quantile and expected shortfall regression
- GARCH-UGH: a bias-reduced approach for dynamic extreme value-at-risk estimation in financial time series
- Parametric quantile autoregressive moving average models with exogenous terms
- Random distributions via sequential quantile array
- Bias-reduced and variance-corrected asymptotic Gaussian inference about extreme expectiles
- Model selection with Pearson's correlation, concentration and Lorenz curves under autocalibration
- Calibrating Distribution Models from PELVE
- Sequential testing for elicitable functionals via supermartingales
- Convex and Lorenz orders under balance correction in nonlife insurance pricing: review and new developments
- Two-stage weighted least squares estimator of the conditional mean of observation-driven time series models
- On duration effects in non-life insurance pricing
- Regression diagnostics meets forecast evaluation: conditional calibration, reliability diagrams, and coefficient of determination
- Multiple measures realized GARCH models
- A Beaufort Scale of Predictability
- Safe machine learning
- Memoryless sequences for general losses
- Elicitability of Instance and Object Ranking
- Robust estimation in regression and classification methods for large dimensional data
- Model selection with Gini indices under auto-calibration
- Using proxies to improve forecast evaluation
- Reconstruction of late Holocene climate based on tree growth and mechanistic hierarchical models
- Better the devil you know: improved forecasts from imperfect models
- Testing for auto-calibration with Lorenz and concentration curves
- Local bias adjustment, duration-weighted probabilities, and automatic construction of tariff cells
- Evaluating Forecasts for High-Impact Events Using Transformed Kernel Scores
This page was built for publication: Making and evaluating point forecasts
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q91134)
