Making and evaluating point forecasts
From MaRDI portal
Abstract: Typically, point forecasting methods are compared and assessed by means of an error measure or scoring function, such as the absolute error or the squared error. The individual scores are then averaged over forecast cases, to result in a summary measure of the predictive performance, such as the mean absolute error or the (root) mean squared error. I demonstrate that this common practice can lead to grossly misguided inferences, unless the scoring function and the forecasting task are carefully matched. Effective point forecasting requires that the scoring function be specified ex ante, or that the forecaster receives a directive in the form of a statistical functional, such as the mean or a quantile of the predictive distribution. If the scoring function is specified ex ante, the forecaster can issue the optimal point forecast, namely, the Bayes rule. If the forecaster receives a directive in the form of a functional, it is critical that the scoring function be consistent for it, in the sense that the expected score is minimized when following the directive. A functional is elicitable if there exists a scoring function that is strictly consistent for it. Expectations, ratios of expectations and quantiles are elicitable. For example, a scoring function is consistent for the mean functional if and only if it is a Bregman function. It is consistent for a quantile if and only if it is generalized piecewise linear. Similar characterizations apply to ratios of expectations and to expectiles. Weighted scoring functions are consistent for functionals that adapt to the weighting in peculiar ways. Not all functionals are elicitable; for instance, conditional value-at-risk is not, despite its popularity in quantitative finance.
Recommendations
- Of quantiles and expectiles: consistent scoring functions, Choquet representations and forecast rankings. With discussion and authors' reply
- Order-sensitivity and equivariance of scoring functions
- Higher order elicitability and Osband's principle
- Point forecasting and forecast evaluation with generalized Huber loss
- Forecast evaluation of quantiles, prediction intervals, and other set-valued functionals
Cited in
(only showing first 100 items - show all)- Imputation Scores
- Tail risk inference via expectiles in heavy-tailed time series
- Forecast dominance testing via sign randomization
- Phenomenological forecasting of disease incidence using heteroskedastic Gaussian processes: a dengue case study
- Quantifying market risk with value-at-risk or expected shortfall? -- Consequences for capital requirements and model risk
- Using the Bayesian Shtarkov solution for predictions
- A relative error-based approach for variable selection
- Robust and Pareto optimality of insurance contracts
- Focusing on regions of interest in forecast evaluation
- Extreme M-quantiles as risk measures: from \(L^{1}\) to \(L^{p}\) optimization
- Marked self-exciting point process modelling of information diffusion on twitter
- Optimal investment under VaR-regulation and minimum insurance
- Computing near-optimal value-at-risk portfolios using integer programming techniques
- Forecaster's dilemma: extreme events and forecast evaluation
- Conditional expectiles, time consistency and mixture convexity properties
- A dynamic nonstationary spatio-temporal model for short term prediction of precipitation
- Robust VIF regression with application to variable selection in large data sets
- A survey of Bayesian predictive methods for model assessment, selection and comparison
- A parsimonious parametric model for generating margin requirements for futures
- Dynamic semiparametric models for expected shortfall (and value-at-risk)
- Generalized quantiles as risk measures
- Elicitability and identifiability of set-valued measures of systemic risk
- On the indirect elicitability of the mode and modal interval
- Probabilistic sensitivity measures as information value
- Expectile depth: theory and computation for bivariate datasets
- Semi-parametric estimation of multivariate extreme expectiles
- Joint generalized quantile and conditional tail expectation regression for insurance risk analysis
- Scoring interval forecasts: equal-tailed, shortest, and modal interval
- Forecast evaluation of quantiles, prediction intervals, and other set-valued functionals
- On the elicitability of range value at risk
- Inventory -- forecasting: mind the gap
- Feature extraction for functional time series: theory and application to NIR spectroscopy data
- Optimal operational service levels in vendor managed inventory contracts -- an exact approach
- Measurability of functionals and of ideal point forecasts
- Uniform calibration tests for forecasting systems with small lead time
- Scoring predictions at extreme quantiles
- Characterizing the optimal solutions to the isotonic regression problem for identifiable functionals
- Point forecasting and forecast evaluation with generalized Huber loss
- Optimal estimation of the supremum and occupation times of a self-similar Lévy process
- Joint inference on extreme expectiles for multivariate heavy-tailed distributions
- Risks in emerging markets equities: time-varying versus spatial risk analysis
- Estimating and backtesting risk under heavy tails
- What can we learn from telematics car driving data: a survey
- Performance measurement with expectiles
- Forecasting intra-individual changes of affective states taking into account inter-individual differences using intensive longitudinal data from a university student dropout study in math
- Isotonic regression for elicitable functionals and their Bayes risk
- Encoded value-at-risk: a machine learning approach for portfolio risk measurement
- On automatic bias reduction for extreme expectile estimation
- Random distributions via sequential quantile array
- Properization: constructing proper scoring rules via Bayes acts
- Virtual historical simulation for estimating the conditional VaR of large portfolios
- Dual representation of expectile-based expected shortfall and its properties
- Dominating countably many forecasts
- Superquantile regression with applications to buffered reliability, uncertainty quantification, and conditional value-at-risk
- Estimating value-at-risk and expected shortfall using the intraday low and range data
- Spatio-temporal short-term wind forecast: a calibrated regime-switching method
- Predicting paleoclimate from compositional data using multivariate Gaussian process inverse prediction
- Estimation combining unbiased and possibly biased estimators
- Multistep quantile forecasts for supply chain and logistics operations: bootstrapping, the GARCH model and quantile regression based approaches
- Econometric modeling of risk measures: a selective review of the recent literature
- Semiparametric empirical best prediction for small area estimation of unemployment indicators
- Backtesting VaR and expectiles with realized scores
- Why scoring functions cannot assess tail properties
- Measuring and adjusting for overconfidence
- On multivariate extensions of the conditional value-at-risk measure
- Elicitable distortion risk measures: a concise proof
- Grouped multivariate and functional time series forecasting: an application to annuity pricing
- Risk bounds for factor models
- Tail asymptotics of generalized deflated risks with insurance applications
- Optimal insurance design in the presence of exclusion clauses
- On the \(L_p\)-quantiles for the Student \(t\) distribution
- Order-sensitivity and equivariance of scoring functions
- Optimal robust insurance with a finite uncertainty set
- Efficient regularized isotonic regression with application to gene-gene interaction search
- Bayesian spline method for assessing extreme loads on wind turbines
- Extreme-quantile tracking for financial time series
- The role of the information set for forecasting -- with applications to risk management
- Time-varying sparsity in dynamic regression models
- Verification of internal risk measure estimates
- Understanding predictive information criteria for Bayesian models
- Asymptotic stability of empirical processes and related functionals
- The consistency and asymptotic normality of the kernel type expectile regression estimator for functional data
- Deep quantile and deep composite triplet regression
- Oil-price density forecasts of US GDP
- Density forecast of financial returns using decomposition and maximum entropy
- Efficient estimation of financial risk by regressing the quantiles of parametric distributions: an application to CARR models
- Coherence and elicitability
- Risk measures with the CxLS property
- A Beaufort Scale of Predictability
- On a capital allocation by minimization of some risk indicators
- Scenario aggregation method for portfolio expectile optimization
- Expectile asymptotics
- Higher order elicitability and Osband's principle
- On the measurement of economic tail risk
- How superadditive can a risk measure be?
- Estimation and testing for spatially indexed curves with application to ionospheric and magnetic field trends
- Semi-parametric Bayesian tail risk forecasting incorporating realized measures of volatility
- On the properties of the lambda value at risk: robustness, elicitability and consistency
- Optimal reinsurance with expectile
- Multivariate geometric expectiles
This page was built for publication: Making and evaluating point forecasts
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q91134)
