Making and evaluating point forecasts
From MaRDI portal
Publication:91134
DOI10.1198/JASA.2011.R10138zbMATH Open1232.62028arXiv0912.0902OpenAlexW2075965721MaRDI QIDQ91134FDOQ91134
Authors: Tilmann Gneiting, Tilmann Gneiting 
Publication date: June 2011
Published in: Journal of the American Statistical Association (Search for Journal in Brave)
Abstract: Typically, point forecasting methods are compared and assessed by means of an error measure or scoring function, such as the absolute error or the squared error. The individual scores are then averaged over forecast cases, to result in a summary measure of the predictive performance, such as the mean absolute error or the (root) mean squared error. I demonstrate that this common practice can lead to grossly misguided inferences, unless the scoring function and the forecasting task are carefully matched. Effective point forecasting requires that the scoring function be specified ex ante, or that the forecaster receives a directive in the form of a statistical functional, such as the mean or a quantile of the predictive distribution. If the scoring function is specified ex ante, the forecaster can issue the optimal point forecast, namely, the Bayes rule. If the forecaster receives a directive in the form of a functional, it is critical that the scoring function be consistent for it, in the sense that the expected score is minimized when following the directive. A functional is elicitable if there exists a scoring function that is strictly consistent for it. Expectations, ratios of expectations and quantiles are elicitable. For example, a scoring function is consistent for the mean functional if and only if it is a Bregman function. It is consistent for a quantile if and only if it is generalized piecewise linear. Similar characterizations apply to ratios of expectations and to expectiles. Weighted scoring functions are consistent for functionals that adapt to the weighting in peculiar ways. Not all functionals are elicitable; for instance, conditional value-at-risk is not, despite its popularity in quantitative finance.
Full work available at URL: https://arxiv.org/abs/0912.0902
Recommendations
- Of quantiles and expectiles: consistent scoring functions, Choquet representations and forecast rankings. With discussion and authors' reply
- Order-sensitivity and equivariance of scoring functions
- Higher order elicitability and Osband's principle
- Point forecasting and forecast evaluation with generalized Huber loss
- Forecast evaluation of quantiles, prediction intervals, and other set-valued functionals
Bayes ruleBregman functionconditional value-at-risk (CVaR)decision theoryelicitabilityexpectilemeanmedianmodeproper scoring rulequantilesstatistical functionals
Decision theory (91B06) Bayesian problems; characterization of Bayes procedures (62C10) Statistical decision theory (62C99)
Cited In (only showing first 100 items - show all)
- Conditionally Elicitable Dynamic Risk Measures for Deep Reinforcement Learning
- Marked self-exciting point process modelling of information diffusion on twitter
- Uniform calibration tests for forecasting systems with small lead time
- Scoring predictions at extreme quantiles
- A parsimonious parametric model for generating margin requirements for futures
- Tail asymptotics of generalized deflated risks with insurance applications
- Semi-parametric Bayesian tail risk forecasting incorporating realized measures of volatility
- Optimal estimation of the supremum and occupation times of a self-similar Lévy process
- On the properties of the lambda value at risk: robustness, elicitability and consistency
- Semiparametric empirical best prediction for small area estimation of unemployment indicators
- Grouped multivariate and functional time series forecasting: an application to annuity pricing
- Point forecasting and forecast evaluation with generalized Huber loss
- Joint inference on extreme expectiles for multivariate heavy-tailed distributions
- Risks in emerging markets equities: time-varying versus spatial risk analysis
- On the indirect elicitability of the mode and modal interval
- Bayesian spline method for assessing extreme loads on wind turbines
- Estimating and backtesting risk under heavy tails
- Probabilistic sensitivity measures as information value
- Order-sensitivity and equivariance of scoring functions
- Joint generalized quantile and conditional tail expectation regression for insurance risk analysis
- Backtesting VaR and expectiles with realized scores
- Forecasting intra-individual changes of affective states taking into account inter-individual differences using intensive longitudinal data from a university student dropout study in math
- Functional prediction of intraday cumulative returns
- Scoring interval forecasts: equal-tailed, shortest, and modal interval
- A relative error-based approach for variable selection
- Forecast evaluation of quantiles, prediction intervals, and other set-valued functionals
- Using the Bayesian Shtarkov solution for predictions
- Focusing on regions of interest in forecast evaluation
- Econometric modeling of risk measures: a selective review of the recent literature
- Isotonic regression for elicitable functionals and their Bayes risk
- Optimal insurance design in the presence of exclusion clauses
- Why scoring functions cannot assess tail properties
- Measuring and adjusting for overconfidence
- Testing the reliability of forecasting systems
- Quantile evaluation, sensitivity to bracketing, and sharing business payoffs
- Bayes risk, elicitability, and the Expected Shortfall
- On the elicitability of range value at risk
- General truthfulness characterizations via convex analysis
- Density forecast of financial returns using decomposition and maximum entropy
- Optimal trading policies for wind energy producer
- Robust VIF regression with application to variable selection in large data sets
- Characterizing the optimal solutions to the isotonic regression problem for identifiable functionals
- Distributional transforms, probability distortions, and their applications
- Inventory -- forecasting: mind the gap
- Feature extraction for functional time series: theory and application to NIR spectroscopy data
- Estimating value-at-risk and expected shortfall using the intraday low and range data
- Spatio-temporal short-term wind forecast: a calibrated regime-switching method
- Optimal operational service levels in vendor managed inventory contracts -- an exact approach
- Measurability of functionals and of ideal point forecasts
- Backtesting expected shortfall and beyond
- A theory for measures of tail risk
- The consistency and asymptotic normality of the kernel type expectile regression estimator for functional data
- Backtesting extreme value theory models of expected shortfall
- Generalization error for Tweedie models: decomposition and error reduction with bagging
- A survey of Bayesian predictive methods for model assessment, selection and comparison
- Relative bound and asymptotic comparison of expectile with respect to expected shortfall
- Of quantiles and expectiles: consistent scoring functions, Choquet representations and forecast rankings. With discussion and authors' reply
- Expectile depth: theory and computation for bivariate datasets
- Dynamic semiparametric models for expected shortfall (and value-at-risk)
- On the \(L_p\)-quantiles for the Student \(t\) distribution
- On elicitable risk measures
- Extreme M-quantiles as risk measures: from \(L^{1}\) to \(L^{p}\) optimization
- Estimation and testing for spatially indexed curves with application to ionospheric and magnetic field trends
- Generalized quantiles as risk measures
- Semi-parametric estimation of multivariate extreme expectiles
- Optimal investment under VaR-regulation and minimum insurance
- Time-varying sparsity in dynamic regression models
- Forecast dominance testing via sign randomization
- Risk measures with the CxLS property
- Dynamic quantile function models
- Hierarchical probabilistic forecasting of electricity demand with smart meter data
- Optimal reinsurance with expectile
- Multivariate geometric expectiles
- Distortion riskmetrics on general spaces
- Risk bounds for factor models
- What can we learn from telematics car driving data: a survey
- Quantifying market risk with value-at-risk or expected shortfall? -- Consequences for capital requirements and model risk
- On a capital allocation by minimization of some risk indicators
- Scenario aggregation method for portfolio expectile optimization
- Expectile asymptotics
- Coherence and elicitability
- The mode functional is not elicitable
- A dynamic nonstationary spatio-temporal model for short term prediction of precipitation
- Aggregation-robustness and model uncertainty of regulatory risk measures
- Comparison of value-at-risk models using the MCS approach
- The role of the information set for forecasting -- with applications to risk management
- Comments on: Space-time wind speed forecasting for improved power system dispatch
- Comments on: Space-time wind speed forecasting for improved power system dispatch
- Dominating countably many forecasts
- Superquantile regression with applications to buffered reliability, uncertainty quantification, and conditional value-at-risk
- Robust and Pareto optimality of insurance contracts
- Percentage and relative error measures in forecast evaluation
- Optimal robust insurance with a finite uncertainty set
- Efficient regularized isotonic regression with application to gene-gene interaction search
- Uncertainty quantification in complex simulation models using ensemble copula coupling
- Where does the tail begin? An approach based on scoring rules
- Regulatory arbitrage of risk measures
- Forecaster's dilemma: extreme events and forecast evaluation
- A joint quantile and expected shortfall regression framework
- Projecting the future burden of cancer: Bayesian age-period-cohort analysis with integrated nested Laplace approximations
This page was built for publication: Making and evaluating point forecasts
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q91134)
