Making and evaluating point forecasts
From MaRDI portal
Publication:91134
DOI10.1198/JASA.2011.R10138zbMATH Open1232.62028arXiv0912.0902OpenAlexW2075965721MaRDI QIDQ91134FDOQ91134
Authors: Tilmann Gneiting, Tilmann Gneiting 
Publication date: June 2011
Published in: Journal of the American Statistical Association (Search for Journal in Brave)
Abstract: Typically, point forecasting methods are compared and assessed by means of an error measure or scoring function, such as the absolute error or the squared error. The individual scores are then averaged over forecast cases, to result in a summary measure of the predictive performance, such as the mean absolute error or the (root) mean squared error. I demonstrate that this common practice can lead to grossly misguided inferences, unless the scoring function and the forecasting task are carefully matched. Effective point forecasting requires that the scoring function be specified ex ante, or that the forecaster receives a directive in the form of a statistical functional, such as the mean or a quantile of the predictive distribution. If the scoring function is specified ex ante, the forecaster can issue the optimal point forecast, namely, the Bayes rule. If the forecaster receives a directive in the form of a functional, it is critical that the scoring function be consistent for it, in the sense that the expected score is minimized when following the directive. A functional is elicitable if there exists a scoring function that is strictly consistent for it. Expectations, ratios of expectations and quantiles are elicitable. For example, a scoring function is consistent for the mean functional if and only if it is a Bregman function. It is consistent for a quantile if and only if it is generalized piecewise linear. Similar characterizations apply to ratios of expectations and to expectiles. Weighted scoring functions are consistent for functionals that adapt to the weighting in peculiar ways. Not all functionals are elicitable; for instance, conditional value-at-risk is not, despite its popularity in quantitative finance.
Full work available at URL: https://arxiv.org/abs/0912.0902
Recommendations
- Of quantiles and expectiles: consistent scoring functions, Choquet representations and forecast rankings. With discussion and authors' reply
- Order-sensitivity and equivariance of scoring functions
- Higher order elicitability and Osband's principle
- Point forecasting and forecast evaluation with generalized Huber loss
- Forecast evaluation of quantiles, prediction intervals, and other set-valued functionals
Bayes ruleBregman functionconditional value-at-risk (CVaR)decision theoryelicitabilityexpectilemeanmedianmodeproper scoring rulequantilesstatistical functionals
Decision theory (91B06) Bayesian problems; characterization of Bayes procedures (62C10) Statistical decision theory (62C99)
Cited In (only showing first 100 items - show all)
- Generalization error for Tweedie models: decomposition and error reduction with bagging
- A survey of Bayesian predictive methods for model assessment, selection and comparison
- Relative bound and asymptotic comparison of expectile with respect to expected shortfall
- Of quantiles and expectiles: consistent scoring functions, Choquet representations and forecast rankings. With discussion and authors' reply
- Expectile depth: theory and computation for bivariate datasets
- Dynamic semiparametric models for expected shortfall (and value-at-risk)
- On the \(L_p\)-quantiles for the Student \(t\) distribution
- On elicitable risk measures
- Extreme M-quantiles as risk measures: from \(L^{1}\) to \(L^{p}\) optimization
- Estimation and testing for spatially indexed curves with application to ionospheric and magnetic field trends
- Generalized quantiles as risk measures
- Semi-parametric estimation of multivariate extreme expectiles
- Optimal investment under VaR-regulation and minimum insurance
- Time-varying sparsity in dynamic regression models
- Forecast dominance testing via sign randomization
- Risk measures with the CxLS property
- Dynamic quantile function models
- Hierarchical probabilistic forecasting of electricity demand with smart meter data
- Optimal reinsurance with expectile
- Multivariate geometric expectiles
- Distortion riskmetrics on general spaces
- Risk bounds for factor models
- What can we learn from telematics car driving data: a survey
- Quantifying market risk with value-at-risk or expected shortfall? -- Consequences for capital requirements and model risk
- On a capital allocation by minimization of some risk indicators
- Scenario aggregation method for portfolio expectile optimization
- Expectile asymptotics
- Coherence and elicitability
- The mode functional is not elicitable
- A dynamic nonstationary spatio-temporal model for short term prediction of precipitation
- Aggregation-robustness and model uncertainty of regulatory risk measures
- Comparison of value-at-risk models using the MCS approach
- The role of the information set for forecasting -- with applications to risk management
- Comments on: Space-time wind speed forecasting for improved power system dispatch
- Comments on: Space-time wind speed forecasting for improved power system dispatch
- Dominating countably many forecasts
- Superquantile regression with applications to buffered reliability, uncertainty quantification, and conditional value-at-risk
- Robust and Pareto optimality of insurance contracts
- Percentage and relative error measures in forecast evaluation
- Optimal robust insurance with a finite uncertainty set
- Efficient regularized isotonic regression with application to gene-gene interaction search
- Uncertainty quantification in complex simulation models using ensemble copula coupling
- Where does the tail begin? An approach based on scoring rules
- Regulatory arbitrage of risk measures
- Forecaster's dilemma: extreme events and forecast evaluation
- A joint quantile and expected shortfall regression framework
- Projecting the future burden of cancer: Bayesian age-period-cohort analysis with integrated nested Laplace approximations
- Probabilistic wind speed forecasting on a grid based on ensemble model output statistics
- Bayesian realized-GARCH models for financial tail risk forecasting incorporating the two-sided Weibull distribution
- Using conditional kernel density estimation for wind power density forecasting
- How superadditive can a risk measure be?
- Understanding predictive information criteria for Bayesian models
- CRPS Learning
- Elicitability and identifiability of set-valued measures of systemic risk
- Higher order elicitability and Osband's principle
- On the measurement of economic tail risk
- ASYMPTOTIC EXPANSIONS OF GENERALIZED QUANTILES AND EXPECTILES FOR EXTREME RISKS
- scoringfunctions
- Computing near-optimal value-at-risk portfolios using integer programming techniques
- Encoded value-at-risk: a machine learning approach for portfolio risk measurement
- Assessing point forecast accuracy by stochastic loss distance
- Imputation Scores
- Extreme-quantile tracking for financial time series
- On multivariate extensions of the conditional value-at-risk measure
- Elicitable distortion risk measures: a concise proof
- Bayesian structured additive distributional regression with an application to regional income inequality in Germany
- Tail risk inference via expectiles in heavy-tailed time series
- A note on the use of empirical AUC for evaluating probabilistic forecasts
- Verification of internal risk measure estimates
- Phenomenological forecasting of disease incidence using heteroskedastic Gaussian processes: a dengue case study
- Performance measurement with expectiles
- Multivariate extensions of expectiles risk measures
- Conditional expectiles, time consistency and mixture convexity properties
- Asymptotic stability of empirical processes and related functionals
- Quantile-based risk sharing
- Conditionally Elicitable Dynamic Risk Measures for Deep Reinforcement Learning
- Marked self-exciting point process modelling of information diffusion on twitter
- Uniform calibration tests for forecasting systems with small lead time
- Scoring predictions at extreme quantiles
- A parsimonious parametric model for generating margin requirements for futures
- Tail asymptotics of generalized deflated risks with insurance applications
- Semi-parametric Bayesian tail risk forecasting incorporating realized measures of volatility
- Optimal estimation of the supremum and occupation times of a self-similar Lévy process
- On the properties of the lambda value at risk: robustness, elicitability and consistency
- Semiparametric empirical best prediction for small area estimation of unemployment indicators
- Grouped multivariate and functional time series forecasting: an application to annuity pricing
- Point forecasting and forecast evaluation with generalized Huber loss
- Joint inference on extreme expectiles for multivariate heavy-tailed distributions
- Risks in emerging markets equities: time-varying versus spatial risk analysis
- On the indirect elicitability of the mode and modal interval
- Bayesian spline method for assessing extreme loads on wind turbines
- Estimating and backtesting risk under heavy tails
- Probabilistic sensitivity measures as information value
- Order-sensitivity and equivariance of scoring functions
- Joint generalized quantile and conditional tail expectation regression for insurance risk analysis
- Backtesting VaR and expectiles with realized scores
- Forecasting intra-individual changes of affective states taking into account inter-individual differences using intensive longitudinal data from a university student dropout study in math
- Functional prediction of intraday cumulative returns
- Scoring interval forecasts: equal-tailed, shortest, and modal interval
- A relative error-based approach for variable selection
This page was built for publication: Making and evaluating point forecasts
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q91134)
