Of quantiles and expectiles: consistent scoring functions, Choquet representations and forecast rankings. With discussion and authors' reply
From MaRDI portal
Publication:5378146
quantiledecision theoryexpectileprobability forecastforecast rankingeconomic utilityChoquet representationpoint forecastconsistent scoring functionelicitableorder sensitivity
Point estimation (62F10) Nonparametric estimation (62G05) Statistical ranking and selection procedures (62F07) Inference from stochastic processes and prediction (62M20) Decision theory (91B06) General considerations in statistical decision theory (62C05) Research exposition (monographs, survey articles) pertaining to statistics (62-02)
Abstract: In the practice of point prediction, it is desirable that forecasters receive a directive in the form of a statistical functional, such as the mean or a quantile of the predictive distribution. When evaluating and comparing competing forecasts, it is then critical that the scoring function used for these purposes be consistent for the functional at hand, in the sense that the expected score is minimized when following the directive. We show that any scoring function that is consistent for a quantile or an expectile functional, respectively, can be represented as a mixture of extremal scoring functions that form a linearly parameterized family. Scoring functions for the mean value and probability forecasts of binary events constitute important examples. The quantile and expectile functionals along with the respective extremal scoring functions admit appealing economic interpretations in terms of thresholds in decision making. The Choquet type mixture representations give rise to simple checks of whether a forecast dominates another in the sense that it is preferable under any consistent scoring function. In empirical settings it suffices to compare the average scores for only a finite number of extremal elements. Plots of the average scores with respect to the extremal scoring functions, which we call Murphy diagrams, permit detailed comparisons of the relative merits of competing forecasts.
Recommendations
Cited in
(48)- Making and evaluating point forecasts
- Calibrating sufficiently
- Bias-corrected score decomposition for generalized quantiles
- Receiver operating characteristic (ROC) movies, universal ROC (UROC) curves, and coefficient of predictive ability (CPA)
- Comparative evaluation of point process forecasts
- Range-based risk measures and their applications
- Regression diagnostics meets forecast evaluation: conditional calibration, reliability diagrams, and coefficient of determination
- Scalable spatio‐temporal Bayesian analysis of high‐dimensional electroencephalography data
- Testing for auto-calibration with Lorenz and concentration curves
- Generic Conditions for Forecast Dominance
- Comparing Possibly Misspecified Forecasts
- Forecast dominance testing via sign randomization
- A discrete density approach to Bayesian quantile and expectile regression with discrete responses
- Tail expectile process and risk assessment
- Point forecasting and forecast evaluation with generalized Huber loss
- Joint inference on extreme expectiles for multivariate heavy-tailed distributions
- Using proxies to improve forecast evaluation
- Order-sensitivity and equivariance of scoring functions
- ExpectHill estimation, extreme risk and heavy tails
- Backtesting VaR and expectiles with realized scores
- Scoring interval forecasts: equal-tailed, shortest, and modal interval
- Forecast evaluation of quantiles, prediction intervals, and other set-valued functionals
- Comparing Predictive Accuracy in the Presence of a Loss Function Shape Parameter
- The local linear functional \(k\)NN estimator of the conditional expectile: uniform consistency in number of neighbors
- Isotonic regression for elicitable functionals and their Bayes risk
- An elastic-net penalized expectile regression with applications
- The \(k\)th power expectile regression
- Forecaster's dilemma: extreme events and forecast evaluation
- A joint quantile and expected shortfall regression framework
- Density Regression with Conditional Support Points
- Why scoring functions cannot assess tail properties
- Quantile evaluation, sensitivity to bracketing, and sharing business payoffs
- Sensitivity measures based on scoring functions
- On automatic bias reduction for extreme expectile estimation
- On the elicitability of range value at risk
- Higher order elicitability and Osband's principle
- Elicitability and identifiability of set-valued measures of systemic risk
- A hierarchical Bayesian model for predicting ecological interactions using scaled evolutionary relationships
- On the nonparametric estimation of the functional expectile regression
- Encoded value-at-risk: a machine learning approach for portfolio risk measurement
- PDE-regularised spatial quantile regression
- Properization: constructing proper scoring rules via Bayes acts
- The functional \(k\mathrm{NN}\) estimator of the conditional expectile: uniform consistency in number of neighbors
- Characterizing the optimal solutions to the isotonic regression problem for identifiable functionals
- Simultaneous Semiparametric Estimation of Clustering and Regression
- Estimating value-at-risk and expected shortfall using the intraday low and range data
- Consistent scoring functions for quantiles
- The consistency and asymptotic normality of the kernel type expectile regression estimator for functional data
This page was built for publication: Of quantiles and expectiles: consistent scoring functions, Choquet representations and forecast rankings. With discussion and authors' reply
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5378146)
