Of quantiles and expectiles: consistent scoring functions, Choquet representations and forecast rankings. With discussion and authors' reply
From MaRDI portal
Publication:5378146
quantiledecision theoryexpectileprobability forecastforecast rankingeconomic utilityChoquet representationpoint forecastconsistent scoring functionelicitableorder sensitivity
Point estimation (62F10) Nonparametric estimation (62G05) Statistical ranking and selection procedures (62F07) Inference from stochastic processes and prediction (62M20) Decision theory (91B06) General considerations in statistical decision theory (62C05) Research exposition (monographs, survey articles) pertaining to statistics (62-02)
Abstract: In the practice of point prediction, it is desirable that forecasters receive a directive in the form of a statistical functional, such as the mean or a quantile of the predictive distribution. When evaluating and comparing competing forecasts, it is then critical that the scoring function used for these purposes be consistent for the functional at hand, in the sense that the expected score is minimized when following the directive. We show that any scoring function that is consistent for a quantile or an expectile functional, respectively, can be represented as a mixture of extremal scoring functions that form a linearly parameterized family. Scoring functions for the mean value and probability forecasts of binary events constitute important examples. The quantile and expectile functionals along with the respective extremal scoring functions admit appealing economic interpretations in terms of thresholds in decision making. The Choquet type mixture representations give rise to simple checks of whether a forecast dominates another in the sense that it is preferable under any consistent scoring function. In empirical settings it suffices to compare the average scores for only a finite number of extremal elements. Plots of the average scores with respect to the extremal scoring functions, which we call Murphy diagrams, permit detailed comparisons of the relative merits of competing forecasts.
Recommendations
Cited in
(48)- Comparing Predictive Accuracy in the Presence of a Loss Function Shape Parameter
- The local linear functional \(k\)NN estimator of the conditional expectile: uniform consistency in number of neighbors
- On the elicitability of range value at risk
- Estimating value-at-risk and expected shortfall using the intraday low and range data
- Forecaster's dilemma: extreme events and forecast evaluation
- An elastic-net penalized expectile regression with applications
- Scoring interval forecasts: equal-tailed, shortest, and modal interval
- Isotonic regression for elicitable functionals and their Bayes risk
- Forecast evaluation of quantiles, prediction intervals, and other set-valued functionals
- PDE-regularised spatial quantile regression
- Forecast dominance testing via sign randomization
- The consistency and asymptotic normality of the kernel type expectile regression estimator for functional data
- Calibrating sufficiently
- On the nonparametric estimation of the functional expectile regression
- Characterizing the optimal solutions to the isotonic regression problem for identifiable functionals
- A hierarchical Bayesian model for predicting ecological interactions using scaled evolutionary relationships
- Regression diagnostics meets forecast evaluation: conditional calibration, reliability diagrams, and coefficient of determination
- Point forecasting and forecast evaluation with generalized Huber loss
- Joint inference on extreme expectiles for multivariate heavy-tailed distributions
- Making and evaluating point forecasts
- Using proxies to improve forecast evaluation
- Testing for auto-calibration with Lorenz and concentration curves
- A joint quantile and expected shortfall regression framework
- Elicitability and identifiability of set-valued measures of systemic risk
- Higher order elicitability and Osband's principle
- ExpectHill estimation, extreme risk and heavy tails
- Simultaneous Semiparametric Estimation of Clustering and Regression
- Why scoring functions cannot assess tail properties
- Bias-corrected score decomposition for generalized quantiles
- Scalable spatio‐temporal Bayesian analysis of high‐dimensional electroencephalography data
- On automatic bias reduction for extreme expectile estimation
- The functional \(k\mathrm{NN}\) estimator of the conditional expectile: uniform consistency in number of neighbors
- Tail expectile process and risk assessment
- Quantile evaluation, sensitivity to bracketing, and sharing business payoffs
- Comparative evaluation of point process forecasts
- A discrete density approach to Bayesian quantile and expectile regression with discrete responses
- Sensitivity measures based on scoring functions
- Consistent scoring functions for quantiles
- Receiver operating characteristic (ROC) movies, universal ROC (UROC) curves, and coefficient of predictive ability (CPA)
- Generic Conditions for Forecast Dominance
- Range-based risk measures and their applications
- Density Regression with Conditional Support Points
- Comparing Possibly Misspecified Forecasts
- Backtesting VaR and expectiles with realized scores
- Order-sensitivity and equivariance of scoring functions
- Encoded value-at-risk: a machine learning approach for portfolio risk measurement
- The \(k\)th power expectile regression
- Properization: constructing proper scoring rules via Bayes acts
This page was built for publication: Of quantiles and expectiles: consistent scoring functions, Choquet representations and forecast rankings. With discussion and authors' reply
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5378146)
