Order-sensitivity and equivariance of scoring functions
According to the authors, the general decision-theoretical framework consists of the following four elements: \par $\bullet$ observed realization $\{y_1,\ldots,y_n\}$, $n\geqslant 1$, of a time series $\{Y_t\}_{t\in\mathbb{N}}$, taking values in an observation domain $O$ with a $\sigma$-algebra $\mathcal{O}$, \par $\bullet$ a family $\mathcal{F}$ of probability distributions on $(O,\mathcal{O})$, containing distributions of $Y_t$, \par $\bullet$ forecasts $\{x_1^{(i)},\ldots, x_n^{(i)}\}$, $i\in\{1,\ldots,m\}$, competing forecasters taking values in an action domain $A\subset\mathbb{R}^k$, $k\geqslant 1$, \par $\bullet$ a scoring function $S:A\times O\rightarrow\mathbb{R}$. \par The scoring function is assumed to have the negative meaning. If forecaster reports the quantity $x\in A$ and $y\in O$ appears, is assigned penalty $S(x,y)$. Different forecasts ($i\in\{1,\ldots,m\}$) are compared by values of \textit{realized scores} \[ \overline{S}_n^{(i)}=\sum\limits_{i=1}^{n}S(x_t^{(i)},y_t). \] A forecast is supposed to be better if it's \textit{realized score} $\overline{S}_n$ is lower. \par To give a guidance on the choice of a suitable scoring function, the authors of the paper consider various properties of the scoring functions and introduce new quality criteria for such functions.
- Making and evaluating point forecasts
- Higher order elicitability and Osband's principle
- Of quantiles and expectiles: consistent scoring functions, Choquet representations and forecast rankings. With discussion and authors' reply
- On scoring ordered classifications
- Should Scoring Rules be “Effective”?
- Elicitability and identifiability of set-valued measures of systemic risk
- On the indirect elicitability of the mode and modal interval
- Forecast evaluation of quantiles, prediction intervals, and other set-valued functionals
- On the elicitability of range value at risk
- Measurability of functionals and of ideal point forecasts
- Point forecasting and forecast evaluation with generalized Huber loss
- Isotonic regression for elicitable functionals and their Bayes risk
- Why scoring functions cannot assess tail properties
- Deep quantile and deep composite triplet regression
- Higher order elicitability and Osband's principle
- On scoring ordered classifications
- Assessing point forecast accuracy by stochastic loss distance
- Of quantiles and expectiles: consistent scoring functions, Choquet representations and forecast rankings. With discussion and authors' reply
- Bias-corrected score decomposition for generalized quantiles
- Consistent scoring functions for quantiles
- Regression diagnostics meets forecast evaluation: conditional calibration, reliability diagrams, and coefficient of determination
- Sensitivity measures based on scoring functions
- Osband's Principle for Identification Functions
- Backtesting Systemic Risk Forecasts Using Multi-Objective Elicitability
- Estimation and backtesting of risk measures with emphasis on distortion risk measures
- Proper scoring rules, gradients, divergences, and entropies for paths and time series
- Elicitability and identifiability of tail risk measures
- Making and evaluating point forecasts
This page was built for publication: Order-sensitivity and equivariance of scoring functions
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2414489)
