Any Inspection is Manipulable

From MaRDI portal





The calibration tests treated in the literature are pure: After any history it is known what specific event (for instance, `rain', `foggy') is being considered. Here, mixed calibration tests are introduced, where, after each history, a randomly chosen event is checked. A mixed test checks whether the average prediction of these randomly chosen events is asymptotically equal to the frequency of their realizations. Like a deterministic test, a mixed calibration test is also type \(I\)-error free. The main result of this paper shows that there is no mixed calibration test that rejects any inaccurate forecast. That is, for any test of this kind, there exists a forecasting scheme that is always consistent with the realized events. A constructive method to produce this scheme is provided as well. Another interpretation of the main result involves a forecaster and an inspector. The inspector randomly chooses a calibration test according to a known distribution. It is shown that a forecaster, without knowing anything regarding the way states are selected can pass almost every test chosen by the inspector, independently of the sequence of realizations. In other words, the forecaster has a forecasting method that is consistent with the observed facts in the meaning induced by almost all tests. Instead of randomly choosing one test, as discussed previously, the inspector can employ a set of tests simultaneously. The main result implies that if the inspector employs countably many calibration tests, for instance all those computable by a Turing machine, a forecaster can, likewise, pass them all simultaneously. It means that a consultant or a scientist (e.g., a weather forecaster) who knows the (countable) set of tests his model needs to pass, can manipulate. That is, no matter what the sequence of realizations is, his manipulative forecast will pass all the tests. The proof of the main result relies on an extension of Blackwell's approachability theorem to games with payoffs in infinite dimensional spaces.











This page was built for publication: Any Inspection is Manipulable

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4531035)