Inferactive data analysis

DOI10.1111/SJOS.12425MaRDI QIDQ5108979zbMATH OpenOpenAlexWikidataFDO

Authors

Publication date 7 May 2020

Published in Scandinavian Journal of Statistics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1707.06692

zbMATH Keywords

randomizationselective inferenceinteractive data analysisconfirmatory and exploratory data analysis

Mathematics Subject Classification ID

Computational methods for problems pertaining to statistics (62-08) Parametric hypothesis testing (62F03) Bayesian inference (62F15) Applications of statistics to biology and medical sciences; meta analysis (62P10) Statistical aspects of big data and data science (62R07)

Abstract: We describe inferactive data analysis, so-named to denote an interactive approach to data analysis with an emphasis on inference after data analysis. Our approach is a compromise between Tukey's exploratory (roughly speaking "model free") and confirmatory data analysis (roughly speaking classical and "model based"), also allowing for Bayesian data analysis. We view this approach as close in spirit to current practice of applied statisticians and data scientists while allowing frequentist guarantees for results to be reported in the scientific literature, or Bayesian results where the data scientist may choose the statistical model (and hence the prior) after some initial exploratory analysis. While this approach to data analysis does not cover every scenario, and every possible algorithm data scientists may use, we see this as a useful step in concrete providing tools (with frequentist statistical guarantees) for current data scientists. The basis of inference we use is selective inference [Lee et al., 2016, Fithian et al., 2014], in particular its randomized form [Tian and Taylor, 2015a]. The randomized framework, besides providing additional power and shorter confidence intervals, also provides explicit forms for relevant reference distributions (up to normalization) through the {em selective sampler} of Tian et al. [2016]. The reference distributions are constructed from a particular conditional distribution formed from what we call a DAG-DAG -- a Data Analysis Generative DAG. As sampling conditional distributions in DAGs is generally complex, the selective sampler is crucial to any practical implementation of inferactive data analysis. Our principal goal is in reviewing the recent developments in selective inference as well as describing the general philosophy of selective inference.

Recommendations

• Statistical learning and selective inference
• Selective inference with a randomized response
• Valid post-selection inference
• Preserving statistical validity in adaptive data analysis (extended abstract)
• Integrative methods for post-selection inference under convex constraints

Cited in

(14)

• Preserving statistical validity in adaptive data analysis (extended abstract)
• Valid Inference Corrected for Outlier Removal
• Combining inductive and deductive tools for data analysis
• The reusable holdout: preserving validity in adaptive data analysis
• Statistical inference for exploratory data analysis and model diagnostics
• A selective survey of selective inference
• Post-selection inference via algorithmic stability
• Statistical learning and selective inference
• Carving model-free inference
• scientific article; zbMATH DE number 4159846 (Why is no real title available?)
• Integrative methods for post-selection inference under convex constraints
• Exploratory inference: localizing relevant effects with confidence
• Testing retrospective hypotheses
• Concept-analytical data evaluation

This page was built for publication: Inferactive data analysis

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5108979)