Inferactive data analysis

DOI10.1111/SJOS.12425zbMATH Open1446.62340arXiv1707.06692OpenAlexW2982593149WikidataQ126976400 ScholiaQ126976400MaRDI QIDQ5108979FDOQ5108979

Authors:

Publication date: 7 May 2020

Published in: Scandinavian Journal of Statistics (Search for Journal in Brave)

Abstract: We describe inferactive data analysis, so-named to denote an interactive approach to data analysis with an emphasis on inference after data analysis. Our approach is a compromise between Tukey's exploratory (roughly speaking "model free") and confirmatory data analysis (roughly speaking classical and "model based"), also allowing for Bayesian data analysis. We view this approach as close in spirit to current practice of applied statisticians and data scientists while allowing frequentist guarantees for results to be reported in the scientific literature, or Bayesian results where the data scientist may choose the statistical model (and hence the prior) after some initial exploratory analysis. While this approach to data analysis does not cover every scenario, and every possible algorithm data scientists may use, we see this as a useful step in concrete providing tools (with frequentist statistical guarantees) for current data scientists. The basis of inference we use is selective inference [Lee et al., 2016, Fithian et al., 2014], in particular its randomized form [Tian and Taylor, 2015a]. The randomized framework, besides providing additional power and shorter confidence intervals, also provides explicit forms for relevant reference distributions (up to normalization) through the {em selective sampler} of Tian et al. [2016]. The reference distributions are constructed from a particular conditional distribution formed from what we call a DAG-DAG -- a Data Analysis Generative DAG. As sampling conditional distributions in DAGs is generally complex, the selective sampler is crucial to any practical implementation of inferactive data analysis. Our principal goal is in reviewing the recent developments in selective inference as well as describing the general philosophy of selective inference.

Full work available at URL: https://arxiv.org/abs/1707.06692

Recommendations

• Statistical learning and selective inference
• Selective inference with a randomized response
• Valid post-selection inference
• Preserving statistical validity in adaptive data analysis (extended abstract)
• Integrative methods for post-selection inference under convex constraints

zbMATH Keywords

randomizationselective inferenceinteractive data analysisconfirmatory and exploratory data analysis

Mathematics Subject Classification ID

Computational methods for problems pertaining to statistics (62-08) Parametric hypothesis testing (62F03) Bayesian inference (62F15) Applications of statistics to biology and medical sciences; meta analysis (62P10) Statistical aspects of big data and data science (62R07)

Cited In (14)

• Preserving statistical validity in adaptive data analysis (extended abstract)
• Valid Inference Corrected for Outlier Removal
• Combining inductive and deductive tools for data analysis
• The reusable holdout: preserving validity in adaptive data analysis
• Statistical inference for exploratory data analysis and model diagnostics
• A selective survey of selective inference
• Post-selection inference via algorithmic stability
• Statistical learning and selective inference
• Carving model-free inference
• Title not available (Why is that?)
• Integrative methods for post-selection inference under convex constraints
• Exploratory inference: localizing relevant effects with confidence
• Testing retrospective hypotheses
• Concept-analytical data evaluation

This page was built for publication: Inferactive data analysis

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5108979)