Statistic selection and MCMC for differentially private Bayesian estimation

DOI10.1007/S11222-022-10129-8zbMATH Open1495.62008arXiv2203.13377OpenAlexW4292176483MaRDI QIDQ2172115FDOQ2172115

Authors: Barış Alparslan, Sinan Yıldırım 

Publication date: 15 September 2022

Published in: Statistics and Computing (Search for Journal in Brave)

Abstract: This paper concerns differentially private Bayesian estimation of the parameters of a population distribution, when a statistic of a sample from that population is shared in noise to provide differential privacy. This work mainly addresses two problems: (1) What statistic of the sample should be shared privately? For the first question, i.e., the one about statistic selection, we promote using the Fisher information. We find out that, the statistic that is most informative in a non-privacy setting may not be the optimal choice under the privacy restrictions. We provide several examples to support that point. We consider several types of data sharing settings and propose several Monte Carlo-based numerical estimation methods for calculating the Fisher information for those settings. The second question concerns inference: (2) Based on the shared statistics, how could we perform effective Bayesian inference? We propose several Markov chain Monte Carlo (MCMC) algorithms for sampling from the posterior distribution of the parameter given the noisy statistic. The proposed MCMC algorithms can be preferred over one another depending on the problem. For example, when the shared statistics is additive and added Gaussian noise, a simple Metropolis-Hasting algorithm that utilizes the central limit theorem is a decent choice. We propose more advanced MCMC algorithms for several other cases of practical relevance. Our numerical examples involve comparing several candidate statistics to be shared privately. For each statistic, we perform Bayesian estimation based on the posterior distribution conditional on the privatized version of that statistic. We demonstrate that, the relative performance of a statistic, in terms of the mean squared error of the Bayesian estimator based on the corresponding privatized statistic, is adequately predicted by the Fisher information of the privatized statistic.

Full work available at URL: https://arxiv.org/abs/2203.13377

Recommendations

• Differential privacy for Bayesian inference through posterior sampling
• Exact MCMC with differentially private moves. Exact MCMC with differentially private moves, revisiting the penalty algorithm in a data privacy framework
• Robust and private Bayesian inference
• Privacy-preserving statistical estimation with optimal convergence rates
• Minimax Optimal Procedures for Locally Private Estimation

zbMATH Keywords

Markov chain Monte CarloBayesian statisticsdifferential privacyFisher informationstatistic selection

Mathematics Subject Classification ID

Computational methods for problems pertaining to statistics (62-08) Bayesian inference (62F15) Numerical analysis or methods applied to Markov chains (65C40)

Cites Work

• Differential Privacy
• The pseudo-marginal approach for efficient Monte Carlo computations
• Particle Markov Chain Monte Carlo Methods
• Constructing Summary Statistics for Approximate Bayesian Computation: Semi-Automatic Approximate Bayesian Computation
• Asymptotic methods in statistical decision theory
• The correlated pseudomarginal method
• Differential Privacy: A Survey of Results
• Title not available (Why is that?)
• The algorithmic foundations of differential privacy
• Differential privacy and robust statistics
• Concentrated differential privacy: simplifications, extensions, and lower bounds
• Privacy-preserving statistical estimation with optimal convergence rates
• Exact MCMC with differentially private moves. Exact MCMC with differentially private moves, revisiting the penalty algorithm in a data privacy framework
• Privacy-preserving parametric inference: a case for robust statistics
• Gaussian Differential Privacy

Cited In (2)

• Exact MCMC with differentially private moves. Exact MCMC with differentially private moves, revisiting the penalty algorithm in a data privacy framework
• Differential privacy for Bayesian inference through posterior sampling

Uses Software

• PrivateLR

This page was built for publication: Statistic selection and MCMC for differentially private Bayesian estimation

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2172115)