Distributionally-robust machine learning using locally differentially-private data

DOI10.1007/S11590-021-01765-6MaRDI QIDQ2128767zbMATH OpenOpenAlexFDO

Authors Farhad Farokhi

Publication date 22 April 2022

Published in Optimization Letters (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2006.13488

zbMATH Keywords

distributionally-robust optimizationlocal differential privacymachine learningregularizationWasserstein distance

Mathematics Subject Classification ID

Robustness in mathematical programming (90C17) Applications of mathematical programming (90C90)

Abstract: We consider machine learning, particularly regression, using locally-differentially private datasets. The Wasserstein distance is used to define an ambiguity set centered at the empirical distribution of the dataset corrupted by local differential privacy noise. The ambiguity set is shown to contain the probability distribution of unperturbed, clean data. The radius of the ambiguity set is a function of the privacy budget, spread of the data, and the size of the problem. Hence, machine learning with locally-differentially private datasets can be rewritten as a distributionally-robust optimization. For general distributions, the distributionally-robust optimization problem can relaxed as a regularized machine learning problem with the Lipschitz constant of the machine learning model as a regularizer. For linear and logistic regression, this regularizer is the dual norm of the model parameters. For Gaussian data, the distributionally-robust optimization problem can be solved exactly to find an optimal regularizer. This approach results in an entirely new regularizer for training linear regression models. Training with this novel regularizer can be posed as a semi-definite program. Finally, the performance of the proposed distributionally-robust machine learning training is demonstrated on practical datasets.

Recommendations

• Differentially private empirical risk minimization
• Privacy aware learning
• Differentially private distributed logistic regression with the objective function perturbation
• An optimal (, )-differentially private learning of distributed deep fuzzy models
• What can we learn privately?

Cites work

• A class of Wasserstein metrics for probability distributions
• Computationally tractable counterparts of distributionally robust constraints on risk measures
• Data-driven distributionally robust optimization using the Wasserstein metric: performance guarantees and tractable reformulations
• Distributionally robust inverse covariance estimation: the Wasserstein shrinkage estimator
• Distributionally robust optimization under moment uncertainty with application to data-driven problems
• Empirical risk minimization for heavy-tailed losses
• Extremal mechanisms for local differential privacy
• scientific article; zbMATH DE number 3134565 (Why is no real title available?)
• scientific article; zbMATH DE number 107482 (Why is no real title available?)
• Limit laws of the empirical Wasserstein distance: Gaussian distributions
• Neural Network Learning
• Robust optimization
• Semidefinite Programming
• Some asymptotic theory for the bootstrap
• The algorithmic foundations of differential privacy
• Theory of Cryptography

Cited in

(3)

• An optimal (, )-differentially private learning of distributed deep fuzzy models
• Differentially Private Distributed Learning
• scientific article; zbMATH DE number 7415075 (Why is no real title available?)

Describes a project that uses

Uses Software

• RAPPOR

This page was built for publication: Distributionally-robust machine learning using locally differentially-private data

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2128767)