Distributed Empirical Risk Minimization With Differential Privacy

DOI10.1016/J.AUTOMATICA.2024.111514MaRDI QIDQ6427145zbMATH OpenWikidataFDO

Authors Changxin Liu, Karl Henrik Johansson, Yang Shi

Publication date 21 February 2023

Full work available at URL https://doi.org/10.1016/J.AUTOMATICA.2024.111514

Mathematics Subject Classification ID

Mathematical programming (90Cxx) Information and communication theory, circuits (94-XX)

Abstract: This work studies the distributed empirical risk minimization (ERM) problem under differential privacy (DP) constraint. Standard distributed algorithms achieve DP typically by perturbing all local subgradients with noise, leading to significantly degenerated utility. To tackle this issue, we develop a class of private distributed dual averaging (DDA) algorithms, which activates a fraction of nodes to perform optimization. Such subsampling procedure provably amplifies the DP guarantee, thereby achieving an equivalent level of DP with reduced noise. We prove that the proposed algorithms have utility loss comparable to centralized private algorithms for both general and strongly convex problems. When removing the noise, our algorithm attains the optimal O(1/t) convergence for non-smooth stochastic optimization. Finally, experimental results on two benchmark datasets are given to verify the effectiveness of the proposed algorithms.

Recommendations

• A differentially private distributed optimization method for constrained optimization
• Differentially Private Accelerated Optimization Algorithms
• Differentially private distributed logistic regression with the objective function perturbation
• Differentially Private Distributed Learning
• A novel adaptive differential privacy algorithm for empirical risk minimization

Cited in

(7)

• Differentially Private Accelerated Optimization Algorithms
• Empirical risk minimization in the non-interactive local model of differential privacy
• A differentially private distributed optimization method for constrained optimization
• Optimal algorithms for differentially private stochastic monotone variational inequalities and saddle-point problems
• Privacy-preserving Frank-Wolfe on shuffle model
• Ef21-RR: fast \(\mathcal{O} (1 / T)\) rate for non-convex federated optimization with error feedback
• Federated cubic regularized Newton learning with sparsification-amplified differential privacy

This page was built for publication: Distributed Empirical Risk Minimization With Differential Privacy

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6427145)