Differentially Private Distributed Learning

Yaqin Zhou () and Shaojie Tang ()
Additional contact information
Yaqin Zhou: Nanyang Technological University, Singapore 639798, Singapore
Shaojie Tang: University of Texas at Dallas, Richardson, Texas 75080

INFORMS Journal on Computing, 2020, vol. 32, issue 3, 779-789

Abstract: The rich data used to train learning models increasingly tend to be distributed and private. It is important to efficiently perform learning tasks without compromising individual users’ privacy even considering untrusted learning applications and, furthermore, understand how privacy-preservation mechanisms impact the learning process. To address the problem, we design a differentially private distributed algorithm based on the stochastic variance reduced gradient (SVRG) algorithm, which prevents the learning server from accessing and inferring private training data with a theoretical guarantee. We quantify the impact of the adopted privacy-preservation measure on the learning process in terms of convergence rate, by which it indicates noises added at each gradient update results in a bounded deviation from the optimum. To further evaluate the impact on the trained models, we compare the proposed algorithm with SVRG and stochastic gradient descent using logistic regression and neural nets. The experimental results on benchmark data sets show that the proposed algorithm has minor impact on the accuracy of trained models under a moderate amount of privacy budget.

Keywords: differential privacy; distributed deep learning; SVRG; optimization (search for similar items in EconPapers)
Date: 2020
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://doi.org/10.1287/ijoc.2019.0912 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:orijoc:v:32:y:3:i:2020:p:779-789

Access Statistics for this article

More articles in INFORMS Journal on Computing from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().