Stochastic gradient descent with differentially private updates

2013 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2013 - Proceedings

Volumn , Issue , 2013, Pages 245-248

  Song, Shuang ^a   Chaudhuri, Kamalika ^a   Sarwate, Anand D ^b  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b TOYOTA TECHNOLOGICAL INSTITUTE AT CHICAGO   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BATCH SIZES; DIFFERENTIAL PRIVACIES; IMPROVE PERFORMANCE; LARGE DATASETS; SENSITIVE DATAS; STOCHASTIC GRADIENT DESCENT; STOCHASTIC GRADIENT METHODS;

DATA PROCESSING;

TIME VARYING NETWORKS;

EID: 84897680504     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/GlobalSIP.2013.6736861     Document Type: Conference Paper

References (19)

1. C. Dwork, F. McSherry, K. Nissim, and A. Smith, "Calibrating noise to sensitivity in private data analysis," in Theory of Cryptography, vol. 3876, March 2006, pp. 265-284.
2. A. Rakhlin, O. Shamir, and K. Sridharan, "Making gradient descent optimal for strongly convex stochastic optimization," ArXiV, Tech. Rep. arXiv:1109. 5647 [cs. LG], 2012.
3. S. Shalev-Shwartz, Y. Singer, N. Srebro, and A. Cotter, "Pegasos: Primal Estimated sub-GrAdient SOlver for SVM," Mathematical Programming, Series B, vol. 127, no. 1, pp. 3-30, 2011.
4. A. Cotter, O. Shamir, N. Srebro, and K. Sridharan, "Better mini-batch algorithms via accelerated gradient methods," in Adv. NIPS 24, 2011, pp. 1647-1655.
5. O. Dekel, R. Gilad-Bachrach, O. Shamir, and L. Xiao, "Optimal distributed online prediction using mini-batches," J. Mach. Learn. Res., vol. 13, pp. 165-202, 2012.
6. M. Takác, A. Bijral, P. Richtárik, and N. Srebro, "Mini-batch primal and dual methods for SVMs," in Proc. ICML, 2013.
7. C. Dwork and A. Smith, "Differential privacy for statistics: What we know and what we want to learn," Journal of Privacy and Confidentiality, vol. 1, no. 2, pp. 135-154, 2009.
8. A. D. Sarwate and K. Chaudhuri, "Signal processing and machine learning with differential privacy: Algorithms and challenges for continuous data," IEEE Sig. Proc. Mag., vol. 30, no. 5, pp. 86-94, September 2013.
9. J. Le Ny and G. J. Pappas, "Differentially private filtering," in Proc. CDC, Maui, HI, USA, December 2012, pp. 3398-3403.
10. -, "Differentially private Kalman filtering," in Proc. Allerton, Monticello, IL, USA, October 2012, pp. 1618-1625.
11. K. Chaudhuri, C. Monteleoni, and A. D. Sarwate, "Differentially private empirical risk minimization," J. Mach. Learn. Res., vol. 12, pp. 1069-1109, March 2011.
12. B. I. P. Rubinstein, P. L. Bartlett, L. Huang, and N. Taft, "Learning in a large function space: Privacy-preserving mechanisms for SVM learning," Journal of Privacy and Confidentiality, vol. 4, no. 1, pp. 65-100, 2012.
13. J. Duchi, M. Jordan, and M. Wainwright, "Privacy aware learning," in Adv. NIPS 25, 2012, pp. 1439-1447.
14. P. Jain, P. Kothari, and A. Thakurta, "Differentially private online learning," in Proc. COLT, June 2012.
15. F. McSherry, "Privacy integrated queries: an extensible platform for privacy-preserving data analysis," Communications of the ACM, vol. 53, no. 9, pp. 89-97, September 2010.
16. O. Williams and F. McSherry, "Probabilistic inference and differential privacy," in Adv. NIPS 23, J. Lafferty, C. K. I. Williams, J. Shawe-Taylor, R. S. Zemel, and A. Culotta, Eds., 2010, pp. 2451-245.
17. A. Blum, C. Dwork, F. McSherry, and K. Nissim, "Practical privacy: The SuLQ framework," in Proc. PODS. New York, NY, USA: ACM, 2005, pp. 128-138.
18. S. Rosset and A. Inger, "KDD-cup 99: knowledge discovery in a charitable organization's donor database," SIGKDD Explor. Newsl., vol. 1, no. 2, pp. 85-90, Jan. 2000.
19. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition," in Proc. IEEE, 1998, pp. 2278-2324.