Deep learning with differential privacy

Proceedings of the ACM Conference on Computer and Communications Security

Volumn 24-28-October-2016, Issue , 2016, Pages 308-318

  Abadi, Martín ^a   McMahan, H Brendan ^a   Chu, Andy ^a   Mironov, Ilya ^a   Zhang, Li ^a   Goodfellow, Ian ^a,b   Talwar, Kunal ^a  

^a GOOGLE INC   (United States)

^b OpenAI LLC   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; BUDGET CONTROL; COMPLEX NETWORKS; LEARNING SYSTEMS;

ALGORITHMIC TECHNIQUES; DEEP NEURAL NETWORKS; DIFFERENTIAL PRIVACIES; MACHINE LEARNING TECHNIQUES; PRIVATE INFORMATION; SENSITIVE INFORMATIONS; SOFTWARE COMPLEXITY; TRAINING EFFICIENCY;

COST BENEFIT ANALYSIS;

EID: 84995527907     PISSN: 15437221     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2976749.2978318     Document Type: Conference Paper

References (58)

1. CIFAR-10 and CIFAR-100 datasets. www.cs.toronto.edu/~kriz/cifar.html.
2. TensorFlow convolutional neural networks tutorial. www.tensorflow.org/tutorials/deep cnn.
3. TensorFlow: Large-scale machine learning on heterogeneous systems, 2015. Software available from tensorflow.org.
4. M. Abadi, A. Chu, I. Goodfellow, H. B. McMahan, I. Mironov, K. Talwar, and L. Zhang. Deep learning with differential privacy. CoRR, abs/1607.00133, 2016.
5. C. C. Aggarwal. On k-anonymity and the curse of dimensionality. In VLDB, pages 901-909, 2005.
6. R. Agrawal and R. Srikant. Privacy-preserving data mining. In SIGMOD, pages 439-450. ACM, 2000.
7. R. Bassily, K. Nissim, A. Smith, T. Steinke, U. Stemmer, and J. Ullman. Algorithmic stability for adaptive data analysis. In STOC, pages 1046-1059. ACM, 2016.
8. R. Bassily, A. D. Smith, and A. Thakurta. Private empirical risk minimization: Efficient algorithms and tight error bounds. In FOCS, pages 464-473. IEEE, 2014.
9. A. Beimel, H. Brenner, S. P. Kasiviswanathan, and K. Nissim. Bounds on the sample complexity for private learning and private data release. Machine Learning, 94(3):401-437, 2014.
10. J. Brickell and V. Shmatikov. The cost of privacy: Destruction of data-mining utility in anonymized data publishing. In KDD, pages 70-78. ACM, 2008.
11. M. Bun and T. Steinke. Concentrated differential privacy: Simplifications, extensions, and lower bounds. CoRR, abs/1605.02065, 2016.
12. K. Chaudhuri, C. Monteleoni, and A. D. Sarwate. Differentially private empirical risk minimization. J. Machine Learning Research, 12:1069-1109, 2011.
13. R. Collobert, K. Kavukcuoglu, and C. Farabet. Torch7: A Matlab-like environment for machine learning. In BigLearn, NIPS Workshop, number EPFL-CONF-192376, 2011.
14. D. D. Cox and N. Pinto. Beyond simple features: A large-scale feature search approach to unconstrained face recognition. In FG 2011, pages 8-15. IEEE, 2011.
15. D. Silver, A. Huang, C. J. Maddison et al. Mastering the game of Go with deep neural networks and tree search. Nature, 529(7587):484-489, 2016.
16. A. Daniely, R. Frostig, and Y. Singer. Toward deeper understanding of neural networks: The power of initialization and a dual view on expressivity. CoRR, abs/1602.05897, 2016.
17. J. Duchi, E. Hazan, and Y. Singer. Adaptive subgradient methods for online learning and stochastic optimization. J. Machine Learning Research, 12:2121-2159, July 2011.
18. C. Dwork. A firm foundation for private data analysis. Commun. ACM, 54(1):86-95, Jan. 2011.
19. C. Dwork, K. Kenthapadi, F. McSherry, I. Mironov, and M. Naor. Our data, ourselves: Privacy via distributed noise generation. In EUROCRYPT, pages 486-503. Springer, 2006.
20. C. Dwork and J. Lei. Differential privacy and robust statistics. In STOC, pages 371-380. ACM, 2009.
21. C. Dwork, F. McSherry, K. Nissim, and A. Smith. Calibrating noise to sensitivity in private data analysis. In TCC, pages 265-284. Springer, 2006.
22. C. Dwork and A. Roth. The algorithmic foundations of differential privacy. Foundations and Trends in Theoretical Computer Science, 9 (3-4) :211-407, 2014.
23. C. Dwork and G. N. Rothblum. Concentrated differential privacy. CoRR, abs/1603.01887, 2016.
24. C. Dwork, G. N. Rothblum, and S. Vadhan. Boosting and differential privacy. In FOCS, pages 51-60. IEEE, 2010.
25. C. Dwork, K. Talwar, A. Thakurta, and L. Zhang. Analyze Gauss: Optimal bounds for privacy-preserving principal component analysis. In STOC, pages 11-20. ACM, 2014.
26. M. Fredrikson, S. Jha, and T. Ristenpart. Model inversion attacks that exploit confidence information and basic countermeasures. In CCS, pages 1322-1333. ACM, 2015.
27. I. Goodfellow. Efficient per-example gradient computations. CoRR, abs/1510.01799v2, 2015.
28. B. Graham. Fractional max-pooling. CoRR, abs/1412.6071, 2014.
29. A. Gupta, K. Ligett, F. McSherry, A. Roth, and K. Talwar. Differentially private combinatorial optimization. In SODA, pages 1106-1125, 2010.
30. K. He, X. Zhang, S. Ren, and J. Sun. Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification. In ICCV, pages 1026-1034. IEEE, 2015.
31. R. Ierusalimschy, L. H. de Figueiredo, and W. Filho. Lua-an extensible extension language. Software: Practice and Experience, 26(6):635-652, 1996.
32. K. Jarrett, K. Kavukcuoglu, M. Ranzato, and Y. LeCun. What is the best multi-stage architecture for object recognition? In ICCV, pages 2146-2153. IEEE, 2009.
33. R. Johnson and T. Zhang. Accelerating stochastic gradient descent using predictive variance reduction. In NIPS, pages 315-323, 2013.
34. P. Kairouz, S. Oh, and P. Viswanath. The composition theorem for differential privacy. In ICML, pages 1376-1385. ACM, 2015.
35. S. P. Kasiviswanathan, H. K. Lee, K. Nissim, S. Raskhodnikova, and A. D. Smith. What can we learn privately? SIAM J. Comput., 40(3):793-826, 2011.
36. D. Kifer, A. D. Smith, and A. Thakurta. Private convex optimization for empirical risk minimization with applications to high-dimensional regression. In COLT, pages 25.1-25.40, 2012.
37. A. Krizhevsky, I. Sutskever, and G. E. Hinton. ImageNet classification with deep convolutional neural networks. In NIPS, pages 1097-1105, 2012.
38. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 1998.
39. Y. Lindell and B. Pinkas. Privacy preserving data mining. In CRYPTO, pages 36-54. Springer, 2000.
40. C. J. Maddison, A. Huang, I. Sutskever, and D. Silver. Move evaluation in Go using deep convolutional neural networks. In ICLR, 2015.
41. F. McSherry and I. Mironov. Differentially private recommender systems: Building privacy into the Netflix Prize contenders. In KDD, pages 627-636. ACM, 2009.
42. F. D. McSherry. Privacy integrated queries: An extensible platform for privacy-preserving data analysis. In SIGMOD, pages 19-30. ACM, 2009.
43. T. Mikolov, K. Chen, G. Corrado, and J. Dean. Efficient estimation of word representations in vector space. CoRR abs/1301.3781, 2013
44. I. Mironov. Rényi differential privacy. Private communication, 2016.
45. Y. Nesterov. Introductory Lectures on Convex Optimization. A Basic Course. Springer, 2004.
46. J. Pennington, R. Socher, and C. D. Manning. GloVe: Global vectors for word representation. In EMNLP, pages 1532-1543, 2014.
47. N. Phan, Y. Wang, X. Wu, and D. Dou. Differential privacy preservation for deep auto-encoders: an application of human behavior prediction. In AAAI, pages 1309-1316, 2016.
48. N. Pinto, Z. Stone, T. E. Zickler, and D. Cox. Scaling up biologically-inspired computer vision: A case study in unconstrained face recognition on Facebook. In CVPR, pages 35-42. IEEE, 2011.
49. R. M. Rogers, A. Roth, J. Ullman, and S. P. Vadhan. Privacy odometers and filters: Pay-as-you-go composition. CoRR, abs/1605.08294, 2016.
50. D. E. Rumelhart, G. E. Hinton, and R. J. Williams. Learning representations by back-propagating errors. Nature, 323:533-536, Oct. 1986.
51. A. Saxe, P. W. Koh, Z. Chen, M. Bhand, B. Suresh, and A. Ng. On random weights and unsupervised feature learning. In ICML, pages 1089-1096. ACM, 2011.
52. R. Shokri and V. Shmatikov. Privacy-preserving deep learning. In CCS, pages 1310-1321. ACM, 2015.
53. S. Song, K. Chaudhuri, and A. Sarwate. Stochastic gradient descent with differentially private updates. In GlobalSIP Conference, 2013.
54. L. Sweeney. k-anonymity: A model for protecting privacy. International J. of Uncertainty, Fuzziness and Knowledge-Based Systems, 10(05):557-570, 2002.
55. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich. Going deeper with convolutions. In CVPR, pages 1-9. IEEE, 2015.
56. S. Tu, R. Roelofs, S. Venkataraman, and B. Recht. Large scale kernel learning using block coordinate descent. CoRR, abs/1602.05310, 2016.
57. O. Vinyals, L. Kaiser, T. Koo, S. Petrov, I. Sutskever, and G. E. Hinton. Grammar as a foreign language. In NIPS, pages 2773-2781, 2015.
58. X. Wu, A. Kumar, K. Chaudhuri, S. Jha, and J. F. Naughton. Differentially private stochastic gradient descent for in-RDBMS analytics. CoRR, abs/1606.04722, 2016.