Differentially private empirical risk minimization

Journal of Machine Learning Research

Volumn 12, Issue , 2011, Pages 1069-1109

  Chaudhuri, Kamalika ^a   Monteleoni, Claire ^b   Sarwate, Anand D ^c  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b Och Spine at New York Presbyterian Hospitals   (United States)

^c University of California at San Diego   (United States)

Author keywords

Classification; Empirical risk minimization; Logistic regression; Optimization; Privacy; Support vector machines

Indexed keywords

BENCHMARK DATA; CLASSIFICATION; DIFFERENTIABILITY; DIFFERENTIAL PRIVACIES; EMPIRICAL RISK MINIMIZATION; FINANCIAL RECORDS; GENERALIZATION BOUND; LEARNING PERFORMANCE; LOGISTIC REGRESSION; LOGISTIC REGRESSIONS; MACHINE LEARNING ALGORITHMS; NONLINEAR KERNELS; OBJECTIVE FUNCTIONS; OUTPUT PERTURBATIONS; PERSONAL DATA; PRIVACY; PRIVACY PRESERVING; REGULARIZER; THEORETICAL RESULT; TRAINING PROCESS;

DATA PRIVACY; MACHINE DESIGN; OPTIMIZATION; PERTURBATION TECHNIQUES; REGRESSION ANALYSIS; SUPPORT VECTOR MACHINES;

LEARNING ALGORITHMS;

EID: 79955858775     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (50)

1. R. Agrawal and R. Srikant. Privacy-preserving data mining. SIGMOD Record, 29(2): 439-450, 2000.
2. A. Asuncion and D.J. Newman. UCI Machine Learning Repository. University of California, Irvine, School of Information and Computer Sciences, 2007. URL http://www. ics.uci.edu/~mlearn/MLRepository.html.
3. L. Backstrom, C. Dwork, and J. Kleinberg. Wherefore art thou R3579X? Anonymized social networks, hidden patterns, and structural steganography. In Proceedings of the 16th International World Wide Web Conference, 2007.
4. K. Ball. An elementary introduction to modern convex geometry. In S. Levy, editor, Flavors of Geometry, volume 31 of Mathematical Sciences Research Institute Publications, pages 1-58. Cambridge University Press, 1997.
5. B. Barak, K. Chaudhuri, C. Dwork, S. Kale, F. McSherry, and K. Talwar. Privacy, accuracy, and consistency too: a holistic solution to contingency table release. In Proceedings of the 26th ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems (PODS), pages 273-282, 2007. (Pubitemid 47620902)
6. A. Beimel, S. P. Kasiviswanathan, and K. Nissim. Bounds on the sample complexity for private learning and private data release. In Proceedings of the 7th IACR Theory of Cryptography Conference (TCC), pages 437-454, 2010.
7. P Billingsley. Probability and measure. A Wiley-Interscience publication. Wiley, New York [u.a.], 3. ed edition, 1995. ISBN 0471007102.
8. A. Blum, K. Ligett, and A. Roth. A learning theory approach to non-interactive database privacy. In R. E. Ladner and C. Dwork, editors, Proceedings of the 40th ACM Symposium on Theory of Computing (STOC), pages 609-618. ACM, 2008. ISBN 978-1-60558-047-0.
9. S. Boyd and L. Vandenberghe. Convex Optimization. Cambridge University Press, Cambridge, UK, 2004.
10. O. Chapelle. Training a support vector machine in the primal. Neural Computation, 19(5):1155-1178, May 2007.
11. K. Chaudhuri and N. Mishra. When random sampling preserves privacy. In Cynthia Dwork, editor, CRYPTO, volume 4117 of Lecture Notes in Computer Science, pages 198-213. Springer, 2006. ISBN 3-540-37432-9. (Pubitemid 44532117)
12. K. Chaudhuri and C. Monteleoni. Privacy-preserving logistic regression. In Proceedings of the 22nd Annual Conference on Neural Information Processing Systems (NIPS), 2008.
13. C. Dwork. Differential privacy. In M. Bugliesi, B. Preneel, V. Sassone, and I. Wegener, editors, ICALP (2), volume 4052 of Lecture Notes in Computer Science, pages 1-12. Springer, 2006. ISBN 3-540-35907-9. (Pubitemid 44113232)
14. C. Dwork and J. Lei. Differential privacy and robust statistics. In Proceedings of the 41st ACM Symposium on Theory of Computing (STOC), 2009.
15. C. Dwork, K. Kenthapadi, F. McSherry, I. Mironov, and M. Naor. Our data, ourselves: Privacy via distributed noise generation. In Serge Vaudenay, editor, EUROCRYPT, volume 4004 of Lecture Notes in Computer Science, pages 486-503. Springer, 2006a. (Pubitemid 44072257)
16. C. Dwork, F. McSherry, K. Nissim, and A. Smith. Calibrating noise to sensitivity in private data analysis. In 3rd IACR Theory of Cryptography Conference., pages 265-284, 2006b. (Pubitemid 43979853)
17. A. Evfimievski, J. Gehrke, and R. Srikant. Limiting privacy breaches in privacy preserving data mining. In Proceedings of the 22nd ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems (PODS), pages 211-222, 2003.
18. S.R. Ganta, S.P. Kasiviswanathan, and A. Smith. Composition attacks and auxiliary information in data privacy. In Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), pages 265-273, 2008.
19. A. Gupta, K. Ligett, F. McSherry, A. Roth, and K. Talwar. Differentially private approximation algorithms. In Proceedings of the 2010 ACM-SIAM Symposium on Discrete Algorithms (SODA), 2010.
20. T. Hastie, S. Rosset, R. Tibshirani, and J. Zhu. The entire regularization path for the support vector machine. Journal of Machine Learning Research, 5:1391-1415, 2004.
21. S. Hettich and S.D. Bay. The UCI KDD Archive. University of California, Irvine, Department of Information and Computer Science, 1999. URL http://kdd.ics.uci.edu.
22. N. Homer, S. Szelinger, M. Redman, D. Duggan, W. Tembe, J. Muehling, J. V. Pearson, D. A. Stephan, S. F. Nelson, and D. W. Craig. Resolving individuals contributing trace amounts of dna to highly complex mixtures using high-density snp genotyping microarrays. PLoS Genetics, 4 (8):e1000167, 08 2008.
23. R. Jones, R. Kumar, B. Pang, and A. Tomkins. "i know what you did last summer": query logs and user privacy. In CIKM '07: Proceedings of the Sixteenth ACM Conference on Information and Knowledge Management, pages 909-914, New York, NY, USA, 2007. ACM. ISBN 978-159593-803-9.
24. S.P. Kasivishwanathan, M. Rudelson, A. Smith, and J. Ullman. The price of privately releasing contingency tables and the spectra of random matrices with correlated rows. In Proceedings of the 42nd ACM Symposium on Theory of Computing (STOC), 2010.
25. S. A. Kasiviswanathan, H. K. Lee, K. Nissim, S. Raskhodnikova, and A. Smith. What can we learn privately? In Proc. of FOCS, 2008.
26. G.S. Kimeldorfand G. Wahba. A correspondence between Bayesian estimation on stochastic processes and smoothing by splines. Annals of Mathematical Statistics, 41:495-502, 1970.
27. S. Laur, H. Lipmaa, and T. Mielikainen. Cryptographically private support vector machines. In Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), pages 618-624, 2006. (Pubitemid 44535564)
28. A. Machanavajjhala, J. Gehrke, D. Kifer, and M. Venkitasubramaniam. l-diversity: Privacy beyond k-anonymity. In Proceedings of the 22nd International Conference on Data Engineering (ICDE), 2006.
29. A. Machanavajjhala, D. Kifer, J. M. Abowd, J. Gehrke, and L. Vilhuber. Privacy: Theory meets practice on the map. In Proceedings of the 24th International Conference on Data Engineering (ICDE), pages 277-286, 2008.
30. O. L. Mangasarian, E. W. Wild, and G. Fung. Privacy-preserving classification of vertically partitioned data via random kernels. ACM Transactions on Knowledge Discovery from Data, 2(3), 2008.
31. F. McSherry and K. Talwar. Mechanism design via differential privacy. In Proceedings of the 48th Annual IEEE Symposium on Foundations of Computer Science (FOCS), pages 94-103, 2007.
32. A. Narayanan and V. Shmatikov. Robust de-anonymization of large sparse datasets (how to break anonymity of the netflix prize dataset). In Proceedings of 29th IEEE Symposium on Security and Privacy, pages 111-125, 2008.
33. K. Nissim, S. Raskhodnikova, and A. Smith. Smooth sensitivity and sampling in private data analysis. In David S. Johnson and Uriel Feige, editors, Proceedings of the 39th ACM Symposium on the Theory of Computing (STOC), pages 75-84. ACM, 2007. ISBN 978-1-59593-631-8. (Pubitemid 47630723)
34. N. Okazaki. liblbfgs: a library of limited-memory Broyden-Fletcher- Goldfarb-Shanno (L-BFGS). 2009. URL http://www.chokkan.org/software/liblbfgs/ index.html.
35. A. Rahimi and B. Recht. Random features for large-scale kernel machines. In Proceedings of the 21st Annual Conference on Neural Information Processing Systems (NIPS), 2007.
36. A. Rahimi and B. Recht. Uniform approximation of functions with random bases. In Proceedings of the 46th Allerton Conference on Communication, Control, and Computing, 2008a.
37. A. Rahimi and B. Recht. Weighted sums of random kitchen sinks : Replacing minimization with randomization in learning. In Proceedings of the 22nd Annual Conference on Neural Information Processing Systems (NIPS), 2008b.
38. R.T. Rockafellar and R J-B. Wets. Variational Analysis. Springer, Berlin, 1998.
39. B. I. P. Rubinstein, P. L. Bartlett, L. Huang, and N. Taft. Learning in a large function space: Privacy- preserving mechanisms for SVM learning. In http://arxiv.org/abs/0911.5708, 2009.
40. S. Shalev-Shwartz. Online Learning: Theory, Algorithms, and Applications. PhD thesis, The Hebrew University of Jerusalem, July 2007.
41. S. Shalev-Shwartz and N. Srebro. SVM optimization : Inverse dependence on training set size. In The 25th International Conference on Machine Learning (ICML), 2008.
42. K. Sridharan, N. Srebro, and S. Shalev-Shwartz. Fast rates for regularized objectives. In Proceedings of the 22nd Annual Conference on Neural Information Processing Systems (NIPS), 2008.
43. L. Sweeney. Weaving technology and policy together to maintain conidentiality. Journal of Law, Medicine and Ethics, 25:98-110, 1997. (Pubitemid 127448006)
44. L. Sweeney. k-anonymity: a model for protecting privacy. International Journal on Uncertainty, Fuzziness and Knowledge-Based Systems, 2002.
45. V. Vapnik. Statistical Learning Theory. John Wiley & Sons, New York, 1998.
46. R. Wang, Y. F. Li, X. Wang, H. Tang, and X.-Y Zhou. Learning your identity and disease from research papers: information leaks in genome wide association study. In ACM Conference on Computer and Communications Security, pages 534-544, 2009.
47. L. Wasserman and S. Zhou. A statistical framework for differential privacy. Journal of the American Statistical Association, 105(489):375-389, 2010.
48. A. C.-C. Yao. Protocols for secure computations (extended abstract). In 23rd Annual Symposium on Foundations of Computer Science (FOCS), pages 160-164, 1982.
49. J. Z. Zhan and S. Matwin. Privacy-preserving support vector machine classification. International Journal of Intelligent Information and Database Systems, 1(3/4):356-385, 2007.
50. S. Zhou, K. Ligett, and L. Wasserman. Differential privacy with compression. In Proceedings of the 2009 International Symposium on Information Theory, Seoul, South Korea, 2009.