Privacy aware learning

Journal of the ACM

Volumn 61, Issue 6, 2014, Pages 1-57

  Duchi, John C ^a   Jordan, Michael I ^b   Wainwright, Martin J ^b  

^a STANFORD UNIVERSITY   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Differential privacy; Lower bounds; Machine learning; Minimax convergence rates; Saddle points

Indexed keywords

ESTIMATION; LEARNING SYSTEMS;

CONVERGENCE RATES; DIFFERENTIAL PRIVACIES; LEARNING PROCEDURES; LOWER BOUNDS; SADDLE POINT; STATISTICAL ESTIMATION; STATISTICAL ESTIMATORS; UPPER AND LOWER BOUNDS;

DATA PRIVACY;

EID: 84919772168     PISSN: 00045411     EISSN: 1557735X     Source Type: Journal    
DOI: 10.1145/2666468     Document Type: Article

References (53)

1. A. Agarwal, P. L. Bartlett, P. Ravikumar, and M. J. Wainwright. 2012. Information-theoretic lower bounds on the oracle complexity of convex optimization. IEEE Trans. Inf. Theory 58, 5, 3235-3249.
2. A. Beck and M. Teboulle. 2003. Mirror descent and nonlinear projected subgradient methods for convex optimization. Oper. Res. Lett. 31, 167-175.
3. D. P. Bertsekas and J. N. Tsitsiklis. 1989. Parallel and Distributed Computation: Numerical Methods. Prentice-Hall, Inc.
4. P. Billingsley. 1986. Probability and Measure, 2nd Ed. Wiley.
5. A. Blum, K. Ligett, and A. Roth. 2008. A learning theory approach to non-interactive database privacy. In Proceedings of the 40th Annual ACM Symposium on the Theory of Computing.
6. S. Boyd and L. Vandenberghe. 2004. Convex Optimization. Cambridge University Press.
7. K. Chaudhuri, C. Monteleoni, and A. D. Sarwate. 2011. Differentially private empirical risk minimization. J. Machine Learn. Res. 12, 1069-1109.
8. T. M. Cover and J. A. Thomas. 2006. Elements of information theory, 2nd Ed. Wiley.
9. L. H. Cox, A. F. Karr, and S. K. Kinney. 2011. Risk-utility paradigms for statistical disclosure limitation: How to think, but not how to act. Int. Stat. Rev. 79, 2, 160-199.
10. I. Csisz'ar and J. Körner. 1981. Information Theory: Coding Theorems for Discrete Memoryless Systems. Academic Press.
11. I. Dinur and K. Nissim. 2003. Revealing information while preserving privacy. In Proceedings of the 22nd Symposium on Principles of Database Systems. 202-210.
12. J. C. Duchi, M. I. Jordan, and M. J. Wainwright. 2013. Local privacy and statistical minimax rates. arXiv:1302.3203 [math.ST].
13. G. T. Duncan and D. Lambert. 1986. Disclosure-limited data dissemination. J. Amer. Stat. Assoc. 81, 393, 10-18.
14. G. T. Duncan and D. Lambert. 1989. The risk of disclosure for microdata. J. Busin. Economic Statistics 7, 2, 207-217.
15. C. Dwork. 2008. Differential privacy: A survey of results. In Theory and Applications of Models of Computation, Lecture Notes in Computer Science Series, vol. 4978, Springer, 1-19.
16. C. Dwork and J. Lei. 2009. Differential privacy and robust statistics. In Proceedings of the 41st Annual ACM Symposium on the Theory of Computing.
17. C. Dwork, F. Mcsherry, K. Nissim, and A. Smith. 2006. Calibrating noise to sensitivity in private data analysis. In Proceedings of the 3rd Theory of Cryptography Conference. 265-284.
18. C. Dwork, G. N. Rothblum, and S. P. Vadhan. 2010. Boosting and differential privacy. In Proceedings of the 51st Annual Symposium on Foundations of Computer Science. 51-60.
19. A. V. Evfimievski, J. Gehrke, and R. Srikant. 2003. Limiting privacy breaches in privacy preserving data mining. In Proceedings of the 22nd Symposium on Principles of Database Systems. 211-222.
20. I. P. Fellegi. 1972. On the question of statistical confidentiality. J. Amer. Stat. Assoc. 67, 337, 7-18.
21. S. R. Ganta, S. Kasiviswanathan, and A. Smith. 2008. Composition attacks and auxiliary information in data privacy. In Proceedings of the 14th ACM SIGKDD Conference on Knowledge and Data Discovery (KDD).
22. A. Ghosh, T. Roughgarden, and M. Sundararajan. 2009. Universally utility-maximizing privacy mechanisms. In Proceedings of the 41st Annual ACM Symposium on the Theory of Computing.
23. R. M. Gray. 1990. Entropy and information theory. Springer.
24. R. Hall, A. Rinaldo, and L. Wasserman. 2011. Random differential privacy. arXiv:1112.2680 [stat.ME].
25. M. Hardt and K. Talwar. 2010. On the geometry of differential privacy. In Proceedings of the 42nd Annual ACM Symposium on the Theory of Computing. 705-714.
26. J. Hiriart-Urruty and C. Lemaréchal. 1996. Convex Analysis and Minimization Algorithms I & II. Springer, New York.
27. O. Kallenberg. 1997. Foundations of Modern Probability. Springer.
28. A. F. Karr, C. N. Kohnen, A. Oganian, J. P. Reiter, and A. P. Sanil. 2006. A framework for evaluating the utility of data altered to protect confidentiality. Amer. Statistician 60, 3, 224-232.
29. S. P. Kasiviswanathan, H. K. Lee, K. Nissim, S. Raskhodnikova, and A. Smith. 2011. What can we learn privately? SIAM J. Comput. 40, 3, 793-826.
30. S. P. Kasiviswanathan, M. Rudelson, and A. Smith. 2013. The power of linear reconstruction attacks. In Proceedings of the 45th Annual ACM Symposium on the Theory of Computing.
31. M. Kearns. 1998. Efficient noise-tolerant learning from statistical queries. J. ACM 45, 6, 983-1006.
32. L. Le Cam. 1956. On the asymptotic theory of estimation and hypothesis testing. In Proceedings of the 3rd Berkeley Symposium on Mathematical Statistics and Probability, 129-156.
33. L. Le Cam. 1973. Convergence of estimates under dimensionality restrictions. Ann. Stat. 1, 1, 38-53.
34. Y. Liang, H. V. Poor, and S. Shamai. 2008. Information theoretic security. Found. Trends Commun. Inf. Theory 5, 4, 355-580.
35. O. L. Mangasarian. 1979. Uniqueness of solution in linear programming. Linear Algebra Appl. 25, 151-162.
36. A. Nemirovski, A. Juditsky, G. Lan, and A. Shapiro. 2009. Robust stochastic approximation approach to stochastic programming. SIAM J. Optimiz. 19, 4, 1574-1609.
37. A. Nemirovski and D. Yudin. 1983. Problem Complexity and Method Efficiency in Optimization. Wiley.
38. A. Nikolov, K. Talwar, and L. Zhang. 2013. The geometry of differential privacy: The sparse and approximate case. In Proceedings of the 45th Annual ACM Symposium on the Theory of Computing.
39. R. R. Phelps. 2001. Lectures on Choquet's Theorem, 2nd Ed. Springer.
40. B. T. Polyak and A. B. Juditsky. 1992. Acceleration of stochastic approximation by averaging. SIAM J. Cont. Optimiz. 30, 4, 838-855.
41. J. P. Reiter. 2005. Estimating risks of identification disclosure in microdata. J. Amer. Stat. Assoc. 100, 1103-1113.
42. B. I. P. Rubinstein, P. L. Bartlett, L. Huang, and N. Taft. 2012. Learning in a large function space: privacypreserving mechanisms for SVM learning. J. Priv. Confident. 4, 1, 65-100.
43. L. Sankar, S. R. Rajagopalan, and H. V. Poor. 2010. An information-theoretic approach to privacy. In Proceedings of the 48th Allerton Conference on Communication, Control, and Computing. 1220-1227.
44. A. Smith. 2011. Privacy-preserving statistical estimation with optimal convergence rates. In Proceedings of the 43rd Annual ACM Symposium on the Theory of Computing.
45. A. W. Van Der Vaart. 1998. Asymptotic Statistics. Cambridge Series in Statistical and Probabilistic Mathematics, Cambridge University Press.
46. A. Wald. 1939. Contributions to the theory of statistical estimation and testing hypotheses. Ann. Math. Stat. 10, 4, 299-326.
47. S. Warner. 1965. Randomized response: A survey technique for eliminating evasive answer bias. J. Amer. Stat. Assoc. 60, 309, 63-69.
48. L. Wasserman and S. Zhou. 2010. A statistical framework for differential privacy. J. Amer. Stat. Assoc. 105, 489, 375-389.
49. Y. Yang and A. Barron. 1999. Information-theoretic determination of minimax rates of convergence. Ann. Statics 27, 5, 1564-1599.
50. B. Yu. 1997. Assouad, Fano, and Le Cam. In Festschrift for Lucien Le Cam. Springer-Verlag, 423-435.
51. S. Zhou, J. Lafferty, and L. Wasserman. 2009a. Compressed regression. IEEE Trans. Inf. Theory 55, 2, 846-866.
52. S. Zhou, K. Ligett, and L. Wasserman. 2009b. Differential privacy with compression. In Proceedings of the IEEE International Symposium on Information Theory.
53. M. Zinkevich. 2003. Online convex programming and generalized infinitesimal gradient ascent. In Proceedings of the 20th International Conference on Machine Learning.