Differentially private model selection via stability arguments and the robustness of the Lasso

Journal of Machine Learning Research

Volumn 30, Issue , 2013, Pages 819-850

  Smith, Adam ^a   Thakurta, Abhradeep ^b  

^a PENNSYLVANIA STATE UNIVERSITY   (United States)

^b STANFORD UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

PROBABILITY DISTRIBUTIONS; STABILITY;

HIGH PROBABILITY; INPUT DATAS; MODEL SELECTION; MODEL SELECTION PROCEDURES; SAMPLE COMPLEXITY; SPARSE LINEAR REGRESSIONS; STABILITY PROPERTIES; STATISTICAL MODELING;

ALGORITHMS;

EID: 84898021295     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Conference Paper

References (31)

1. Francis R. Bach. Bolasso: model consistent lasso estimation through the bootstrap. In ICML, 2008.
2. Olivier Bousquet and André Elisseeff. Stability and generalization. Journal of Machine Learning Research, 2: 499-526, 2002.
3. Gerda Claeskens and Nils Lid Hjort. Model selection and model averaging. Cambridge University Press, 2008.
4. David L. Donoho. Aide-memoire. high-dimensional data analysis: The curses and blessings of dimensionality, 2000.
5. Cynthia Dwork. Differential privacy. In ICALP, 2006.
6. Cynthia Dwork and Jing Lei. Differential privacy and robust statistics. In STOC, 2009.
7. Cynthia Dwork, Krishnaram Kenthapadi, Frank Mcsherry, Ilya Mironov, and Moni Naor. Our data, ourselves: Privacy via distributed noise generation. In EUROCRYPT, 2006a.
8. Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating noise to sensitivity in private data analysis. In TCC, 2006b.
9. Srivatsava Ranjit Ganta, Shiva Prasad Kasiviswanathan, and Adam Smith. Composition attacks and auxiliary information in data privacy. In KDD, 2008.
10. Eitan Greenshtein and Ya'acov Ritov. Persistence in high-dimensional linear predictor selection and the virtue of overparametrization. Bernoulli, 2004.
11. Peter Huber. Robust Statistics. Wiley, 1981.
12. Anatoli Juditsky and Arkadii Nemirovski. Functional aggregation for nonparametric regression. The Annals of Statistics, 2000.
13. Vishesh Karwa, Sofya Raskhodnikova, Adam Smith, and Grigory Yaroslavtsev. Private analysis of graph structure. PVLDB, 2011.
14. Shiva Prasad Kasiviswanathan and Adam Smith. A note on differential privacy: Defining resistance to arbitrary side information. CoRR, arXiv: 0803.39461 [cs.CR], 2008.
15. Daniel Kifer, Adam Smith, and Abhradeep Thakurta. Private convex empirical risk minimization and high-dimensional regression. In COLT, 2012.
16. Yongdai Kim, Sunghoon Kwon, and Hosik Choi. Consistent model selection criteria on high dimensions. Journal of Machine Learning Research, 2012.
17. Y. Lee, S. N. MacEachern, and Y. Jung. Regularization of case-specific parameters for robustness and efficiency. Technical report, Statistics Department, Ohio State University, April 2011.
18. Frank McSherry and Kunal Talwar. Mechanism design via differential privacy. In FOCS, 2007.
19. M. Meilǎ. The uniqueness of a good optimum for k-means. In ICML, 2006.
20. Nicolai Meinshausen and Peter Buehlmann. Stability selection. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 2010.
21. Sahand Negahban, Pradeep Ravikumar, Martin J. Wainwright, and Bin Yu. A unified framework for high-dimensional analysis of $m$-estimators with decomposable regularizers. In NIPS, 2009.
22. Kobbi Nissim, Sofya Raskhodnikova, and Adam Smith. Smooth sensitivity and sampling in private data analysis. In STOC, 2007.
23. p-balls. 2011.
24. WH Rogers and TJ Wagner. A finite sample distribution-free performance bound for local discrimination rules. The Annals of Statistics, pages 506-514, 1978.
25. S. Shalev-Shwartz, O. Shamir, N. Srebro, and K. Sridharan. Learnability, stability and uniform convergence. The Journal of Machine Learning Research, 2010.
26. Jun Shao. Bootstrap model selection. Journal of the American Statistical Association, 1996.
27. Adam Smith. Privacy-preserving statistical estimation with optimal convergence rates. In STOC, 2011.
28. 1-constrained quadratic programs. In IEEE Transactions on Information Theory, 2006.
29. H. Xu, C. Caramanis, and S. Mannor. Robust regression and lasso. IEEE Transactions on Information Theory, 2010.
30. P. Zhao and B. Yu. On model selection consistency of lasso. Journal of Machine Learning Research, 2007.
31. Shuheng Zhou, Katrina Ligett, and Larry Wasserman. Differential privacy with compression. In ISIT, 2009.