Compressed and privacy-sensitive sparse regression

IEEE Transactions on Information Theory

Volumn 55, Issue 2, 2009, Pages 846-866

  Zhou, Shuheng ^a   Lafferty, John ^b   Wasserman, Larry ^c  

^a ETH ZURICH   (Switzerland)

^b Carnegie Mellon University   (United States)

^c CARNEGIE MELLON UNIVERSITY   (United States)

Author keywords

1 regularization; Capacity of multiple antenna channels; Compressed sensing; High dimensional regression; Lasso; Privacy; Sparsity

Indexed keywords

ANTENNAS; CURVE FITTING; MATHEMATICAL MODELS; SIGNAL RECONSTRUCTION;

CAPACITY OF MULTIPLE-ANTENNA CHANNELS; COMPRESSED SENSING; HIGH-DIMENSIONAL REGRESSION; LASSO; PRIVACY; SPARSITY;

REGRESSION ANALYSIS;

EID: 61349111475     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2008.2009605     Document Type: Article

References (37)

1. G. Duncan and R. Pearson, "Enhancing access to microdata while protecting confidentiality: Prospects for the future," Statisti. Sci., vol. 6, no. 3, pp. 219-232, Aug. 1991.
2. R. Tibshirani, "Regression shrinkage and selection via the lasso," J. Roy. Statist. Soc. Ser. B, vol. 58, no. 1, pp. 267-288, 1996.
3. E. Greenshtein and Y. Ritov, "Persistency in high dimensional linear predictor-selection and the virtue of over-parametrization," Bernoulli, vol. 10, pp. 971-988, 2004.
4. T. L. Marzetta and B. M. Hochwald, "Capacity of a mobile multiple-antenna communication link in rayleigh flat fading," IEEE Trans. Inf. Theory, vol. 45, no. 1, pp. 139-157, Jan. 1999.
5. I. E. Telatar, "Capacity of multi-antenna Gaussian channels," Europ. Trans. Telecommun., vol. 10, no. 6, pp. 585-595, Nov. 1999.
6. M. Wainwright, "Sharp Thresholds for High-Dimensional and Noisy Recovery of Sparsity," Dept. Statist., Univ. Calif. Berkeley, Berkeley, CA, 2006, Tech. Rep. 709.
7. N. Meinshausen and B. Yu, "Lasso-Type Recovery of Sparse Representations for High-Dimensional Data," Dept. Statist., Univ. Calif. Berkeley, Berkeley, CA, 2006, Tech. Rep. 720.
8. N. Meinshausen and P. Buhlmann, "High dimensional graphs and variable selection with the lasso," Ann. Statist., vol. 34, no. 3, pp. 1436-1462, 2006.
9. P. Zhao and B. Yu, "On model selection consistency of lasso," J. Machine Learn. Res., vol. 7, pp. 2541-2567, 2007.
10. D. Donoho, "Compressed sensing," IEEE Trans. Inf. Theory, vol. 52, no. 4, pp. 1289-1306, Apr. 2006.
11. E. Candès, J. Romberg, and T. Tao, "Stable signal recovery from incomplete and inaccurate measurements," Commun. Pure Appl. Math. vol. 59, no. 8, pp. 1207-1223, Aug. 2006.
12. E. Candès and T. Tao, "Near optimal signal recovery from random projections: Universal encoding strategies?," IEEE Trans. Inf. Theory, vol. 52, no. 12, pp. 5406-5425, Dec. 2006.
13. H. Rauhut, K. Schnass, and P. Vandergheynst, "Compressed sensing and redundant dictionaries," IEEE Trans. Inf. Theory, vol. 54, no. 5, pp. 2210-2219, May 2008.
14. M. Duarte, M. Davenport, M. Wakin, and R. Baraniuk, "Sparse signal detection from incoherent projections," in Proc. IEEE Int. Conf. Acoustics, Speech, and Signal Processing (ICASSP), Toulouse, France, 2006, pp. III-305-308.
15. M. Davenport, M. Wakin, and R. Baraniuk, "Detection and Estimation With Compressive Measurements," Elec. Comp. Eng. Dept., Rice Univ., Houston, TX, 2006, TREE 0610, Tech. Rep..
16. J. Haupt, R. Castro, R. Nowak, G. Fudge, and A. Yeh, "Compressive sampling for signal classification," in Proc. Asilomar Conf. Signals, Systems, and Computers, Pacific Grove, CA, Oct.\Nov. 2006, pp. 1430-1434.
17. M. Davenport, M. Duarte, M. Wakin, J. Laska, D. Takhar, K. Kelly, and R. Baraniuk, "The smashed filter for compressive classification and target recognition," in Proc. Computational Imaging V, San Jose, CA, Feb. 2007, vol. 6498, p. 153.
18. W. B. Johnson and J. Lindenstrauss, "Extensions of Lipschitz mappings into a Hilbert space," in Proc. Conf. Modern Analysis and Probability, R. Beals, A. Beck, A. Bellow, and A. Hajian, Eds., New Haven, CT, Jun. 1984, pp. 189-206.
19. T. Dalenius, "Privacy transformations for statistical information systems," J. Statis. Plann. Inference, vol. 1, pp. 73-86, 1977.
20. A. P. Sanil, A. Karr, X. Lin, and J. P. Reiter, "Privacy preserving regression modelling via distributed computation," in Proc. 10th ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, Seattle, WA, Aug. 2004, pp. 677-682.
21. D. Ting, S. E. Fienberg, and M. Trottini, "Random orthogonal matrix masking methodology for microdata release," Int. J. Inf. Comp.Security, vol. 2, no. 1, pp. 86-105, Jan. 2008.
22. C. E. Shannon, "Communication theory of secrecy systems," Bell Syst. Tech. J., vol. 28, pp. 656-715, Oct. 1949.
23. A. D. Wyner, "The wire-tap channel," Bell Syst. Tech. J., vol. 54, no. 8, pp. 1355-1387, Oct. 1975.
24. K. Liu, H. Kargupta, and J. Ryan, "Random projection-based multiplicative data perturbation for privacy preserving distributed data mining," IEEE Trans. Knowl. Data Eng., vol. 18, no. 1, pp. 92-106, Jan. 2006.
25. D. Agrawal and C. C. Aggarwal, "On the design and quantification of privacy preserving data mining algorithms," in Proc. 20th ACM SIGACT-SIGMOD-SIGART Symp. Principles of Database Systems, Santa Barbara, CA, May 2001, pp. 247-255.
26. C. Dwork, "Differential privacy," in Proc. 33rd Int. Colloquium on Automata, Languages and Programming, Venice, Italy, 2006, pp. 1-12.
27. T. Dalenius, "Towards a methodology for statistical disclosure control," Statistik Tidskrift, vol. 15, pp. 429-444, 1977.
28. C. Dwork, F. McSherry, and K. Talwar, "The price of privacy and the limits of LP decoding," in Proc. Symp. Theory of Computing (STOC San Diego, CA, Jun. 2007, pp. 85-94.
29. D. Donoho, M. Elad, and V. Temlyakov, "Stable recovery of sparse overcomplete representations in the presence of noise," IEEE Trans. Inf. Theory, vol. 52, no. 1, pp. 6-18, Jan. 2006.
30. J. Tropp. "Greed is good: Algorithmic results for sparse approximation," IEEE Trans. Inf. Theory, vol. 50, no. 10, pp. 2231-2242, Oct. 2004.
31. I. Johnstone, "Chi-square oracle inequalities," in State of the Art in Probability and Statistics, Festchrift for Willem R. van Zwet, M. de Gunst. C. Klaassen, and C. van der Waart, Eds. Beachwood, OH: Inst. Math. Statist., 2001, vol. 36, IMS Lecture Notes-Monographs, pp. 399-418.
32. I. Johnstone and A. Y. Lu, "Sparse Principal Components Analysis," Dept. Statistics, Stanford Univ., Stanford, CA, Tech. Rep., 2004.
33. B. Efron, T. Hastie, I. Johnstone, and R. Tibshirani, "Least angle regression," Ann. Statist., vol. 32, no. 2, pp. 407-499, 2004.
34. M. Osborne, B. Presnell, and B. Turlach, "On the lasso and its dual," J. Computat. Graph. Statist., vol. 9, no. 2, pp. 319-337, 2000.
35. R. Horn and C. Johnson, Matrix Analysis, reprint ed. Cambridge, U.K.: Cambridge Univ. Press, 1990.
36. 1 -regularized logistic regression," in Advances in Neural Information Processing Systems 19 Cambridge, MA: MIT Press, 2007.
37. J. Lafferty and L. Wasserman, "Rodeo: Sparse, greedy nonparametric regression," Ann. Statist., vol. 36, no. 1, pp. 28-63, 2008.