Robust 1-Bit compressive sensing via binary stable embeddings of sparse vectors

IEEE Transactions on Information Theory

Volumn 59, Issue 4, 2013, Pages 2082-2102

  Jacques, Laurent ^a   Laska, Jason N ^b   Boufounos, Petros T ^c   Baraniuk, Richard G ^d  

^a UNIVERSITÉ CATHOLIQUE DE LOUVAIN   (Belgium)

^b Dropcam Inc   (United States)

^c MITSUBISHI ELECTRIC RESEARCH LABORATORIES   (United States)

^d Rice University   (United States)

Author keywords

1 bit compressed sensing; Approximation error; Compressed sensing; Consistent reconstruction; Dimensionality reduction; Iterative reconstruction; Quantization; Reconstruction algorithms; Signal reconstruction; Sparsity

Indexed keywords

ANALOG TO DIGITAL CONVERSION; APPROXIMATION ALGORITHMS; COMPRESSED SENSING; ITERATIVE METHODS; OPTIMIZATION; SIGNAL ANALYSIS;

APPROXIMATION ERRORS; CONSISTENT RECONSTRUCTION; DIMENSIONALITY REDUCTION; ITERATIVE RECONSTRUCTION; QUANTIZATION; RECONSTRUCTION ALGORITHMS; SPARSITY;

SIGNAL RECONSTRUCTION;

EID: 84896689375     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2012.2234823     Document Type: Article

References (69)

1. E. Candès, "Compressive sampling," presented at the Int. Congr. Math., Madrid, Spain, Aug. 2006.
2. D. Donoho, "Compressed sensing," IEEE Trans. Inf. Theory, vol. 6, no. 4, pp. 1289-1306, Apr. 2006.
3. J. Tropp, J. Laska, M. Duarte, J. Romberg, and R. Baraniuk, "Beyond Nyquist: Efficient sampling of sparse, bandlimited signals," IEEE Trans. Inf. Theory, vol. 56, no. 1, pp. 520-544, Jan. 2010.
4. M. Duarte, M. Davenport, D. Takhar, J. Laska, T. Sun, K. Kelly, and R. Baraniuk, "Single-pixel imaging via compressive sampling," IEEE Signal Process. Mag., vol. 25, no. 2, pp. 83-91, Mar. 2008.
5. B. K. Natarajan, "Sparse approximate solutions to linear systems," SIAM J. Comput., vol. 24, pp. 227-234, 1995.
6. S. S. Chen,D. L. Donoho, andM.A. Saunders, "Atomic decomposition by basis pursuit," SIAM J. Sci. Comput., vol. 20, no. 1, pp. 33-61, 1998. (Pubitemid 128689501)
7. E. Candès, "The restricted isometry property and its implications for compressed sensing," Comptes Rendus de l'Académie des Sci., Série I, vol. 346, no. 9-10, pp. 589-592, 2008.
8. E. Candès, J. Romberg, and T. Tao, "Stable signal recovery from incomplete and inaccurate measurements," Commun. Pure Appl. Math., vol. 59, no. 8, pp. 1207-1223, 2006. (Pubitemid 43988295)
9. E. T. Hale,W. Yin, and Y. Zhang, A fixed-point continuation method for regularized minimization with applications to compressed sensing Rice University, Houston, TX, 2007, Tech. Rep. TR07-07.
10. M. A. T. Figueiredo, R. D. Nowak, and S. J.Wright, "Gradient projection for sparse reconstruction: Application to compressed sensing and other inverse problems," IEEE J. Sel. Topics Signal Process., vol. 1, no. 4, pp. 586-597, Sep. 2007. (Pubitemid 351276802)
11. E. Candès and T. Tao, "The Dantzig selector: Statistical estimation when is much larger than, " Ann. Statist., vol. 35, no. 6, pp. 2313-2351, 2007.
12. D. Needell and R. Vershynin, "Uniform uncertainty principle and signal recovery via regularized orthogonal matching pursuit," Found. Comput. Math., vol. 9, no. 3, pp. 317-334, 2009.
13. D. Needell and J. Tropp, "CoSaMP: Iterative signal recovery from incomplete and inaccurate samples," Appl. Comput. Harmon. Anal., vol. 26, no. 3, pp. 301-321, 2009.
14. T. Blumensath and M. Davies, "Iterative hard thresholding for compressive sensing," Appl. Comput. Harmon. Anal., vol. 27, no. 3, pp. 265-274, 2009.
15. R. Baraniuk,M.Davenport, R. DeVore, and M.Wakin, "A simple proof of the restricted isometry property for random matrices," Const. Approx., vol. 28, no. 3, pp. 253-263, 2008.
16. G. G. Lorentz, M. von Golitschek, and Y. Makovoz, Constructive Approximation: Advanced Problems. Berlin, Germany: Springer-Verlag, 1996, vol. 304.
17. J. Romberg, "Compressive sensing by random convolution," SIAM J. Imag. Sci., vol. 2, pp. 1098-1128, 2009.
18. J. Tropp, M. Wakin, M. Duarte, D. Baron, and R. Baraniuk, "Random filters for compressive sampling and reconstruction," presented at the IEEE Int. Conf. Acoustics, Speech, Signal Process., Toulouse, France, May 2006.
19. L. Jacques, P. Vandergheynst, A. Bibet, V.Majidzadeh, A. Schmid, and Y. Leblebici, "CMOS compressed imaging by random convolution," presented at the IEEE Int. Conf. Acoustics, Speech, Signal Process., Taipei, Taiwan, Apr. 2009.
20. L. Jacques, D. Hammond, and M. Fadili, "Dequantizing compressed sensing: When oversampling and non-Gaussian constraints combine," IEEE Trans. Inf. Theory, vol. 57, no. 1, pp. 559-571, Jan. 2011.
21. J. Laska, P. Boufounos, M. Davenport, and R. Baraniuk, "Democracy in action: Quantization, saturation, and compressive sensing," Appl. Comput. Harmon. Anal., vol. 31, pp. 429-443, 2011, to be published.
22. W. Dai, H. Pham, and O. Milenkovic, "Distortion-rate functions for quantized compressive sensing," in Proc. IEEE Inf. Theory Workshop Netw. Inf. Theory, Jun. 2009, pp. 171-175.
23. A. Zymnis, S. Boyd, and E. Candès, "Compressed sensing with quantized measurements," IEEE Signal Process. Lett., vol. 17, no. 2, pp. 149-152, Feb. 2010.
24. J. Sun and V. Goyal, "Quantization for compressed sensing reconstruction," presented at the Sampling Theory Appl., Marseille, France, May 2009.
25. R.Walden, "Analog-to-digital converter survey and analysis," IEEE J. Sel. Areas Commun., vol. 17, no. 4, pp. 539-550, Apr. 1999.
26. B. Le, T. W. Rondeau, J. H. Reed, and C.W. Bostian, "Analog-to- digital converters," IEEE Signal Process Mag., vol. 22, no. 6, pp. 69-77, Nov. 2005. (Pubitemid 46404904)
27. P. Boufounos, "Reconstruction of sparse signals from distorted randomized measurements," in Proc. IEEE Int. Conf. Acoustics, Speech, Signal Process., Dallas, TX, Mar. 2010, pp. 3998-4001.
28. P. Boufounos and R. Baraniuk, "1-bit compressive sensing," in Proc. 42nd Annu. Conf. Inf. Sci. Syst., Princeton, NJ, Mar. 2008, pp. 16-21.
29. P. Boufounos, "Greedy sparse signal reconstruction from sign measurements," in Proc. 43rd Asilomar Conf. Signals Syst. Comput., Pacific Grove, CA, Nov. 2009, pp. 1305-1309.
30. J. Laska, Z.Wen, W. Yin, and R. Baraniuk, "Trust, but verify: Fast and accurate signal recovery from 1-bit compressive measurements," IEEE Trans. Signal Process., vol. 59, no. 11, pp. 5289-5301, Nov. 2011.
31. Y. Plan and R. Vershynin, Dimension reduction by random hyperplane tessellations 2011 [Online]. Available: arXiv:1111.4452
32. Y. Plan and R. Vershynin, One-bit compressed sensing by linear programming 2011 [Online]. Available: arXiv:1109.4299
33. Y. Plan and R. Vershynin, Robust 1-bit compressed sensing and sparse logistic regression: A convex programming approach 2012 [Online]. Available: arXiv:1202.1212
34. P. Boufounos, "Quantization and erasures in frame representations," Ph.D. dissertation, Massachusetts Inst. Technol. Dept. Electr. Eng. Comput. Sci., Cambridge, MA, 2006.
35. P. Boufounos and R. Baraniuk, "Quantization of sparse representations," in Proc. Data Compress. Conf., Mar. 2007, p. 378. (Pubitemid 47201891)
36. J. Z. Sun and V. K. Goyal, "Optimal quantization of random measurements in compressed sensing," in Proc. Int. Symp. Inf. Theory, Jun. 2009, pp. 6-10.
37. N. Thao and M. Vetterli, "Reduction of the MSE in-times oversampled A/D conversion to, " IEEE Trans. Signal Process., vol. 42, no. 1, pp. 200-203, Jan. 1994.
38. V. K. Goyal, M. Vetterli, and N. Thao, "Quantized overcomplete expansions in : Analysis, synthesis, and algorithms," IEEE Trans. Inf. Theory, vol. 44, no. 1, pp. 16-31, Jan. 1998. (Pubitemid 128737877)
39. P. M. Aziz, H. V. Sorensen, and J. Van Der Spiegel, "An overview of Sigma-Delta converters," IEEE Signal Process. Mag., vol. 13, no. 1, pp. 61-84, Jan. 1996.
40. N. T. Thao, "Vector quantization analysis of modulation," IEEE Trans. Signal Process., vol. 44, no. 4, pp. 808-817, Apr. 1996.
41. J. C. Candy and G. C. Temes, Eds., Oversampling Delta-Sigma Converters. Piscataway, NJ: IEEE Press, 1992.
42. J. J. Benedetto, A. M. Powell, and O. Yilmaz, "Sigma-delta quantization and finite frames," IEEE Trans. Inf. Theory, vol. 52, no. 5, pp. 1990-2005, May 2006.
43. P. Boufounos and A. V. Oppenheim, "Quantization noise shaping on arbitrary frame expansions," EURASIP J. Appl. Signal Process., vol. 2006, pp. 53807-1-53807-12, 2006.
44. P. Boufounos and R. Baraniuk, "Sigma delta quantization for compressive sensing," in Proc. SPIEWaveletts XII., San Diego, CA, Aug. 2007, vol. 6701, p. 670104.
45. S. Gunturk, A. Powell, R. Saab, and O. Yülmaz, Sobolev duals for random frames and sigma-delta quantization of compressed sensing measurements Feb. 2010 [Online]. Available: arXiv:1002.0182v1
46. P. Boufounos, "Universal rate-efficient scalar quantization," IEEE Trans. Inf. Theory, vol. 3, no. 58, pp. 1861-1872, Mar. 2012.
47. C. S. Güntürk, "One-bit sigma-delta quantization with exponential accuracy," Commun. Pure Appl. Math., vol. 56, no. 11, pp. 1608-1630, 2003. (Pubitemid 38614632)
48. P. T. Boufounos, "Hierarchical distributed scalar quantization," presented at the Int. Conf. Sampling Theory Appl., Singapore, May 2-6, 2011.
49. H. Q. Nguyen, V. K. Goyal, and L. R. Varshney, "Frame permutation quantization," Appl. Comput. Harmon. Anal., vol. 31, pp. 74-97, Jul. 2011.
50. P. Indyk and R. Motwani, "Approximate nearest neighbors: Towards removing the curse of dimensionality," in Proc. 13th Annu. ACM Symp. Theory Comput., 1998, pp. 604-613.
51. A. Andoni, M. Datar, N. Immorlica, P. Indyk, and V. Mirrokni, "Locality-sensitive hashing scheme based on p-stable distributions," in Nearest Neighbor Methods in Learning and Vision: Theory and Practice. Cambridge, MA: MIT Press, 2006.
52. A. Andoni and P. Indyk, "Near-optimal hashing algorithms for approximate nearest neighbor in high dimensions," Commun. ACM, vol. 51, no. 1, pp. 117-122, 2008.
53. M. Raginsky and S. Lazebnik, "Locality-sensitive binary codes from shift-invariant kernels," Neural Inf. Process. Syst., vol. 22, pp. 1509-1517, 2009.
54. M. Bridson, T. Gowers, J. Barrow-Green, and I. Leader, Eds., "Geometric and combinatorial group theory," in The Princeton Companion to Mathematics. Princeton, NJ: Princeton Univ. Press, 2008, ch. IV.10, pp. 431-447.
55. M. Goemans and D.Williamson, "Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming," J. ACM, vol. 42, no. 6, pp. 1115-1145, 1995.
56. S. Shariati, L. Jacques, F. X. Standaert, B. Macq, M. A. Salhi, and P. Antoine, "Randomly driven fuzzy key extraction of unclonable images," in Proc. IEEE Int. Conf. Image Process., 2010, pp. 4329-4332.
57. A. Gupta, B. Recht, and R. Nowak, "Sample complexity for 1-bit compressed sensing and sparse classification," in Proc. Int. Symp. Inf. Theory, 2010, pp. 1553-1557.
58. T. Blumensath, Compressed sensing with nonlinear observations 2010 [Online]. Available: arXiv:1205.1650v1
59. T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning, 2nd ed. New York: Springer-Verlag, 2009, Series in Statistics.
60. L. Rosasco, E. De Vito, A. Caponnetto, M. Piana, and A. Verri, "Are loss functions all the same?," Neural Comput., vol. 16, no. 5, pp. 1063-1076, Mar. 2004. (Pubitemid 38483051)
61. N. Srebro, K. Sridharan, and A. Tewari, "Smoothness, low-noise, and fast rates," in Proc. Adv. Neural Inf. Process. Syst., Dec. 2010, pp. 2199-2207.
62. R. Tibshirani, "Regression shrinkage and selection via the Lasso," Royal Statist. Soc. Series B (Methodol.), vol. 58, pp. 267-288, 1996.
63. O. Bartlett, Y. Freund, W. S. Lee, and R. E. Schapire, "Boosting the margin: A new explanation for the effectiveness of voting methods," Ann. Statist., vol. 26, no. 5, pp. 1651-1686, 1998. (Pubitemid 128376902)
64. G. Ratsch and M. Warmuth, "Efficient margin maximizing with boosting," J. Mach. Learn. Res., vol. 6, Dec. 2005.
65. A. Blum, On-Line Algorithms in Machine Learning. New York: Springer-Verlag, 1998, vol. 1442/1998, Lecture Notes in Computer Science.
66. T. M. Cover, "Geometrical and statistical properties of systems of linear inequalities with applications in pattern recognition," IEEE Trans. Electron. Comput., vol. 14, no. 3, pp. 326-334, Jun. 1965.
67. L. Flatto, "A new proof of the transposition theorem," Proc. Amer. Math. Soc., vol. 24, no. 1, pp. 29-31, Jan. 1970.
68. K. Ball, "An elementary introduction to modern convex geometry," in Flavors of Geometry, S. Levy, Ed. Cambridge, U.K.: Cambridge Univ. Press, 1997, vol. 31, MSRI Publications, pp. 1-58.
69. N. Johnson, S. Kotz, and N. Balakrishnan, Continuous Univariate Distributions. New York: Wiley, 1994, vol. 1.