Distributed Compressive Sensing: A Deep Learning Approach

IEEE Transactions on Signal Processing

Volumn 64, Issue 17, 2016, Pages 4504-4518

  Palangi, Hamid ^a   Ward, Rabab ^a   Deng, Li ^b  

^a UNIVERSITY OF BRITISH COLUMBIA   (Canada)

^b MICROSOFT RESEARCH   (United States)

Author keywords

Compressive sensing; deep learning; long short term memory

Indexed keywords

BAYESIAN NETWORKS; BRAIN; COST FUNCTIONS; DECODING; PROBABILITY; SIGNAL RECONSTRUCTION; VECTORS;

COMPRESSIVE SENSING; CONDITIONAL PROBABILITIES; DATA-DRIVEN METHODS; DEEP LEARNING; LONG SHORT TERM MEMORY; MULTIPLE MEASUREMENT VECTORS; ORTHOGONAL MATCHING PURSUIT; REAL-WORLD DATASETS;

COMPRESSED SENSING;

EID: 84982839116     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2016.2557301     Document Type: Article

References (53)

1. S. Hochreiter and J. Schmidhuber, "Long short-term memory, " Neural Comput., vol. 9, no. 8, pp. 1735-1780, Nov. 1997.
2. D. Donoho, "Compressed sensing, " IEEE Trans. Inf. Theory, vol. 52, no. 4, pp. 1289-1306, Apr. 2006.
3. E. Candes, J. Romberg, and T. Tao, "Stable signal recovery from incomplete and inaccurate measurements, " Commun. Pure Appl. Math., vol. 59, no. 8, pp. 1207-1223, 2006.
4. R. Baraniuk, "Compressive sensing [lecture notes], " IEEE Signal Process. Mag., vol. 24, no. 4, pp. 118-121, Jul. 2007.
5. E. J. Candes, Y. C. Eldar, D. Needell, and P. Randall, "Compressed sensing with coherent and redundant dictionaries, " Appl. Comput. Harmon. Anal., vol. 31, no. 1, pp. 59-73, 2011.
6. M. Duarte and Y. Eldar, "Structured compressed sensing: From theory to applications, " IEEE Trans. Signal Process., vol. 59, no. 9, pp. 4053-4085, Sept. 2011.
7. M. Davies and Y. Eldar, "Rank awareness in joint sparse recovery, " IEEE Trans. Inf. Theory, vol. 58, no. 2, pp. 1135-1146, Feb. 2012.
8. Y. Eldar and H. Rauhut, "Average case analysis of multichannel sparse recovery using convex relaxation, " IEEE Trans. Inf. Theory, vol. 56, no. 1, pp. 505-519, Jan. 2010.
9. J. Tropp, A. Gilbert, andM. Strauss, "Algorithms for simultaneous sparse approximation. Part I: Greedy pursuit, " Signal Process., vol. 86, no. 3, pp. 572-588, 2006.
10. J. Tropp, "Algorithms for simultaneous sparse approximation. Part II: Convex relaxation, " Signal Process., vol. 86, no. 3, pp. 589-602, 2006.
11. D. P. Wipf and B. D. Rao, "An empirical Bayesian strategy for solving the simultaneous sparse approximation problem, " IEEE Trans. Signal Process., vol. 55, no. 7, pp. 3704-3716, 2007.
12. S. Ji, D. Dunson, and L. Carin, "Multitask compressive sensing, " IEEE Trans. Signal Process., vol. 57, no. 1, pp. 92-106, 2009.
13. L. Deng and D. Yu, "Deep learning: Methods and applications, " Found. Trends Signal Process., vol. 7, pp. 197-387, 2014.
14. D. Yu and L. Deng, Automatic Speech Recognition: A Deep Learning Approach. New York, NY, USA: Springer, 2014.
15. Z. Zhang and B. D. Rao, "Sparse signal recovery with temporally correlated source vectors using sparse Bayesian learning, " IEEE J. Sel. Topcs Signal Process., vol. 5, no. 5, pp. 912-926, 2011.
16. R. Baraniuk, V. Cevher, M. Duarte, and C. Hegde, "Model-based compressive sensing, " IEEE Trans. Inf. Theory, vol. 56, no. 4, pp. 1982-2001, Apr. 2010.
17. S. Ji, Y. Xue, and L. Carin, "Bayesian compressive sensing, " IEEE Trans. Signal Process., vol. 56, no. 6, pp. 2346-2356, 2008.
18. D. Merhej, C. Diab, M. Khalil, and R. Prost, "Embedding prior knowledge within compressed sensing by neural networks, " IEEE Trans. Neural Netw., vol. 22, no. 10, pp. 1638-1649, Oct. 2011.
19. D. Merhej, C. Diab, M. Khalil, and R. Prost, "Embedding prior knowledge within compressed sensing by neural networks, " IEEE Trans. Neural Netw., vol. 22, no. 10, pp. 1638-1649, Oct. 2011.
20. H. Palangi, R. Ward, and L. Deng, "Using deep stacking network to improve structured compressed sensing with multiple measurement vectors, " in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., May 2013, pp. 3337-3341.
21. H. Palangi, R. Ward, and L. Deng, "Using deep stacking network to improve structured compressed sensing with multiple measurement vectors, " in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 2013.
22. A. Mousavi, A. B. Patel, and R. G. Baraniuk, "A deep learning approach to structured signal recovery, " 2015, http://arxiv. org/abs/1508. 04065
23. H. Palangi, R. Ward, and L. Deng, "Exploiting correlations among channels in distributed compressive sensing with convolutional deep stacking networks, " in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), Mar. 2016, pp. 2692-2696.
24. H. Palangi, R. Ward, and L. Deng, "Convolutional deep stacking networks for distributed compressive sensing, " Signal Process., 2016, to be published.
25. N. Vaswani, "Ls-cs-residual (ls-cs): Compressive sensing on least squares residual, " IEEE Trans. Signal Process., vol. 58, no. 8, pp. 4108-4120, 2010.
26. J. Fang, Y. Shen, and H. Li, "Pattern coupled sparse Bayesian learning for recovery of time varying sparse signals, " in Proc. 19th Int. Conf. Digit. Signal Process. (DSP), 2014, pp. 705-709.
27. D. Angelosante, G. Giannakis, and E. Grossi, "Compressed sensing of time-varying signals, " in Proc. 16th Int. Conf. Digit. Signal Process., 2009, pp. 1-8.
28. M. E. Tipping, "Sparse Bayesian learning and the relevance vector machine, " J. Mach. Learn. Res., vol. 1, pp. 211-244, Sep. 2001.
29. A. C. Faul and M. E. Tipping, "Analysis of sparse Bayesian learning, " in Neural Inf. Process. Syst. (NIPS). Cambridge, MA, USA: MIT Press, 2001, pp. 383-389.
30. L. Deng, D. Yu, and J. Platt, "Scalable stacking and learning for building deep architectures, " in Proc. ICASSP, Mar. 2012, pp. 2133-2136.
31. G. E. Hinton, "Training products of experts by minimizing contrastive divergence, " Neural Comput., vol. 14, no. 8, pp. 1771-1800, Aug. 2002.
32. P. Vincent, H. Larochelle, Y. Bengio, and P.-A. Manzagol, "Extracting and composing robust features with denoising autoencoders, " in Proc. ICML, pp. 1096-1103.
33. L. Deng, D. Yu, and J. Platt, "Scalable stacking and learning for building deep architectures, " in Proc. ICASSP, Mar. 2012, pp. 2133. 2136.
34. P. Vincent, "A connection between score matching and denoising autoencoders, " Neural Comput., vol. 23, no. 7, pp. 1661-1674, Jul. 2011.
35. G. Hinton, L. Deng, D. Yu, G. E. Dahl, A. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T. N. Sainath, and B. Kingsbury, "Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups, " IEEE Signal Process. Mag., vol. 29, no. 6, pp. 82-97, Nov. 2012.
36. G. Dahl, D. Yu, L. Deng, and A. Acero, "Context-dependent pretrained deep neural networks for large-vocabulary speech recognition, " IEEE Trans. Audio, Speech, Lang. Process., vol. 20, no. 1, pp. 30-42, Jan. 2012.
37. J. L. Elman, "Finding structure in time, " Cognit. Sci., vol. 14, no. 2, pp. 179-211, 1990.
38. A. J. Robinson, "An application of recurrent nets to phone probability estimation, " IEEE Trans. Neural Netw., vol. 5, no. 2, pp. 298-305, Aug. 1994.
39. L. Deng, K. Hassanein, and M. Elmasry, "Analysis of the correlation structure for a neural predictive model with application to speech recognition, " Neural Netw., vol. 7, no. 2, pp. 331-339, 1994.
40. T. Mikolov, M. Karafiát, L. Burget, J. Cernockỳ, and S. Khudanpur, "Recurrent neural network based language model, " in Proc. INTERSPEECH, Makuhari, Japan, Sep. 2010, pp. 1045-1048.
41. A. Graves, "Sequence transduction with recurrent neural networks, " in Proc. Represent. Learn. Workshop, ICML, 2012.
42. Y. Bengio, N. Boulanger-Lewandowski, and R. Pascanu, "Advances in optimizing recurrent networks, " presented at the ICASSP, Vancouver, May 2013.
43. G. Mesnil, X. He, L. Deng, and Y. Bengio, "Investigation of recurrentneural-network architectures and learning methods for spoken language understanding, " presented at the INTERSPEECH, Lyon, Aug. 2013.
44. H. Palangi, L. Deng, and R. Ward, "Recurrent deep-stacking networks for sequence classification, " in Proc. IEEE China Summit Int. Conf. Signal Inf. Process. (ChinaSIP), Jul. 2014, pp. 510-514.
45. H. Palangi, L. Deng, and R. K. Ward, "Learning input and recurrent weight matrices in echo state networks, " in Proc. NIPS Workshop Deep Learn., Dec. 2013, http://research. microsoft. com/apps/pubs/ default. Aspx?id=204701
46. F. A. Gers, J. Schmidhuber, and F. Cummins, "Learning to forget: Continual prediction with LSTM, " Neural Comput., vol. 12, pp. 2451-2471, 1999.
47. F. A. Gers, N. N. Schraudolph, and J. Schmidhuber, "Learning precise timing with LSTM recurrent networks, " J. Mach. Learn. Res., vol. 3, pp. 115-143, Mar. 2003.
48. Y. Nesterov, "A method of solving a convex programming problem with convergence rate o (1/k2), " Sov. Math. Doklady, vol. 27, pp. 372-376, 1983.
49. H. Palangi, L. Deng, Y. Shen, J. Gao, X. He, J. Chen, X. Song, andR. Ward, "Deep sentence embedding using long shorttermmemory networks: Analysis and application to information retrieval, " IEEE/ACM Trans. Audio, Speech, Lang. Process., vol. 24, no. 4, pp. 694-707, Apr. 2016.
50. I. Sutskever, J. Martens, G. E. Dahl, and G. E. Hinton, "On the importance of initialization and momentum in deep learning, " in Proc. ICML, 2013, pp. 1139-1147.
51. Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition, " Proc. IEEE, vol. 86, no. 11, pp. 2278-2324, Nov. 1998, http://yann. lecun. com/exdb/mnist/
52. "Microsoft image dataset. " [Online]. Available: http://research. microsoft. com/en-us/projects/objectclassrecognition/
53. J. Fang, L. Zhang, and H. Li, "Two-dimensional pattern-coupled sparse Bayesian learning via generalized approximate message passing, " 2015, http://arxiv. org/abs/1505. 06270