C-HiLasso: A collaborative hierarchical sparse modeling framework

IEEE Transactions on Signal Processing

Volumn 59, Issue 9, 2011, Pages 4183-4198

  Sprechmann, Pablo ^a   Ramírez, Ignacio ^a   Sapiro, Guillermo ^a   Eldar, Yonina C ^b  

^a University of Minnesota   (United States)

^b TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

Author keywords

Collaborative coding; hierarchical models; sparse models; structured sparsity

Indexed keywords

ACTIVE SETS; BASIS PURSUITS; CODED SIGNAL; COLLABORATIVE CODING; FEATURE LEVEL; GLOBAL OPTIMUM; HIERARCHICAL MODEL; LINEAR REGRESSION PROBLEMS; MODELING FRAMEWORKS; NEW MODEL; OPTIMIZATION APPROACH; OPTIMIZATION PROCEDURES; SOURCE IDENTIFICATION; SPARSE MODELS; SPARSITY PATTERNS; STRUCTURED SPARSITY; THEORETICAL RESULT;

DATA PROCESSING; DATA REDUCTION; HIERARCHICAL SYSTEMS;

OPTIMIZATION;

EID: 80051715215     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2011.2157912     Document Type: Article

References (42)

1. M.Yuan and Y. Lin, "Model selection and estimation in regression with grouped variables," J. Roy. Statist. Soc., Series B, vol. 68, pp. 49-67, 2006. (Pubitemid 43415335)
2. R. Jenatton, J. Audibert, and F. Bach, "Structured variable selection with sparsity-inducing norms," 2009 [Online]. Available: http://arxiv. org/pdf/0904.3523
3. Y. C. Eldar and M. Mishali, "Robust recovery of signals from a structured union of subspaces," IEEE Trans. Inf. Theory, vol. 55, no. 11, pp. 5302-5316, Nov. 2009.
4. J. Wright, A. Yang, A. Ganesh, S. Sastry, and Y. Ma, "Robust face recognition via sparse representation," IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 2, pp. 210-227, 2009.
5. J. Tropp, "Algorithms for simultaneous sparse approximation. Part II: Convex relaxation," Signal Process., vol. 86, no. 3, pp. 589-602, 2006. (Pubitemid 43106573)
6. J. Tropp, A. Gilbert, and M. Strauss, "Algorithms for simultaneous sparse approximation. part I: Greedy pursuit," Signal Process., vol. 86, no. 3, pp. 572-588, 2006. (Pubitemid 43106572)
7. S. Cotter, B. Rao, K. Engan, and K. Kreutz-Delgado, "Sparse solutions to linear inverse problems with multiple measurement vectors," IEEE Trans. Signal Process., vol. 53, no. 7, pp. 2477-2488, Jul. 2005. (Pubitemid 41174440)
8. J. Chen and X. Huo, "Theoretical results on sparse representations of multiple-measurement vectors," IEEE Trans. Signal Process., vol. 54, no. 12, pp. 4634-4643, Dec. 2006. (Pubitemid 46405606)
9. M. Mishali and Y. C. Eldar, "Reduce and boost: Recovering arbitrary sets of jointly sparse vectors," IEEE Trans. Signal Process., vol. 56, no. 10, pp. 4692-4702, Oct. 2008.
10. Y. C. Eldar and H. Rauhut, "Average case analysis of multichannel sparse recovery using convex relaxation," IEEE Trans. Inf. Theory, vol. 56, no. 1, pp. 505-519, 2010.
11. Y. Nesterov, "Gradient methods for minimizing composite objective function," in Center for Operations Research and Econometrics (CORE). Louvain-la-Neuve, Belgium: Catholic Univ. of Louvain, 2007, CORE Discussion Paper 2007/76.
12. S. Wright, R. Nowak, and M. Figueiredo, "Sparse reconstruction by separable approximation," IEEE Trans. Signal Process., vol. 57, no. 7, pp. 2479-2493, 2009.
13. J. Friedman, T. Hastie, and R. Tibshirani, "A note on the Group Lasso and a Sparse Group Lasso Stanford Univ., Stanford, CA [Online]. Available: http://www-stat.stanford.edu/~tibs
14. J. Peng, J. Zhu, A. Bergamaschi, W. Han, D. Noh, J. Pollack, and P. Wang, "Regularized multivariate regression for identifying master predictors with application to integrative genomics study of breast cancer," Ann. Appl. Statist., vol. 4, no. 1, pp. 53-77, 2010.
15. S. Kim and E. P. Xing, "Tree-guided group lasso for multi-task regression with structured sparsity," Proc. 27th Int. Conf. Mach. Learning (ICML), pp. 543-550, Jun. 2010.
16. R. Jenatton, J. Mairal, G. Obozinski, and F. Bach, "Proximal methods for sparse hierarchical dictionary learning," ICML, Jun. 2010.
17. J. Starck, M. Elad, and D. Donoho, "Image decomposition via the combination of sparse representations and a variational approach," IEEE Trans. Image Process., vol. 14, no. 10, pp. 1570-1582, 2004. (Pubitemid 41623087)
18. R. Jenatton, J. Mairal, G. Obozinski, and F. Bach, "Proximal methods for hierarchical sparse coding," Sep. 2010 [Online]. Available: http:// arxiv.org/pdf/1009.2139
19. R. Tibshirani, "Regression shrinkage and selection via the LASSO," J. Roy. Stat. Soc., Series B, vol. 58, no. 1, pp. 267-288, 1996.
20. S. Chen, D. Donoho, and M. Saunders, "Atomic decomposition by basis pursuit," SIAM J. Sci. Comput., vol. 20, no. 1, pp. 33-61, 1999. (Pubitemid 128689501)
21. D. Donoho, "Compressed sensing," IEEE Trans. Inf. Theory, vol. 52, no. 4, pp. 1289-1306, Apr. 2006.
22. R. Giryes, M. Elad, and Y. C. Eldar, "The projected GSURE for automatic parameter tuning in iterative shrinkage methods," Appl. Comput. Harmon. Anal., vol. 30, pp. 407-422, 2010.
23. I. Ramírez and G. Sapiro, "Sparse coding and dictionary learning based on the MDL principle," in Proc. Int. Conf. Acoust., Speech, Signal Process. (IEEE ICASSP), 2011, pp. 2160-2163.
24. M. Zhou, H. Chen, J. Paisley, L. Ren, G. Sapiro, and L. Carin, "Nonparametric Bayesian dictionary learning for sparse image representations," in Proc. Adv. Neural Inf. Process. Syst. (NIPS), 2009.
25. B. Turlach, W. Venables, and S. Wright, "Simultaneous variable selection," Technometrics, vol. 27, pp. 349-363, 2004. (Pubitemid 41164070)
26. P. Sprechmann, I. Ramirez, and G. Sapiro, "Collaborative hierarchical sparse modeling," Proc. CISS, Mar. 2010.
27. P. Boufounos, G. Kutyniok, and H. Rauhut, "Sparse recovery from combined fusion frame measurements," 2010 [Online]. Available: http://arxiv.org/pdf/0912.4988v2
28. Y. C. Eldar, P. Kuppinger, and H. Bölcskei, "Block-sparse signals: Uncertainty relations and efficient recovery," IEEE Trans. Signal Process., vol. 58, no. 6, pp. 3042-3054, Jun. 2010.
29. I. Daubechies, M. Defrise, and C. D. Mol, "An iterative thresholding algorithm for linear inverse problems with a sparsity constraint," Comm. Pure Appl. Math., vol. 57, pp. 1413-1457, 2004. (Pubitemid 39427442)
30. E. Candès, J. Romberg, and T. Tao, "Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information," IEEE Trans. Inf. Theory, vol. 52, no. 2, pp. 489-509, Feb. 2006. (Pubitemid 43174093)
31. E. Candès, "The restricted isometry property and its implications for compressed sensing," C. R. Acad. Sci. Paris, I Math., vol. 346, pp. 589-592, 2008.
32. J.Tropp, "Greed is good: Algorithmic results for sparse approximation," IEEE Trans. Inf. Theory, vol. 50, no. 10, pp. 2231-2242, Oct. 2004.
33. M. Stojnic, "Block-length dependent thresholds in block-sparse compressed sensing," Jul. 2009 [Online]. Available: http://arxiv.org/pdf/ 0907.3679
34. A. d'Aspremont, L. E. Ghaoui, M. Jordan, and G. Lanckriet, "A direct formulation for sparse PCA using semidefinite programming," Neural Inf. Process. Syst., vol. 17, pp. 434-448, 2004.
35. H. Zou, T. Hastie, and R. Tibshirani, "Sparse principal component analysis," J. Comput. Graph. Stat., vol. 15, no. 2, pp. 265-286, 2006. (Pubitemid 43935978)
36. B. Moghaddam, Y. Weiss, and S. Avidan, "Spectral bounds for sparse PCA: Exact & greedy algorithms," Neural Inf. Process. Syst., vol. 18, 2006.
37. I. Ramirez, P. Sprechmann, and G. Sapiro, "Classification and clustering via dictionary learning with structured incoherence," presented at the 23rd IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), San Francisco, CA, Jun. 13-18, 2010.
38. M. Zhou, H. Chen, J. Paisley, L. Ren, L. Li, Z. Xing, D. Dunson, G. Sapiro, and L. Carin, "Nonparametric Bayesian dictionary learning for analysis of noisy and incomplete images," Apr. 2010 [Online]. Available: http://www.ima.umn.edu/preprints/apr2010/2307.pdf, IMA Preprint
39. K. Rosenblum, L. Zelnik-Manor, and Y. C. Eldar, "Sensing matrix optimization for block-sparse decoding," Sep. 2010 [Online]. Available: http://arxiv.org/abs/1009.1533
40. N. Shoham and M. Elad, "Alternating KSVD-denoising for texture separation," Proc. IEEE 25th Convention of Electr. Electron. Eng. in Israel, 2008.
41. P. Sprechmann, I. Ramirez, P. Cancela, and G. Sapiro, "Collaborative sources identification in mixed signals via hierarchical sparse modeling," presented at the Int. Conf. Acoust., Speech, Signal Process. (ICASSP), Prague, Czech Republic, May 22-27, 2011.
42. L. Rabiner and B.-H. Juang, Fundamentals of Speech Recognition. Upper Saddle River, NJ: Prentice-Hall, 1993.