Surveying and comparing simultaneous sparse approximation (or group-lasso) algorithms

Signal Processing

Volumn 91, Issue 7, 2011, Pages 1505-1526

  Rakotomamonjy, A ^a  

^a UNIVERSITÉ DE ROUEN   (France)

Author keywords

Block sparse regression; Group lasso; Iterative reweighted algorithms; Simultaneous sparse approximation

Indexed keywords

APPROXIMATION PROBLEMS; EFFICIENT ALGORITHM; ELEMENTARY FUNCTION; EXPERIMENTAL COMPARISON; GROUP LASSO; ITERATIVE REWEIGHTED ALGORITHMS; MIXED-NORM; OVERCOMPLETE DICTIONARIES; PERFORMANCE MEASURE; SIMULTANEOUS SPARSE APPROXIMATION; SPARSE APPROXIMATIONS; SPARSE REGRESSION; SPARSE SIGNALS; STATISTICAL LEARNING;

COMPUTATIONAL EFFICIENCY; ITERATIVE METHODS; PROBLEM SOLVING; SURVEYS;

APPROXIMATION ALGORITHMS;

EID: 79952039898     PISSN: 01651684     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.sigpro.2011.01.012     Document Type: Review

References (64)

1. R.G. Baraniuk, V. Cevher, M. Duarte, and C. Hegde Model-based compressive sensing IEEE Transactions on Information Theory 56 4 2010 1982 2001
2. D. Bertsekas, A. Nedic, and A. Ozdaglar Convex Analysis and Optimization 2003 Athena Scientific
3. S. Boyd, and L. Vandenberghe Convex Optimization 2004 Cambridge University Press
4. E. Cands, M. Wakin, and S. Boyd Enhancing sparsity by reweighted ℓ 1 minimization Journal of Fourier Analysis and Applications 14 2008 877 905
5. R. Chartrand, W. Yin, Iteratively reweighted algorithms for compressive sensing, in: 33rd International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2008.
6. J. Chen, and X. Huo Sparse representations for multiple measurements vectors (MMV) in an overcomplete dictionary Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing vol. 4 2005 257 260
7. S. Chen, D. Donoho, and M. Saunders Atomic decomposition by basis pursuit SIAM Journal of Scientific Computing 20 1 1999 33 61 (Pubitemid 128689501)
8. S. Cotter, B. Rao, K. Engan, and K. Kreutz-Delgado Sparse solutions to linear inverse problems with multiple measurement vectors IEEE Transactions on Signal Processing 53 7 2005 2477 2488 (Pubitemid 41174440)
9. I. Daubechies, M. Defrise, and C.D. Mol An iterative thresholding algorithm for linear inverse problems with a sparsity constraint Communication on Pure and Applied Mathematics 57 2004 1413 1541
10. I. Daubechies, R. DeVore, M. Fornasier, and S. Gunturk Iteratively reweighted least squares minimization for sparse recovery Communication on Pure and Applied Mathematics 63 1 2010 1 38
11. M. Duarte, V. Cevhler, R. Baraniuk, Model-based compressive sensing for signal ensembles, in: Proceeding of the 47th Allerton Conference on Communication, Control and Computing, 2009.
12. M. Elad Why simple shrinkage is still relevant for redundant representations? IEEE Transactions on Information Theory 52 12 2006 5559 5569 (Pubitemid 46445399)
13. Y. Eldar, P. Kuppinger, and H. Blocskei Compressed sensing of block-sparse signals: uncertainty relations and efficient recovery IEEE Transactions on Signal Processing 58 6 2010 3042 3054
14. Y. Eldar, and M. Mishali Robust recovery of signals from a structured union of subspaces IEEE Transactions on Information Theory 55 11 2009 5302 5316
15. J. Fan, and R. Li Variable selection via nonconcave penalized likelihood and its oracle properties Journal of the American Statistical Association 96 456 2001 1348 1360 (Pubitemid 33695585)
16. M. Figueiredo, J. Bioucas-Dias, R., N., Majorization-minimization algorithms for wavelet-based image for restoration. IEEE Transactions on Image Processing 16(12) (2007) 29802991. (Pubitemid 350201231)
17. M. Fornasier, and H. Rauhut Recovery algorithms for vector valued data with joint sparsity constraints SIAM Journal of Numerical Analysis 46 2 2008 577 613
18. S. Foucart, and M.-J. Lai Sparsest solutions of underdetermined linear systems via ℓ q -minimization for 0 < q ≤ 1 Applied and Computational Harmonic Analysis 26 3 2009 395 407
19. J. Friedman, T. Hastie, H. Hfling, and R. Tibshirani Pathwise coordinate optimization The Annals of Applied Statistics 1 2 2007 302 332
20. G. Gasso, A. Rakotomamonjy, and S. Canu Recovering sparse signals with a certain family of non-convex penalties and dc programming IEEE Transactions on Signal Processing 57 12 2009 4686 4698
21. M. Girolami, and S. Rogers Hierarchic Bayesian models for kernel learning Proceedings of 22nd International Conference on Machine Learning 2005 241 248 (Pubitemid 43183339)
22. I. Gorodnitsky, J. George, and B. Rao Neuromagnetic source imaging with FOCUSS: a recursive weighted minimum norm algorithm Journal of Electroencephalography and Clinical Neurophysiology 95 4 1995 231 251
23. Y. Grandvalet Least absolute shrinkage is equivalent to quadratic penalization L. Niklasson, M. Bodn, T. Ziemske, ICANN'98. Perspectives in Neural Computing vol. 1 1998 Springer 201 206
24. R. Gribonval, and M. Nielsen Sparse approximations in signal and image processing Signal Processing 86 3 2006 415 416 (Pubitemid 43106561)
25. J. Huang, S. Ma, H. Xie, and C. Zhang A group bridge approach for variable selection Biometrika 96 2 2009 339 355
26. D. Hunter, and K. Lange A tutorial on MM algorithms The American Statistician 58 2004 30 37 (Pubitemid 38260327)
27. S. Ji, D. Dunson, and L. Carin Multi-task compressive sensing IEEE Transactions on Signal Processing 57 1 2008 92 106
28. Y. Kim, J. Kim, and Y. Kim Blockwise sparse regression Statistica Sinica 16 2006 375 390 〈 http://www3.stat.sinica.edu.tw/statistica/j16n2/16-2.html 〉 (Pubitemid 44085508)
29. H. Liu, J. Zhang, On the estimation and variable selection consistency of the sum of q-norm regularized regression, Technical Report, Department of Statistics, Carnegie Mellon University, 2009.
30. Z. Luo, M. Gaspar, J. Liu, and A. Swami Distributed signal processing in sensor networks IEEE Signal Processing Magazine 23 4 2006 14 15
31. D. Malioutov, M. Cetin, and A. Willsky Sparse signal reconstruction perspective for source localization with sensor arrays IEEE Transactions on Signal Processing 53 8 2005 3010 3022 (Pubitemid 41372811)
32. S. Mallat, and Z. Zhang Matching pursuit with timefrequency dictionaries IEEE Transactions on Signal Processing 41 12 1993 3397 3415
33. M. Mishali, and Y. Eldar Reduce and boost: recovering arbitrary sets of jointly sparse vectors IEEE Transactions on Signal Processing 56 10 2008 4692 4702
34. D. Needell, and J. Tropp CosAmp: iterative signal recovery from incomplete and inaccurate samples Applied and Computational Harmonic Analysis 26 3 2009 301 321
35. G. Obozinski, B. Taskar, and M. Jordan Joint covariate selection and joint subspace selection for multiple classification problems Statistics and Computing 20 2 2010 231 252
36. Y. Pati, R. Rezaiifar, and P. Krishnaprasad Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition Proceedings of the 27th Annual Asilomar Conference on Signals, Systems and Computers 1993
37. C. Phillips, J. Mattout, M. Rugg, P. Maquet, and K. Friston An empirical Bayesian solution to the source reconstruction problem in EEG NeuroImage 24 2005 997 1011 (Pubitemid 40164502)
38. A. Rakotomamonjy, F. Bach, Y. Grandvalet, and S. Canu SimpleMKL Journal of Machine Learning Research 9 2008 2491 2521
39. A. Rakotomamonjy, R. Flamary, G. Gasso, S. Canu, ℓ p - ℓ q penalty for sparse linear and sparse multiple kernel multi-task learning, IEEE Transactions on Neural Networks, submitted for publication, hal-00509608 〈 http://hal.archives-ouvertes.fr/hal-00509608/fr/ 〉.
40. B. Rao, K. Engan, S. Cotter, J. Palmer, and K. Kreutz-Delgado Subset selection in noise based on diversity measure minimization IEEE Transactions on Signal Processing 51 3 2003 760 770
41. R. Saab, R. Chartrand, zgr Yilmaz, Stable sparse approximations via nonconvex optimization, in: 33rd International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2008.
42. S. Sardy, A. Bruce, and P. Tseng Block coordinate relaxation methods for non-parametric wavelet denoising Journal of Computational and Graphical Statistics 9 2 2000 361 379
43. T. Simila Majorize-minimize algorithm for multiresponse sparse regression IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP) 2007 553 556
44. T. Simila, and J. Tikka Input selection and shrinkage in multiresponse linear regression Computational Statistics and Data analysis 52 1 2007 406 422 (Pubitemid 47331720)
45. M. Stojnic, F. Parvaresh, B. Hassibi, On the reconstruction of block-sparse signals with an optimal number of measurements, Technical Report, ArXiv 0804.0041v1, California Institute of Technology, 2008.
46. J. Sturm Using SeDuMi 1.02 a Matlab toolbox for optimization over symmetric codes Optimization Methods and Software 11 12 1999 625 653
47. L. Sun, J. Liu, J. Chen, J. Ye, Efficient recovery of jointly sparse vectors, in: Advances in Neural Information Processing Systems, vol. 23, 2009.
48. M. Szafranski, Y. Grandvalet, and A. Rakotomamonjy Composite kernel learning Machine Learning Journal 79 1-2 2010 73 103
49. F. Theis, and G. Garcia On the use of sparse signal decomposition in the analysis of multi-channel surface electromyograms Signal Processing 86 3 2006 603 623 (Pubitemid 43106574)
50. R. Tibshirani Regression shrinkage and selection via the lasso Journal of the Royal Statistical Society 46 1996 267 288
51. M. Tipping Sparse Bayesian learning and the relevance vector machine Journal of Machine Learning Research 1 2001 211 244 (Pubitemid 33687203)
52. J. Tropp Algorithms for simultaneous sparse approximation. Part II: convex relaxation Signal Processing 86 2006 589 602 (Pubitemid 43106573)
53. J. Tropp, and A. Gilbert Signal recovery from random measurements via orthogonal matching pursuit IEEE Transactions on Information Theory 53 12 2007 4655 4666
54. J. Tropp, A. Gilbert, and M. Strauss Algorithms for simultaneous sparse approximation. Part I: greedy pursuit Signal Processing 86 2006 572 588 (Pubitemid 43106572)
55. P. Tseng Convergence of block coordinate descent method for nondifferentiable minimization Journal of Optimization Theory and Application 109 2001 475 494
56. E. van den Berg, M. Schmidt, M. Friedlander, K. Murphy, Group sparsity via linear-time projection, Technical Report TR-2008-09, University of British Columbia, Department of Computer Science, 2008.
57. D. Wipf, and S. Nagarajan A new view of automatic relevance determination Advances in Neural Information Processing Systems vol. 20 2008 MIT Press Cambridge, MA
58. D. Wipf, and S. Nagarajan Iterative reweighted ℓ 1 and ℓ 2 methods for finding sparse solutions Journal of Selected Topics in Signal Processing 4 2 2010 317 320
59. D. Wipf, J. Owen, H. Attias, K. Sekihara, and S. Nagarajan Robust Bayesian estimation of the location orientation and time course of multiple correlated neural sources using meg NeuroImage 49 1 2010 641 655
60. D. Wipf, and B. Rao An empirical Bayesian strategy for solving the simultaneous sparse approximation problem IEEE Transactions on Signal Processing 55 7 2007 3704 3716 (Pubitemid 47018850)
61. M. Yuan, and Y. Lin Model selection and estimation in regression with grouped variables Journal of Royal Statistics Society B 68 2006 49 67 (Pubitemid 43415335)
62. H. Zhou, and T. Hastie Regularization and variable selection via the elastic net Journal of the Royal Statistics Society Series B 67 2005 301 320 (Pubitemid 40465877)
63. H. Zou The adaptive lasso and its oracle properties Journal of the American Statistical Association 101 476 2006 1418 1429
64. H. Zou, and R. Li One-step sparse estimates in nonconcave penalized likelihood models The Annals of Statistics 36 4 2008 1509 1533