Convergence and stability of iteratively re-weighted least squares algorithms

IEEE Transactions on Signal Processing

Volumn 62, Issue 1, 2014, Pages 183-195

  Ba, Demba ^a,b   Babadi, Behtash ^a,b   Purdon, Patrick L ^a,b   Brown, Emery N ^a,b  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

^b MASSACHUSETTS GENERAL HOSPITAL   (United States)

Author keywords

Compressive sampling; constrained maximum likelihood estimation; expectation maximization algorithms; Gaussian scale mixtures; mathematical programming

Indexed keywords

COMPRESSIVE SAMPLING; CONSTRAINED MAXIMUM LIKELIHOOD ESTIMATIONS; CONVERGENCE AND STABILITY; EXPECTATION-MAXIMIZATION ALGORITHMS; GAUSSIAN SCALE MIXTURES; RE-WEIGHTED LEAST SQUARES; SPARSE SIGNAL RECOVERIES; THEORETICAL GUARANTEES;

ALGORITHMS; MATHEMATICAL PROGRAMMING; MAXIMUM LIKELIHOOD ESTIMATION; MIXTURES; SIGNAL RECONSTRUCTION;

ITERATIVE METHODS;

EID: 84890945200     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2013.2287685     Document Type: Article

References (33)

1. D. L. Donoho, Compressed sensing, IEEE Trans. Inf. Theory, vol. 52, pp. 1289-1306, Apr. 2006.
2. A. Bruckstein, D. Donoho, and M. Elad, From sparse solutions of systems of equations to sparse modeling of signals and images, SIAM Rev., vol. 51, no. 1, pp. 34-81, 2009.
3. W. U. Bajwa, J. Haupt, A. M. Sayeed, and R. Nowak, Compressed channel sensing: A new approach to estimating sparse multipath channels, Proc. IEEE, vol. 98, no. 6, pp. 1058-1076, 2010.
4. B. Babadi, N. Kalouptsidis, and V. Tarokh, SPARLS: The sparse RLS algorithm, IEEE Trans. Signal Process., vol. 58, no. 8, pp. 4013-4025, 2010.
5. M. Lustig, D. Donoho, and J. M. Pauly, Sparse MRI: The application of compressed sensing for rapid MR imaging, Magn. Reson. Med., vol. 58, no. 6, pp. 1182-1195, 2007. (Pubitemid 350234205)
6. R. N. Neelamani, C. E. Krohn, J. R. Krebs, J. K. Romberg, M. Deffenbaugh, and J. E. Anderson, Efficient seismic forward modeling using simultaneous random sources and sparsity, Geophys., vol. 75, no. 6, pp. WB15-WB27, 2010.
7. W. Dai, M. A. Sheikh, O. Milenkovic, and R. G. Baraniuk, Compressive sensing DNA microarrays, EURASIP J. Bioinformat. Syst. Biol., 2009, 2009:162824, (doi:10.1155/2009/162824).
8. I. Gorodnitsky and B. D. Rao, Sparse signal reconstruction from limited data using FOCUSS: a recursive weighted norm minimization algorithm, IEEE Trans. Signal Process., vol. 45, no. 3, pp. 600-616, 1997. (Pubitemid 127765947)
9. D. L. Donoho, M. Elad, and V. N. Temlyakov, Stable recovery of sparse overcomplete representations in the presence of noise, IEEE Trans. Inf. Theory, vol. 52, pp. 6-18, Jan. 2006. (Pubitemid 43263116)
10. R. Saab, R. Chartrand, and O. Yilmaz, Stable sparse approximations via nonconvex optimization, in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 2008, pp. 3885-3888.
11. R. Chartrand and W. Yin, Iteratively reweighted algorithms for compressive sensing, in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 2008, pp. 3869-3872.
12. R. E. Carrillo and K. Barner, Iteratively re-weighted least squares for sparse signal reconstruction from noisy measurements, in Proc. 43rd Annu. Conf. Inf. Sci. Syst. (CISS), Mar. 2009, pp. 448-453.
13. W.Wang,W. Xu, and A. Tang, On the performance of sparse recovery via-minimization, IEEE Trans. Inf. Theory, vol. 57, pp. 7255-7278, Nov. 2011.
14. I. Daubechies, R. DeVore,M. Fornasier, and C. S. Güntürk, Iteratively reweighted least squares minimization for sparse recovery, Comm. Pure Appl. Math., vol. 63, no. 1, pp. 1-38, 2010.
15. S. Voronin, Regularization of linear systems with sparsity constraints with applications to large scale inverse problems, Ph.D. dissertation, Princeton Univ., Princeton, NJ, USA, 2012.
16. D. P. Bertsekas, Nonlinear Programming. Belmont, MA, USA: Athena Scientific, 1999.
17. K. Lange and J. S. Sinsheimer, Normal/independent distributions and their applications in robust regression, J. Comput. Graph. Statist., vol. 2, no. 2, pp. 175-198, 1993.
18. A. P. Dempster, N. M. Laird, and D. B. Rubin, Iteratively reweighted least squares for linear regression when errors are normal/independent distributed, in Multivariate Analysis V, P. R. Krishnaiah, Ed. Amsterdam, The Netherlands: Elsevier Science, 1980, pp. 35-57.
19. P. Huber and E. Ronchetti, Robust Statistics. NewYork,NY,USA: Wiley Online Library, 1981, vol. 1.
20. D. Andrews and C.Mallows, Scale mixtures of normal distributions, J. Roy. Statist. Soc. Series B (Methodol.), pp. 99-102, 1974.
21. A. P. Dempster, N. M. Laird, and D. B. Rubin, Maximum likelihood from incomplete data via the EM algorithm, J. Roy. Statist. Soc. Series B (Methodol.), vol. 39, no. 1, pp. 1-38, 1977.
22. K. Lange, Optimization. New York, NY, USA: Springer, 2004.
23. G. H. Golub and U. v. Matt, Quadratically constrained least squares and quadratic problems, Numer. Math., vol. 59, no. 1, pp. 561-580, 1992.
24. C. F. J. Wu, On the convergence properties of the EM algorithm, Ann. Statist., vol. 11, no. 1, pp. 95-103, 1983.
25. W. I. Zangwill, Nonlinear Programming: a Unified Approach. Englewood Cliffs, NJ, USA: Prentice-Hall, 1969.
26. D. Nettleton, Convergence properties of the EM algorithm in constrained parameter spaces, Can. J. Stat./La Revue Canadienne de Statistique, vol. 27, no. 3, pp. 639-648, 1999.
27. D. P. Bertsekas, A. Nedic, and A. E. Ozdaglar, Convex Analysis and Optimization, 1st ed. Belmont, MA, USA: Athena Scientific, 2003.
28. E. J. Candès and T. Tao, Decoding by linear programming, IEEE Trans. Inf. Theory, vol. 51, pp. 4203-4215, Dec. 2005. (Pubitemid 41800353)
29. Q. Sun, Sparse approximation property and stable recovery of sparse signals from noisy measurements, IEEE Trans. Signal Process., vol. 59, no. 10, pp. 5086-5090, 2011.
30. M. Grant and S. Boyd, CVX: Matlab Software for Disciplined Convex Programming. ver. 1.21, Apr. 2011 [Online]. Available: http://www.stanford.edu/ boyd/software.html
31. M. Grant and S. Boyd, Graph implementations for nonsmooth convex programs, in Recent Advances in Learning and Control, ser. Lecture Notes in Control and Information Science, V. Blondel, S. Boyd, and H. Kimura, Eds. New York, NY, USA: Springer-Verlag Limited, 2008, pp. 95-110.
32. E. J. Candes, M. B. Wakin, and S. P. Boyd, Enhancing sparsity by reweighted minimization, J. Fourier Anal. Appl., vol. 14, no. 5-6, pp. 877-905, 2008.
33. E. J. Candès, J. Romberg, and T. Tao, Stable signal recovery for incomplete and inaccurate measurements, Commun. Pure Appl. Math, vol. 59, pp. 1207-1223, Aug. 2006. (Pubitemid 43988295)