Performance analysis of l 0 norm constraint least mean square algorithm

IEEE Transactions on Signal Processing

Volumn 60, Issue 5, 2012, Pages 2223-2235

  Su, Guolong ^a   Jin, Jian ^a   Gu, Yuantao ^a   Wang, Jian ^a  

^a TSINGHUA UNIVERSITY   (China)

Author keywords

Adaptive filter; convergence rate; independence assumption; l 0 LMS; mean square deviation; performance analysis; sparse system identification; steady state misalignment; white Gaussian signal

Indexed keywords

CONVERGENCE RATES; GAUSSIAN SIGNALS; INDEPENDENCE ASSUMPTION; L 0-LMS; MEAN SQUARE DEVIATION; PERFORMANCE ANALYSIS; SPARSE SYSTEM IDENTIFICATION; STEADY-STATE MISALIGNMENT;

ADAPTIVE FILTERS;

ALGORITHMS;

EID: 84859976621     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2012.2184537     Document Type: Article

References (52)

1. B. Widrow and S. D. Stearns, Adaptive Signal Processing. Englewood Cliffs, NJ: Prentice-Hall, 1985.
2. S. Haykin, Adaptive Filter Theory. Englewood Cliffs, NJ: Prentice- Hall, 1986.
3. A. H. Sayed, Adaptive Filters. New York: Wiley, 2008.
4. G. Glentis, K. Berberidis, and S. Theodoridis, "Efficient least squares adaptive algorithms for FIR transversal filtering," IEEE Signal Process. Mag., vol. 16, no. 4, pp. 13-41, Jul. 1999.
5. D. L. Duttweiler, "Proportionate normalized least-mean-squares adaptation in echo cancellers," IEEE Trans. Speech Audio Process., vol. 8, no. 5, pp. 508-518, Sep. 2000.
6. W. F. Schreiber, "Advanced television systems for terrestrial broadcasting: Some problems and some proposed solutions," Proc. IEEE, vol. 83, no. 6, pp. 958-981, Jun. 1995.
7. M. Abadi and J. Husoy, "Mean-square performance of the family of adaptive filters with selective partial updates," Signal Process., vol. 88, no. 8, pp. 2008-2018, Aug. 2008.
8. M. Godavarti and A. O. Hero, "Partial update LMS algorithms," IEEE Trans. Signal Process., vol. 53, no. 7, pp. 2382-2399, Jul. 2005. (Pubitemid 41166921)
9. H. Deng and R. A. Dyba, "Partial update PNLMS algorithm for network echo cancellation," in Proc. Int. Conf. Acoust., Speech, Signal Process. (ICASSP), Taiwan, Apr. 2009, pp. 1329-1332.
10. R.K.Martin, W. A. Sethares, R. C.Williamson, and C. R. Johnson, Jr., "Exploiting sparsity in adaptive filters," IEEE Trans. Signal Process., vol. 50, no. 8, pp. 1883-1894, Aug. 2002.
11. B. D. Rao and B. Song, "Adaptive filtering algorithms for promoting sparsity," in Proc. Int. Conf. Acoust., Speech, Signal Process. (ICASSP), Jun. 2003, pp. 361-364.
12. Y. Chen, Y. Gu, and A. O. Hero, "Sparse LMS for system identification," in Proc. Int. Conf. Acoust., Speech, Signal Process. (ICASSP), Taiwan, Apr. 2009, pp. 3125-3128.
13. Y.Gu, J. Jin, and S. Mei, " Norm constraintLMS algorithmfor sparse system identification," IEEE Signal Process. Lett., vol. 16, no. 9, pp. 774-777, Sep. 2009.
14. B. Babadi, N. Kalouptsidis, and V. Tarokh, "SPARLS: The sparse RLS algorithm," IEEE Trans. Signal Process., vol. 58, no. 8, pp. 4013-4025, Aug. 2010.
15. D. Angelosante, J. A. Bazerque, and G. B. Giannakis, "Online adaptive estimation of sparse signals: Where RLS meets the -norm," IEEE Trans. Signal Process., vol. 58, no. 7, pp. 3436-3447, Jul. 2010.
16. C. Paleologu, J. Benesty, and S. Ciochina, "An improved proportionate NLMS algorithm based on the norm," in Proc. Int. Conf. Acoust., Speech, Signal Process. (ICASSP), Dallas, TX, Mar. 2010, pp. 309-312.
17. Y. Murakami, M. Yamagishi, M. Yukawa, and I. Yamada, "A sparse adaptive filtering using time-varying soft-thresholding techniques," in Proc. Int. Conf. Acoust., Speech, Signal Process. (ICASSP),Dallas, TX, Mar. 2010, pp. 3734-3737.
18. G. Mileounis, B. Babadi, N. Kalouptsidis, and V. Tarokh, "An adaptive Greedy algorithm with application to nonlinear communications," IEEE Trans. Signal Process., vol. 58, no. 6, pp. 2998-3007, Jun. 2010.
19. H. Zayyani, M. Babaie-Zadeh, and C. Jutten, "Compressed sensing block MAP-LMS adaptive filter for sparse channel estimation and a Bayesian Cramer-Rao bound," in Proc. IEEE Int. Workshop Mach. Learning for Signal Process. (MLSP), Sep. 2009, pp. 1-6.
20. Y. Kopsinis, K. Slavakis, and S. Theodoridis, "Online sparse system identification and signal reconstruction using projections onto weighted balls," IEEE Trans. Signal Process., vol. 59, no. 3, pp. 905-930, Mar. 2011.
21. D. L. Donoho, "Compressed sensing," IEEE Trans. Inf. Theory, vol. 52, no. 4, pp. 1289-1306, Apr. 2006.
22. E. Candès, J. Romberg, and T. Tao, "Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information," IEEE Trans. Inf. Theory, vol. 52, no. 2, pp. 489-509, Feb. 2006. (Pubitemid 43174093)
23. E. Candès, "Compressive sampling," in Proc. Int. Congr.Math., Spain, Aug. 2006, vol. 3, pp. 1433-1452.
24. H. Mohimani, M. Babaie-Zadeh, and C. Jutten, "A fast approach for overcomplete sparse decomposition based on smoothed L0 norm," IEEE Trans. Signal Process., vol. 57, no. 1, pp. 289-301, Jan. 2009.
25. H. Mohimani, M. Babaie-Zadeh, I. Gorodnitsky, and C. Jutten, "Sparse recovery using smoothed L0 (SL0): Convergence analysis," IEEE Trans. Inf. Theorey, submitted for publication.
26. H. Zayyani, M. Babaie-Zadeh, and C. Jutten, "An iterative Bayesian algorithm for sparse component analysis (SCA) in presence of noise," IEEE Trans. Signal Process., vol. 57, no. 10, pp. 4378-4390, Oct. 2009.
27. J. Jin, Y. Gu, and S.Mei, "A stochastic gradient approach on compressive sensing signal reconstruction based on adaptive filtering framework," IEEE J. Sel. Topics Signal Process., vol. 4, no. 2, pp. 409-420, Apr. 2010.
28. C. Rich and W. Yin, "Iteratively reweighted algorithms for compressive sensing," in Proc. Int. Conf. Acoust., Speech, Signal Process. (ICASSP), Las Vegas, NV, Apr. 2008, pp. 3869-3872.
29. S. Wright, R. Nowak, and M. Figueiredo, "Sparse reconstruction by separable approximation," in Proc. Int. Conf. Acoust., Speech, Signal Process. (ICASSP), Las Vegas, NV, Apr. 2008, pp. 3373-3376.
30. J. Tropp and A. Gilbert, "Signal recovery from random measurements via orthogonal matching pursuit," IEEE Trans. Inf. Theory, vol. 53, no. 12, pp. 4655-4666, Dec. 2007.
31. K. Mayyas and T. Aboulnasr, "Leaky LMS algorithm: MSE analysis for Gaussian data," IEEE Trans. Signal Process., vol. 45, no. 4, pp. 927-934, Apr. 1997. (Pubitemid 127766019)
32. K. Mayyas, "Performance analysis of the deficient length LMS adaptive algorithm," IEEE Trans. Signal Process., vol. 53, no. 8, pp. 2727-2734, Aug. 2005.
33. O. Dabeer and E. Masry, "Analysis of mean-square error and transient speed of the LMS adaptive algorithm," IEEE Trans. Inf. Theory, vol. 48, no. 7, pp. 1873-1894, Jul. 2002.
34. B. Widrow and M. E. Hoff, "Adaptive switching circuits," IRE WESCON Convention Rec., no. 4, pp. 96-104, 1960.
35. B. Widrow, J. M. McCool, M. C. Larimore, and C. R. Johnson, "Stationary and nonstationary learning characteristics of the LMS adaptive filter," Proc. IEEE, vol. 64, no. 8, pp. 1151-1162, Aug. 1976.
36. L. Horowitz and K. Senne, "Performance advantage of complex LMS for controlling narrow-band adaptive arrays," IEEE Trans. Acoust., Speech, Signal Process., vol. ASSP-29, no. 3, pp. 722-736, Jun. 1981.
37. A. Feuer and E. Weinstein, "Convergence analysis of LMS filters with uncorrelated Gaussian data," IEEE Trans. Acoust., Speech, Signal Process., vol. ASSP-33, no. 1, pp. 222-230, Feb. 1985. (Pubitemid 15479043)
38. S. K. Zhao, Z. H. Man, S. Y. Khoo, and H. R. Wu, "Stability and convergence analysis of transform-domain LMS adaptive filters with second-order autoregressive process," IEEE Trans. Signal Process., vol. 57, no. 1, pp. 119-130, Jan. 2009.
39. A. Ahlen, L. Lindbom, and M. Sternad, "Analysis of stability and performance of adaptation algorithms with time-invariant gains," IEEE Trans. Signal Process., vol. 52, no. 1, pp. 103-116, Jan. 2004.
40. S. C. Chan and Y. Zhou, "Convergence behavior of NLMS algorithm for Gaussian inputs: Solutions using generalized Abelian integral functions and step size selection," J. Signal Process. Syst., vol. 3, pp. 255-265, 2010.
41. D. T. M. Slock, "On the convergence behavior of the LMS and the normalized LMS algorithms," IEEE Trans. Signal Process., vol. 41, no. 9, pp. 2811-2825, Sep. 1993.
42. T. Y. Al-Naffouri and A. H. Sayed, "Transient analysis of adaptive filters with error nonlinearities," IEEE Trans. Signal Process., vol. 51, no. 3, pp. 653-663, Mar. 2003.
43. B. Lin, R. X. He, L. M. Song, and B. S. Wang, "Steady-state performance analysis for adaptive filters with error nonlinearities," in Proc. Int. Conf. Acoust., Speech, Signal Process. (ICASSP), Taiwan, Apr. 2009, pp. 3093-3096.
44. A. Zidouri, "Convergence analysis of a mixed controlled adaptive algorithm," EURASIP J. Adv. Signal Process., vol. 2010, article ID. 893809.
45. S. C. Chan and Y. Zhou, "On the performance analysis of the least mean M-estimate and normalized least mean M-estimate algorithms with Gaussian inputs and additive Gaussian and additive Gaussian and contaminated Gaussian noises," J. Signal Process. Syst., vol. 1, pp. 81-103, 2010.
46. K. Shi and P. Shi, "Convergence analysis of sparse LMS algorithms with l1-norm penalty," Signal Process., vol. 90, no. 12, pp. 3289-3293, Dec. 2010.
47. S. Dasgupta, C. R. Johnson, and A. M. Baksho, "Sign-sign LMS convergence with independent stochastic inputs," IEEE Trans. Inf. Theory, vol. 36, no. 1, pp. 197-201, Jan. 1990. (Pubitemid 20678423)
48. B. E. Jun, D. J. Park, and Y. W. Kim, "Convergence analysis of signsign LMS algorithm for adaptive filters with correlated Gaussian data," in Proc. Int. Conf. Acoust., Speech, Signal Process. (ICASSP), Detroit, MI, May 1995, pp. 1380-1383.
49. S. Koike, "Convergence analysis of a data echo canceler with a stochastic gradient adaptive FIR filter using the sign algorithm," IEEE Trans. Signal Process., vol. 43, no. 12, pp. 2852-2861, Dec. 1995.
50. B. E. Jun and D. J. Park, "Performance analysis of sign-sign algorithm for transversal adaptive filters," Signal Process., vol. 62, no. 3, pp. 323-333, Nov. 1997.
51. G. Su, J. Jin, and Y. Gu, "Performance analysis of -LMS with Gaussian input signal," in Proc. Int. Conf. Signal Process., Beijing, China, Oct. 2010, pp. 235-238.
52. J. Weston et al., "Use of the zero norm with linear models and kernel methods," J. Mach. Learn. Res., pp. 1439-1461, Mar. 2003.