Analysis of the gradient-descent total least-squares adaptive filtering algorithm

IEEE Transactions on Signal Processing

Volumn 62, Issue 5, 2014, Pages 1256-1264

  Arablouei, Reza ^a   Werner, Stefan ^b   Dogancay, Kutluyil ^a  

^a UNIVERSITY OF SOUTH AUSTRALIA   (Australia)

^b AALTO UNIVERSITY   (Finland)

Author keywords

Adaptive filtering; mean square deviation; performance analysis; Rayleigh quotient; stability; total least squares

Indexed keywords

ADAPTIVE FILTERING ALGORITHMS; INPUT AND OUTPUTS; MEAN SQUARE DEVIATION; PERFORMANCE ANALYSIS; RAYLEIGH QUOTIENTS; STEADY STATE PERFORMANCE; STEADY-STATE ANALYSIS; TOTAL LEAST SQUARES;

ALGORITHMS; APPROXIMATION THEORY; CONVERGENCE OF NUMERICAL METHODS; SEEBECK EFFECT;

ADAPTIVE FILTERING;

EID: 84896823872     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2014.2301135     Document Type: Article

References (43)

1. P. S. R. Diniz, Adaptive Filtering: Algorithms and Practical Implementations, 3rd ed. Boston, MA, USA: Springer, 2008.
2. S. Van Huffel and J. Vandewalle, The Total Least Squares Problem: Computational Aspects and Analysis. Philadelphia, PA, USA: SIAM, 1991.
3. I. Markovsky and S. Van Huffel, "Overview of total least-squares methods, " Signal Process., vol. 87, pp. 2283-2302, 2007. (Pubitemid 46855921)
4. S. Van Huffel, "Analysis of the Total Least Squares Problem and Its Use in Parameter Estimation, " Ph.D. dissertation, Dept. Elect. Eng., Katholieke Univ., Leuven, Belgium, 1987.
5. K. Pearson, "On lines and planes of closest fit to points in space, " Philosph Mag., vol. 2, pp. 559-572, 1901.
6. G. H. Golub and C. F. Van Loan, "An analysis of the total least squares problem, " SIAM J. Numerical Anal., vol. 17, no. 6, pp. 883-893, Dec. 1980.
7. E. Oja, "Neural networks, principal components, subspaces, " Int. J. Neural Syst., vol. 1, pp. 61-68, 1989.
8. E. Oja, "Principal components, minor components, linear neural networks, " Neural Netw., vol. 5, pp. 927-935, 1992. (Pubitemid 23581250)
9. L. Xu, E. Oja, and C. Suen, "Modified Hebbian learning for curve and surface fitting, " Neural Netw., vol. 5, pp. 441-457, 1992.
10. K. Gao, O. Ahmad, and M. N. S. Swamy, "A constrained anti-Hebbian learning algorithm for total least-squares estimation with applications to adaptive FIR and IIR filtering, " IEEE Trans. Circuits Syst. II, Analog Digit. Signal Process., vol. 41, no. 11, pp. 718-729, Nov. 1994.
11. D. Z. Feng, Z. Bao, and L. C. Jiao, "Total least mean squares algorithm, " IEEE Trans. Signal Process., vol. 46, no. 8, pp. 2122-2130, Aug. 1998. (Pubitemid 128744032)
12. A. Taleb and G. Cirrincione, "Against the convergence of the minor component analysis neurons, " IEEE Trans. Neural Netw., vol. 10, no. 1, pp. 207-210, Jan. 1999.
13. G. Cirrincione, M. Cirrincione, J. Herault, and S. Van Huffel, "The MCA EXIN neuron for the minor component analysis, " IEEE Trans. Neural Netw., vol. 13, no. 1, pp. 160-187, Jan. 2002. (Pubitemid 34236845)
14. D. M. Sima and S. Van Huffel, "Minor component analysis by incremental inverse iteration, " in Proc. 16th Int. Symp. Math. Theory Networks Syst., Leuven, Belgium, Jul. 2004, pp. 1-12.
15. G. Cirrincione and M. Cirrincione, Neural-Based Orthogonal Data Fitting: The EXIN Neural Networks. Hoboken, NJ, USA: Wiley, 2010.
16. G. H. Golub and C. F. Van Loan, Matrix Computations, 3rd ed. Baltimore, MD, USA: Johns Hopkins Univ. Press, 1996.
17. L. N. Trefethen and D. Bau, Numerical Linear Algebra. Philadelphia, PA, USA: SIAM, 1997.
18. B. N. Parlett, "The Rayleigh quotient iteration and some generalizations for nonnormal matrices, " Math. Comput., vol. 28, no. 127, pp. 679-693, Jul. 1974.
19. D.-Z. Feng and W. X. Zheng, "Fast approximate inverse power iteration algorithm for adaptive total least-squares FIR filtering, " IEEE Trans. Signal Process., vol. 54, no. 10, pp. 4032-4039, Oct. 2006. (Pubitemid 44500854)
20. P.-A. Absil, R. Mahony, and R. Sepulchre, Optimization Algorithms on Matrix Manifolds. Princeton, NJ, USA: Princeton Univ. Press, 2008.
21. C. E. Davila, "An efficient recursive total least squares algorithm for FIR adaptive filtering, " IEEE Trans. Signal Process., vol. 42, no. 2, pp. 268-280, Feb. 1994.
22. D.-Z. Feng, X.-D. Zhang, D.-X. Chang, and W. X. Zheng, "A fast recursive total least squares algorithm for adaptive FIR filtering, " IEEE Trans. Signal Process., vol. 52, no. 10, pp. 2729-2737, Oct. 2004.
23. X. Yang, "A survey of various conjugate gradient algorithms for iterative solution of the largest/smallest eigenvalue and eigenvector of a symmetric matrix, " Progress Electromagn. Res., vol. 5, pp. 567-588, 1991.
24. X. Yang, "Conjugate gradient methods for the Rayleigh quotient minimization of generalized eigenvalue problems, " Computing, vol. 51, no. 1, pp. 79-94, 1993.
25. A. Edelman and S. T. Smith, "On conjugate gradient-like methods for eigen-like problems, " BIT Numerical Math., vol. 36, no. 3, pp. 494-508, Sep. 1996. (Pubitemid 126419083)
26. W. Zhu, Y. Wang, Y. Yao, J. Chang, H. L. Graber, and R. L. Barbour, "Iterative total least squares image reconstruction algorithm for optical tomography by the conjugate gradient method, " J. Optical Soc. Amer., vol. 14, no. 4, pp. 799-807, Apr. 1997. (Pubitemid 127579201)
27. R. H. Byrd and J. Nocedal, "A tool for the analysis of quasi-Newton methods with application to unconstrained minimization, " SIAM J. Numerical Anal., vol. 26, no. 3, pp. 727-739, Jun. 1989.
28. B. E. Dunne and G. A. Williamson, "Analysis of gradient algorithms for TLS-based adaptive IIR filters, " IEEE Trans. Signal Process., vol. 52, no. 12, pp. 3345-3356, Dec. 2004.
29. H. C. So and Y. T. Chan, "Analysis of an LMS algorithm for unbiased impulse response estimation, " IEEE Trans. Signal Process., vol. 51, no. 7, pp. 2008-2013, Jul. 2003.
30. X. Kong, C. Han, and R. Wei, "Modified gradient algorithm for total least square filtering, " Neurocomputing, vol. 70, no. 1-3, pp. 568-576, Dec. 2006. (Pubitemid 44615764)
31. X. Kong, C. Hu, and C. Han, "A self-stabilizing neural algorithm for total least squares filtering, " Neural Process. Lett., vol. 30, no. 3, pp. 257-271, Dec. 2009.
32. D. P. Bertsekas and J. N. Tsitsiklis, "Gradient convergence in gradient methods with errors, " SIAM J. Optimization, vol. 10, no. 3, pp. 627-642, 1999.
33. M. Rupp and A. H. Sayed, "A time-domain feedback analysis of fil-tered-error adaptive gradient algorithms, " IEEE Trans. Signal Process., vol. 44, no. 6, pp. 1428-1439, Jun. 1996. (Pubitemid 126777960)
34. N. R. Yousef and A. H. Sayed, "A unified approach to the steady-state and tracking analyses of adaptive filters, " IEEE Trans. Signal Process., vol. 49, no. 2, pp. 314-324, Feb. 2001.
35. A. H. Sayed, Adaptive Filters. Hoboken, NJ, USA: Wiley, 2008.
36. B. N. Parlett, The Symmetric Eigenvalue Problem. Englewood Cliffs, NJ, USA: Prentice-Hall, 1980.
37. C. T. Kelley, Iterative Methods for Optimization. Philadelphia, PA, USA: SIAM, 1999.
38. K. M. Abadir and J. R. Magnus, Matrix Algebra. New York, NY, USA: Cambridge Univ. Press, 2005.
39. S. Jo and S. W. Kim, "Consistent normalized least mean square filtering with noisy data matrix, " IEEE Trans. Signal Process., vol. 53, no. 6, pp. 2112-2123, Jun. 2005. (Pubitemid 40843767)
40. S. Haykin, Adaptive Filter Theory, 4th ed. Upper Saddle River, NJ, USA: Prentice-Hall, 2002.
41. R. A. Hom and C. A. Johnson, Matrix Analysis. New York, NY, USA: Cambridge Univ. Press, 1985.
42. N. L. Johnson, S. Kotz, and N. Balakrishnan, Continuous Univariate Distributions, 2nd ed. New York, NY, USA: Wiley, 1994, vol. 1.
43. B. D. O. Anderson, R. R. Bitmead, C. R. Johnson, Jr., P. V. Kokotovic, R. L. Kosut, I. M. Y. Mareels, L. Praly, and B. D. Riedle, Stability of Adaptive Systems: Passivity and Averaging Analysis. Cambridge, MA, USA: MIT Press, 1986.