Overview of total least-squares methods

Signal Processing

Volumn 87, Issue 10, 2007, Pages 2283-2302

  Markovsky, Ivan ^a   Van Huffel, Sabine ^b  

^a UNIVERSITY OF SOUTHAMPTON   (United Kingdom)

^b UNIVERSITY OF LEUVEN   (Belgium)

Author keywords

Deconvolution; Errors in variables model; Linear prediction; Orthogonal regression; System identification; Total least squares

Indexed keywords

ALGORITHMS; COSTS; DATA STRUCTURES; IDENTIFICATION (CONTROL SYSTEMS); OPTIMIZATION; PROBLEM SOLVING;

ERRORS-IN-VARIABLES MODEL; LINEAR PREDICTION; ORTHOGONAL REGRESSION; TOTAL LEAST SQUARES;

LEAST SQUARES APPROXIMATIONS;

EID: 34249825770     PISSN: 01651684     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.sigpro.2007.04.004     Document Type: Article

References (96)

1. Golub G. Some modified matrix eigenvalue problems. SIAM Rev. 15 (1973) 318-344
2. Golub G., and Van Loan C. An analysis of the total least squares problem. SIAM J. Numer. Anal. 17 (1980) 883-893
3. J. C. Willems, From time series to linear system-Part I. Finite dimensional linear time invariant systems, Part II. Exact modelling, Part III. Approximate modelling, Automatica 22 (1986) 561-580, 22 (1986) 675-694, 23 (1987) 87-115.
4. Markovsky I., Willems J.C., Van Huffel S., and De Moor B. Exact and approximate modeling of linear systems: a behavioral approach. Monographs on Mathematical Modeling and Computation vol. 11 (2006), SIAM, Philadelphia, PA
5. Adcock R. Note on the method of least squares. Analyst 4 (1877) 183-184
6. Adcock R. A problem in least squares. Analyst 5 (1878) 53-54
7. Pearson K. On lines and planes of closest fit to points in space. Philos. Mag. 2 (1901) 559-572
8. Koopmans T. Linear Regression Analysis of Economic Time Series (1937), De Erven F. Bohn
9. Madansky A. The fitting of straight lines when both variables are subject to error. J. Amer. Statist. Assoc. 54 (1959) 173-205
10. York D. Least squares fitting of a straight line. Can. J. Phys. 44 (1966) 1079-1086
11. Sprent P. Models in Regression and Related Topics (1969), Methuen & Co. Ltd.
12. Gleser L. Estimation in a multivariate "errors in variables" regression model: large sample results. Ann. Statist. 9 1 (1981) 24-44
13. J. Staar, Concepts for reliable modelling of linear systems with application to on-line identification of multivariable state space descriptions, Ph.D. Thesis, Department EE, K.U. Leuven, Belgium, 1982.
14. Van Huffel S., and Vandewalle J. Analysis and solution of the nongeneric total least squares problem. SIAM J. Matrix Anal. Appl. 9 (1988) 360-372
15. J. Leuridan, D. De Vis, H. Van Der Auweraer, F. Lembregts, A comparison of some frequency response function measurement techniques, in: Proceedings of the Fourth International Modal Analysis Conference, 1986, pp. 908-918.
16. Levin M. Estimation of a system pulse transfer function in the presence of noise. IEEE Trans. Automat. Control 9 (1964) 229-235
17. Fernando K., and Nicholson H. Identification of linear systems with input and output noise: the Koopmans-Levin method. IEE Proc. D 132 (1985) 30-36
18. Stoica P., and Söderström T. Bias correction in least-squares identification. Int. J. Control 35 3 (1982) 449-457
19. Kumaresan R., and Tufts D. Estimating the angles of arrival of multiple plane waves. IEEE Trans. Aerospace Electronic Systems 19 1 (1983) 134-139
20. Dowling E., and Degroat R. The equivalence of the total least-squares and minimum norm methods. IEEE Trans. Signal Process. 39 (1991) 1891-1892
21. Wentzell P., Andrews D., Hamilton D., Faber K., and Kowalski B. Maximum likelihood principle component analysis. J. Chemometrics 11 (1997) 339-366
22. Schuermans M., Markovsky I., Wentzell P., and Van Huffel S. On the equivalence between total least squares and maximum likelihood PCA. Anal. Chim. Acta 544 (2005) 254-267
23. Van Huffel S., and Vandewalle J. The Total Least Squares Problem: Computational Aspects and Analysis (1991), SIAM, Philadelphia
24. S. Van Huffel (Ed.), Recent Advances in Total Least Squares Techniques and Errors-in-Variables Modeling, SIAM, Philadelphia, 1997.
25. S. Van Huffel, P. Lemmerling (Eds.), Total Least Squares and Errors-in-variables Modeling: Analysis, Algorithms and Applications, Kluwer Academic Publishers, Dordrecht, 2002.
26. Paige C., and Strakos Z. Core problems in linear algebraic systems. SIAM J. Matrix Anal. Appl. 27 (2005) 861-875
27. Fuller W. Measurement error Models (1987), Wiley, New York
28. Degroat R., and Dowling E. The data least squares problem and channel equalization. IEEE Trans. Signal Process. 41 (1991) 407-411
29. Van Huffel S., and Vandewalle J. Analysis and properties of the generalized total least squares problem AX ≈ B when some or all columns in A are subject to error. SIAM J. Matrix Anal. 10 3 (1989) 294-315
30. Van Huffel S., and Zha H. The restricted total least squares problem: formulation, algorithm and properties. SIAM J. Matrix Anal. Appl. 12 2 (1991) 292-309
31. Fierro R., Golub G., Hansen P., and O'Leary D. Regularization by truncated total least squares. SIAM J. Sci. Comput. 18 1 (1997) 1223-1241
32. Golub G., Hansen P., and O'Leary D. Tikhonov regularization and total least squares. SIAM J. Matrix Anal. Appl. 21 1 (1999) 185-194
33. Sima D., Van Huffel S., and Golub G. Regularized total least squares based on quadratic eigenvalue problem solvers. BIT Numer. Math. 44 (2004) 793-812
34. D. Sima, S. Van Huffel, Appropriate cross-validation for regularized errors-in-variables linear models, in: Proceedings of the COMPSTAT 2004 Symposium, Prague, Czech Republic, Physica-Verlag, Springer, Wurzburg, Berlin, August 2004.
35. Beck A., and Ben-Tal A. On the solution of the Tikhonov regularization of the total least squares. SIAM J. Optim. 17 1 (2006) 98-118
36. El Ghaoui L., and Lebret H. Robust solutions to least-squares problems with uncertain data. SIAM J. Matrix Anal. Appl. 18 (1997) 1035-1064
37. Chandrasekaran S., Golub G., Gu M., and Sayed A. Parameter estimation in the presence of bounded data uncertainties. SIAM J. Matrix Anal. Appl. 19 (1998) 235-252
38. Kukush A., and Van Huffel S. Consistency of elementwise-weighted total least squares estimator in a multivariate errors-in-variables model AX = B. Metrika 59 1 (2004) 75-97
39. Arun K. A unitarily constrained total least-squares problem in signal-processing. SIAM J. Matrix Anal. Appl. 13 (1992) 729-745
40. Golub G., and Van Loan C. Matrix Computations. third ed. (1996), Johns Hopkins University Press, Baltimore, MD
41. Abatzoglou T., Mendel J., and Harada G. The constrained total least squares technique and its application to harmonic superresolution. IEEE Trans. Signal Process. 39 (1991) 1070-1087
42. 2 approximation problems. Linear Algebra Appl. 188-189 (1993) 163-207
43. Kukush A., Markovsky I., and Van Huffel S. Consistency of the structured total least squares estimator in a multivariate errors-in-variables model. J. Statist. Plann. Inference 133 2 (2005) 315-358
44. Beck A., and Ben-Tal A. A global solution for the structured total least squares problem with block circulant matrices. SIAM J. Matrix Anal. Appl. 27 1 (2006) 238-255
45. Younan N., and Fan X. Signal restoration via the regularized constrained total least squares. Signal Processing 71 (1998) 85-93
46. Mastronardi N., Lemmerling P., and Van Huffel S. Fast regularized structured total least squares algorithm for solving the basic deconvolution problem. Numer. Linear Algebra Appl. 12 2-3 (2005) 201-209
47. Mesarović V., Galatsanos N., and Katsaggelos A. Regularized constrained total least squares image restoration. IEEE Trans. Image Process. 4 8 (1995) 1096-1108
48. Ng M., Plemmons R., and Pimentel F. A new approach to constrained total least squares image restoration. Linear Algebra Appl. 316 1-3 (2000) 237-258
49. Ng M., Koo J., and Bose N. Constrained total least squares computations for high resolution image reconstruction with multisensors. Int. J. Imaging Systems Technol. 12 (2002) 35-42
50. Rosen J., Park H., and Glick J. Structured total least norm for nonlinear problems. SIAM J. Matrix Anal. Appl. 20 1 (1998) 14-30
51. Lemmerling P., Van Huffel S., and De Moor B. The structured total least squares approach for nonlinearly structured matrices. Numer. Linear Algebra Appl. 9 1-4 (2002) 321-332
52. Mühlich M., and Mester R. The role of total least squares in motion analysis. In: Burkhardt H. (Ed). Proceedings of the Fifth European Conference on Computer Vision (1998), Springer, Berlin 305-321
53. Pruessner A., and O'Leary D. Blind deconvolution using a regularized structured total least norm algorithm. SIAM J. Matrix Anal. Appl. 24 4 (2003) 1018-1037
54. Mastronardi N., Lemmerling P., Kalsi A., O'Leary D., and Van Huffel S. Implementation of the regularized structured total least squares algorithms for blind image deblurring. Linear Algebra Appl. 391 (2004) 203-221
55. Fu H., and Barlow J. A regularized structured total least squares algorithm for high-resolution image reconstruction. Linear Algebra Appl. 391 1 (2004) 75-98
56. Lemmerling P., Mastronardi N., and Van Huffel S. Efficient implementation of a structured total least squares based speech compression method. Linear Algebra Appl. 366 (2003) 295-315
57. Hermus K., Verhelst W., Lemmerling P., Wambacq P., and Van Huffel S. Perceptual audio modeling with exponentially damped sinusoids. Signal Processing 85 (2005) 163-176
58. Verboven P., Guillaume P., Cauberghe B., Parloo E., and Vanlanduit S. Frequency-domain generalized total least-squares identification for modal analysis. J. Sound Vib. 278 1-2 (2004) 21-38
59. Yeredor A. Multiple delays estimation for chirp signals using structured total least squares. Linear Algebra Appl. 391 (2004) 261-286
60. De Moor B., and David J. Total linear least squares and the algebraic Riccati equation. Control Lett. 18 5 (1992) 329-337
61. De Moor B. Total least squares for affinely structured matrices and the noisy realization problem. IEEE Trans. Signal Process. 42 11 (1994) 3104-3113
62. Roorda B., and Heij C. Global total least squares modeling of multivariate time series. IEEE Trans. Automat. Control 40 1 (1995) 50-63
63. Lemmerling P., and De Moor B. Misfit versus latency. Automatica 37 (2001) 2057-2067
64. Pintelon R., Guillaume P., Vandersteen G., and Rolain Y. Analyses, development, and applications of TLS algorithms in frequency domain system identification. SIAM J. Matrix Anal. Appl. 19 4 (1998) 983-1004
65. Markovsky I., Willems J.C., Van Huffel S., De Moor B., and Pintelon R. Application of structured total least squares for system identification and model reduction. IEEE Trans. Automat. Control 50 10 (2005) 1490-1500
66. Branham R. Multivariate orthogonal regression in astronomy. Celestial Mech. Dyn. Astron. 61 3 (1995) 239-251
67. T. Söderström, Errors-in-variables methods in system identification, in: 14th IFAC Symposium on System Identification, Newcastle, Australia, March 2006, pp. 29-41.
68. Laudadio T., Mastronardi N., Vanhamme L., Van Hecke P., and Van Huffel S. Improved Lanczos algorithms for blackbox MRS data quantitation. J. Magn. Res. 157 (2002) 292-297
69. Laudadio T., Selen Y., Vanhamme L., Stoica P., Van Hecke P., and Van Huffel S. Subspace-based MRS data quantitation of multiplets using prior knowledge. J. Magn. Res. 168 (2004) 53-65
70. Fierro R., and Jiang E. Lanczos and the Riemannian SVD in information retrieval applications. Numer. Linear Algebra Appl. 12 (2005) 355-372
71. Schuermans M., Lemmerling P., De Lathauwer L., and Van Huffel S. The use of total least squares data fitting in the shape from moments problem. Signal Process. 86 (2006) 1109-1115
72. L. Zhi, Z. Yang, Computing approximate GCD of univariate polynomials by structure total least norm, in: MM Research Preprints, vol. 24, Acad. Sin. (2004), pp. 375-387.
73. I. Markovsky, S. Van Huffel, An algorithm for approximate common divisor computation, in: Proceedings of the 17th Symposium on Mathematical Theory of Networks and Systems, Kyoto, Japan, 2006, pp. 274-279.
74. Eckart G., and Young G. The approximation of one matrix by another of lower rank. Psychometrika 1 (1936) 211-218
75. Van Huffel S., and Vandewalle J. The total least squares technique: computation, properties and applications. In: Deprettere E. (Ed). SVD and Signal Processing: Algorithms, Applications and Architectures (1988), Elsevier, Amsterdam 189-207
76. Premoli A., and Rastello M.-L. The parametric quadratic form method for solving TLS problems with elementwise weighting. In: Van Huffel S., and Lemmerling P. (Eds). Total Least Squares Techniques and Errors-in-Variables Modeling: Analysis, Algorithms and Applications (2002), Kluwer Academic Publishers, Dordrecht 67-76
77. Markovsky I., Rastello M.-L., Premoli A., Kukush A., and Van Huffel S. The element-wise weighted total least squares problem. Comput. Statist. Data Anal. 50 1 (2005) 181-209
78. Manton J., Mahony R., and Hua Y. The geometry of weighted low-rank approximations. IEEE Trans. Signal Process. 51 2 (2003) 500-514
79. Willems J.C., Rapisarda P., Markovsky I., and De Moor B. A note on persistency of excitation. Control Lett. 54 4 (2005) 325-329
80. Aoki M., and Yue P. On a priori error estimates of some identification methods. IEEE Trans. Automat. Control 15 5 (1970) 541-548
81. Cadzow J. Signal enhancement-a composite property mapping algorithm. IEEE Trans. Signal Process. 36 (1988) 49-62
82. Bresler Y., and Macovski A. Exact maximum likelihood parameter estimation of superimposed exponential signals in noise. IEEE Trans. Acoust. Speech Signal Process. 34 (1986) 1081-1089
83. Tufts D., and Shah A. Estimation of a signal waveform from noisy data using low-rank approximation to a data matrix. IEEE Trans. Signal Process. 41 4 (1993) 1716-1721
84. Tufts D., and Kumaresan R. Estimation of frequencies of multiple sinusoids: making linear prediction perform like maximum likelihood. Proc. IEEE 70 9 (1982) 975-989
85. Kumaresan R., and Tufts D. Estimating the parameters of exponentially damped sinusoids and pole-zero modeling in noise. IEEE Trans. Acoust. Speech Signal Process. 30 6 (1982) 833-840
86. Rosen J., Park H., and Glick J. Total least norm formulation and solution of structured problems. SIAM J. Matrix Anal. Appl. 17 (1996) 110-126
87. Schuermans M., Lemmerling P., and Van Huffel S. Structured weighted low rank approximation. Numer. Linear Algebra Appl. 11 (2004) 609-618
88. Schuermans M., Lemmerling P., and Van Huffel S. Block-row hankel weighted low rank approximation. Numer. Linear Algebra Appl. 13 (2006) 293-302
89. Van Huffel S., Park H., and Rosen J. Formulation and solution of structured total least norm problems for parameter estimation. IEEE Trans. Signal Process. 44 10 (1996) 2464-2474
90. Lemmerling P., Mastronardi N., and Van Huffel S. Fast algorithm for solving the Hankel/Toeplitz structured total least squares problem. Numer. Algorithms 23 (2000) 371-392
91. Mastronardi N., Lemmerling P., and Van Huffel S. Fast structured total least squares algorithm for solving the basic deconvolution problem. SIAM J. Matrix Anal. Appl. 22 (2000) 533-553
92. N. Mastronardi, Fast and reliable algorithms for structured total least squares and related matrix problems, Ph.D. Thesis, ESAT/SISTA, K.U. Leuven, 2001.
93. Markovsky I., Van Huffel S., and Pintelon R. Block-Toeplitz/Hankel structured total least squares. SIAM J. Matrix Anal. Appl. 26 4 (2005) 1083-1099
94. Markovsky I., Van Huffel S., and Kukush A. On the computation of the structured total least squares estimator. Numer. Linear Algebra Appl. 11 (2004) 591-608
95. Markovsky I., and Van Huffel S. On weighted structured total least squares. In: Lirkov I., Margenov S., and Waśniewski J. (Eds). Large-Scale Scientific Computing, Lecture Notes in Computer Science vol. 3743 (2006), Springer, Berlin 695-702
96. Markovsky I., and Van Huffel S. High-performance numerical algorithms and software for structured total least squares. J. Comput. Appl. Math. 180 2 (2005) 311-331