Linear regression with Gaussian model uncertainty: Algorithms and bounds

IEEE Transactions on Signal Processing

Volumn 56, Issue 6, 2008, Pages 2194-2205

  Wiesel, Ami ^a   Eldar, Yonina C ^b   Yeredor, Arie ^c  

^a UNIVERSITY OF MICHIGAN   (United States)

^b TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

^c TEL AVIV UNIVERSITY   (Israel)

Author keywords

Errors in variables (EIV); Linear models; Maximum likelihood (ML) estimation; Random model matrix; Total least squares

Indexed keywords

LINEAR REGRESSION; MATHEMATICAL MODELS; MATRIX ALGEBRA; MAXIMUM LIKELIHOOD ESTIMATION; RANDOM PROCESSES; UNCERTAINTY ANALYSIS;

ERRORS IN VARIABLES (EIV); GAUSSIAN MODEL UNCERTAINTY; LINEAR MODELS; RANDOM MODEL MATRIX; TOTAL LEAST SQUARES;

SIGNAL PROCESSING;

EID: 44849089298     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2007.914323     Document Type: Article

References (33)

1. C. R. Rao, Linear Statistical Inference and Its Applications. New York: Wiley, 1973.
2. S. M. Kay, Fundamentals of Statistical Signal Processing - Estimation Theory. Englewood Cliffs, NJ: Prentice-Hall, 1993.
3. A. N. Tikhonov and V. Y. Arsenin, Solution of Ill Posed Problems. Washington, DC: V. H. Winston, 1997.
4. W. James and C. Stein, "Estimation with quadratic loss," in Proc. 4th Berkeley Symp. Math. Statist. Prob., 1961, vol. 1, pp. 311-319.
5. Y. C. Eldar, A. Ben-Tal, and A. Nemirovski, "Robust mean-squared error estimation in the presence of model uncertainties," IEEE Trans. Signal Process., vol. 53, no. 1, pp. 168-181, Jan. 2005.
6. Z. B. Haim and Y. C. Eldar, "Blind minimax estimation," IEEE Trans. Inf. Theory, vol. 53, no. 9, pp. 3145-3157, Sep. 2007.
7. L. E. Ghaoui and H. Lebret, "Robust solutions to least-squares problems with uncertain data," SIAM J. Matrix Anal. Appl., pp. 1034-1064, Oct. 1997.
8. S. Chandrasekaran, M. G. G. H. Golub, and A. H. Sayed, "Parameter estimation in the presence of bounded data uncertainties," SIAM J. Matrix Anal. Appl., pp. 235-252, Jan. 1998.
9. A.Wiesel, Y. C. Eldar, and A. Beck, "Maximum likelihood estimation in linear models with a Gaussian model matrix," IEEE Signal Process. Lett., vol. 13, no. 5, pp. 292-295, May 2006.
10. 2, l ∞ uncertain linear approximation problems using convex optimization, " in Proc. Amer. Control Conf., Jun. 1998, vol. 6 pp. 3487-3491.
11. Y. C. Eldar, "Minimax estimation of deterministic parameters in linear models with a random model matrix," IEEE Trans. Signal Process., vol. 45, no. 2, pp. 601-612, Feb. 2006.
12. W. A. Fuller, Measurement Error Models, ser. Series in Probability and Mathematical Statistics. New York: Wiley, 1987.
13. S. V. Huffel and J. Vandewalle, "The total least squares problem: Computational aspects and analysis," in Frontiers in Applied Mathematics 9. Philadelphia, PA: SIAM, 1991.
14. A. P. Dempster, N. M. Laird, and D. B. Rubin, "Maximum likelihood from incomplete data via the EM algorithm," J. Roy. Statist. Soc. B, Methodological, vol. 39, no. 1, pp. 1-38, Dec. 1977.
15. Y. Li, C. N. Georghiades, and G. Huang, "Iterative maximum-likelihood sequence estimation for space-time coded systems," IEEE Trans. Commun., vol. 49, no. 6, pp. 948-951, Jun. 2001.
16. D. K. C. So and R. S. Cheng, "Iterative EM receiver for space-time coded systems in MIMO frequency-selective fading channels with channel gain and order estimation," IEEE Trans. Wireless Commun., vol. 3, no. 6, pp. 1928-1935, Nov. 2004.
17. P. Stoica and J. Li, "On nonexistence of the maximum likelihood estimate in blind multichannel identification," IEEE Signal Process. Mag., vol. 22, no. 4, pp. 99-101, Jul. 2005.
18. S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge, U.K.: Cambridge Univ. Press, 2003.
19. Y. C. Eldar and A. Beck, "Hidden convexity based near maximum-likelihood CDMA detection," presented at the 39th Annu. Conf. Inf. Sci. Syst., Baltimore, MD, Mar. 16-18, 2005.
20. M. Sion, "On general minimax theorems," Pac. J. Math., vol. 8, pp. 171-176, 1958.
21. P. C. H. G. H. Golub and D. P. O'Leary, "Tikhonov regularization and total least squares," SIAM J. Matrix Anal. Appl., Oct. 1999.
22. A. Beck and A. Ben-Tal, "On the solution of the Tikhonov regularization of the total least squares," SIAM J. Optim., vol. 17, no. 1, pp. 98-118.
23. A. Beck, A. Ben-Tal, and M. Teboulle, "Finding a global optimal solution for a quadratically constrained fractional quadratic problem with applications to the regularized total least squares," SIAM J. Matrix Anal. Appl., vol. 28, no. 2, pp. 425-445, 2006.
24. A. Yeredor, "The joint MAP-ML and its relation to ML and to extended least-squares," IEEE Trans. Signal Process., vol. 48, no. 12, pp. 3484-3492, Dec. 2000.
25. A. Wiesel and Y. C. Eldar, "Maximum likelihood estimation in random linear models: Generalizations and performance analysis," in Proc. IEEE Int. Conf. Acoust. Speech Signal Process., Toulouse, France, May 2006, vol. 5, pp. 993-996.
26. F. Gini, R. Reggiannini, and U. Mengali, "The modified Cramer-Rao bound in vector parameter estimation," IEEE Trans. Commun., vol. 46, no. 1, pp. 2120-2127, Jan. 1998.
27. R. Miller and C. Chang, "A modified Cramer-Rao bound and its applications," IEEE Trans. Inf. Theory, vol. IT-24, no. 3, pp. 398-400, May 1978.
28. I. Reuven and H. Messer, "A Barankin-type lower bound on the estimation error of a hybrid parameter vector," IEEE Trans. Inf. Theory, vol. 43, no. 3, pp. 1084-1093, May 1997.
29. F. Gini and R. Reggiannini, "On the use of Cramer-Rao like bounds in the presence of random nuisance parameters," IEEE Trans. Commun., vol. 48, no. 12, pp. 2120-2127, Dec. 2000.
30. M. J. Levin, "Estimation of a system pulse transfer function in the presence of noise," IEEE Trans. Autom. Control, vol. AC-9, no. 3, pp. 229-235, Jul. 1964.
31. J. J. Fuchs and S. Maria, "A new approach to variable selection using the TLS approach," IEEE Trans. Signal Process., vol. 55, no. 1, pp. 10-19, Jan. 2007.
32. A. Wald, "The fitting of straight lines if both variables are subject to error," Ann. Math. Statist., vol. 11, no. 3, pp. 284-300, Sep. 1940.
33. A. O. Hero, J. A. Fessler, and M. Usman, "Exploring estimator bias-variance tradeoffs using the uniform CR bound," IEEE Trans. Signal Process., vol. 44, no. 8, pp. 2026-2041, Aug. 1996.