Covariance, subspace, and intrinsic Cramér-Rao bounds

IEEE Transactions on Signal Processing

Volumn 53, Issue 5, 2005, Pages 1610-1630

  Smith, Steven Thomas ^a  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

Adaptive arrays; Adaptive estimation; Adaptive signal processing; Covariance matrices; Differential geometry; Error analysis; Estimation; Estimation bias; Estimation efficiency; Fisher information; Grassmann manifold; Homogeneous space; Matrix decomposition; Maximum likelihood estimation; Natural gradient; Nonlinear estimation; Parameter estimation; Parameter space methods; Positive definitive matrices; Riemannian curvature; Riemannian manifold; Singular value decomposition

Indexed keywords

ARRAYS; ERROR ANALYSIS; MATHEMATICAL MODELS; MATRIX ALGEBRA; MAXIMUM LIKELIHOOD ESTIMATION; PARAMETER ESTIMATION; SIGNAL TO NOISE RATIO; VECTORS;

ADAPTIVE ARRAYS; ADAPTIVE ESTIMATION; ADAPTIVE SIGNAL PROCESSING; COVARIANCE MATRICES; DIFFERENTIAL GEOMETRY; FISHER INFORMATION METRIC; GRASSMANN MANIFOLD; RIEMANNIAN CURVATURE; RIEMANNIAN MANIFOLD; SINGULAR VALUE DECOMPOSITION;

SIGNAL PROCESSING;

EID: 18844440455     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2005.845428     Document Type: Article

References (82)

1. P.-A. Absil, R. Mahoney, and R. Sepulchre, "Riemannian geometry of Grassmann manifolds with a view on algorithmic computation," Acta Applicandae Mathematicae, vol. 80, no. 2, pp. 199-220, Jan. 2004.
2. R. L. Adler, J. P. Dedieu, J. Y. Margulies, M. Martens, and M. Shub, "Newton's method on Riemannian manifolds and a geometric model for the human spine," IMA J. Numer. Anal., vol. 22, pp. 359-390, 2002.
3. S. Amari, "Differential-Geometrical Methods in Statistics," in Lecture Notes in Statistics. Berlin, Germany: Springer-Verlag, 1985, vol. 28.
4. S. Amari, "Differential geometric theory of statistics," in Differential Geometry in Statistical Inference, S. S. Gupta, Ed. Hayward, CA: Inst. Math. Statist., 1987, vol. 10, IMS Lecture Notes - Monograph Series.
5. S. Amari, Methods of Information Geometry. Providence, RI: Amer. Math. Soc., 2000. Originally published in Japanese as Joho kika no hoho (Tokyo, Japan: Iwanami Shoten, 1993).
6. S. Amari, "Information geometry on hierarchy of probability distributions," IEEE Trans. Inf. Theory, vol. 47, no. 5, pp. 1701-1711, Jul. 2001.
7. W. J. Bangs, "Array Processing With Generalized Beamformers," Ph.D. dissertation, Yale Univ., New Haven, CT, 1971.
8. T. A. Barton and S. T. Smith, "Structured covariance estimation for space-time adaptive processing," in Proc. IEEE Conf. Acoust., Speech, Signal Process., Munich, Germany, 1997.
9. R. Bhattacharya and V. Patrangenaru, "Nonparametric estimation of location and dispersion on Riemannian manifolds," J. Statist. Planning Inference, vol. 108, pp. 23-35, 2002.
10. W. M. Boothby, An Introduction to Differential Manifolds and Riemannian Geometry, Second ed. New York: Academic, 1986.
11. D. C. Brody and L. P. Hughston, "Statistical geometry in quantum mechanics," Proc. R. Soc. Lond., vol. A 454, pp. 2445-2475, 1998.
12. I. Chavel, Riemannian Geometry - A Modern Introduction. Cambridge, U.K.: Cambridge Univ. Press, 1993.
13. J. Cheeger and D. G. Ebin, Comparison Theorems in Riemannian Geometry. Amsterdam, The Netherlands: North-Holland, 1975.
14. A. Cichocki and S. Amari, Adaptive Blind Signal and Image Processing. West Sussex, U.K.: Wiley, 2002.
15. A. Cichocki and P. Georgiev, "Blind source separation algorithms with matrix constraints," IEICE Trans. Inf. Syst., vol. E86-A, no. 1, pp. 522-531, Jan. 2003.
16. P. Comon and G. H. Golub, "Tracking a few extreme singular values and vectors in signal processing," Proc. IEEE, vol. 78, no. 8, pp. 1327-1343, Aug. 1990.
17. H. Cramér, Mathematical Methods of Statistics. Princeton, NJ: Princeton Univ. Press, 1946.
18. P. Diaconis, "Group Representations in Probability and Statistics," in IMS Lecture Notes - Monograph Series, S. S. Gupta, Ed. Hayward, CA: Inst. Math. Statist., 1988, vol. 11.
19. S. C. Douglas, "Self-stabilized gradient algorithms for blind source separation with orthogonality constraints," IEEE Trans. Neural Networks, vol. 11, no. 6, pp. 1490-1497. Nov. 2000.
20. S. C. Douglas, S. Amari, and S.-Y. Kung, "On gradient adaption with unit-norm constraints," IEEE Trans. Signal Process., vol. 48, no. 6, pp. 1843-1847, Jun. 2000.
21. A. Edelman and S. T. Smith, "On conjugate-gradient-like methods for eigen-like problems," BIT, vol. 36, no. 3, pp. 494-508, 1996.
22. A. Edelman and S. T. Smith, Linear and Nonlinear Conjugate Gradient-Related Methods, L. Adams and J. L. Nazareth, Eds. Philadelphia, PA: SIAM, 1996.
23. A. Edelman, T. A. Arias, and S. T. Smith, "The geometry of algorithms with orthogonality constraints," SIAM Matrix Anal. Appl., vol. 20, no. 2, pp. 303-353, 1998.
24. K.-T. Fang and Y.-T. Zhang, Generalized Multivariate Analysis. Beijing, China: Science Press, 1990.
25. J. Faraut and A. Korányi, Analysis on Symmetric Cones. London, U.K.: Oxford Univ. Press, 1994.
26. R. A. Fisher, "On the mathematical foundations of theoretical statistics," Phil. Trans. Roy. Soc., ser. A, vol. 222, pp. 309-368, 1922, [Online] http://www.library.adelaide.edu.au/digitised/ fisher/18ptl.pdf.
27. R. A. Fisher, "Dispersion on a sphere," Proc. Roy. Soc. Lond., ser. A, vol. 217, no. 1130, pp. 295-305, 1953.
28. G. H. Golub and C. Van Loan, Matrix Computations, Third ed. Baltimore, MD: Johns Hopkins Univ. Press, 1996.
29. J. D. Gorman and A. O. Hero, "Lower bounds for parametric estimation with constraints," IEEE Trans. Inf. Theory, vol. 26, no. 6, pp. 1285-1301, Nov. 1990.
30. U. Grenander, M. Miller, and A. Srivastava, "Hilbert-Schmidt lower bounds for estimators on matrix Lie groups for ATR," IEEE Trans. Pattern Anal. Machine Intell., vol. 20, no. 8, pp. 790-802, Aug. 1998.
31. S. Helgason, Differential Geometry, Lie Groups, and Symmetric Spaces. New York: Academic, 1978.
32. S. Helgason, Groups and Geometric Analysis. New York: Academic, 1984.
33. U. Helmke and J. B. Moore, Optimization and Dynamical Systems. Berlin, Germany: Springer-Verlag, 1994.
34. H. Hendriks, "A Cramér-Rao type lower bound for estimators with values on a manifold," J. Multivariate Anal., vol. 38, pp. 245-261, 1991.
35. B. M. Hochwald and T. L. Marzetta, "Unitary space-time modulation for multiple-antenna communications in Rayleigh flat fading," IEEE Trans. Inf. Theory, vol. 46, no. 2, pp. 543-564, Mar. 2002.
36. I. M. Johnstone, "On the distribution of the largest eigenvalue in principal components analysis," Ann. Stat., vol. 29, no. 2, pp. 295-327, 2001.
37. H. Karcher, "Riemannian center of mass and mollifier smoothing," Commun. Pure Appl. Math., vol. 30, pp. 509-541, 1977.
38. S. Kobayashi and K. Nomizu, Foundations of Differential Geometry. New York: Interscience, 1969, vol. 1 and 2.
39. J. T.-H. Lo and A. Willsky, "Estimation for rotation with one degree of freedom - Part I: Introduction and continuous time processes," IEEE Trans. Autom. Control, vol. AC-20, no. 1, pp. 10-21, Feb. 1975.
40. E. Lundström and L. Eldén, "Adaptive eigenvalue computations using Newton's method on the Grassmann manifold," SIAM Matrix Anal. Appl., vol. 23, no. 3, pp. 819-839, 2002.
41. Y. Ma, J. Košecká, and S. Sastry, "Optimization criteria and geometric algorithms for motion and structure estimation," Intl. J. Comput. Vision, vol. 44, no. 3, pp. 219-249, 2001.
42. V.A. Marčenko and L. A. Pastur, "Distribution of eigenvalues for some sets of random matrices," Math. USSR-Sb, vol. 1, pp. 457-483, 1967.
43. K. V. Mardia and P. E. Jupp, Directional Statistics. West Sussex, U.K.: Wiley, 2000.
44. P. McCullagh, Tensor Methods in Statistics. London, U.K.: Chapman and Hall, 1987.
45. T. McWhorter and L. L. Scharf, "Cramér-Rao bounds for deterministic modal analysis," IEEE Trans. Signal Process., vol. 41, no. 5, pp. 1847-1866, May 1993.
46. M. L. Menéndez, D. Morales, L. Pardo, and M. Salicrü, "Statistical tests based on geodesic differences," Appl. Math. Lett., vol. 8, no. 1, pp. 65-69, 1995.
47. R. J. Muirhead, Aspects of Multivariate Statistical Theory. New York: Wiley, 1982.
48. M. K. Murray and J. W. Rice, Differential Geometry and Statistics. London, U.K.: Chapman and Hall, 1993.
49. F. D. Neeser and J. L. Massey, "Proper complex random processes with applications to information theory," IEEE Trans. Inf. Theory, vol. 39, no. 7, pp. 1293-1302, Jul. 1993.
50. A. Ohara, N. Suda, and S. Amari, "Dualistic differential geometry of positive definite matrices and its applications to related problems," Lin. Alg. Appl., vol. 247, pp. 31-53, 1996.
51. J. M. Oller, "On an intrinsic bias measure," in Stability Problems for Stochastic Models, V. V. Kalashnikov and V. M. Zolotarev, Eds. Berlin, Germany: Springer-Verlag, 1991, Lecture Notes in Mathematics 1546, pp. 134-158.
52. J. M. Oller and J. M. Corcuera, "Intrinsic analysis of statistical estimation," Ann. Stat., vol. 23, no. 5, pp. 1562-1581, Oct. 1995.
53. D. Petz, "Covariance and Fisher information in quantum mechanics," J. Phys. A: Math. Gen., vol. 35, pp. 929-939, 1995.
54. K. Rahbar and J. P. Reilly, "Blind source separation algorithm for MIMO convolutive mixtures," in Proc. Third Int. Conf. Independent Component Anal. Signal Separation, San Diego, CA, Dec. 2001, pp. 242-247.
55. C. R. Rao, "Information and the accuracy attainable in the estimation of statistical parameters," Bull. Calcutta Math. Soc., vol. 37, pp. 81-89, 1945.
56. C. R. Rao, Selected Papers of C. R. Rao, S. Das Gupta, Ed. New York: Wiley, 1994.
57. C. R. Rao, "Differential metrics in probability spaces," in Differential Geometry in Statistical Inference, S. S. Gupta, Ed. Hayward, CA: Inst. Math. Statist., 1987, vol. 10, IMS Lecture Notes - Monograph Series.
58. I. S. Reed, J. D. Mallett, and L. E. Brennan, "Rapid convergence rate in adaptive arrays," IEEE Trans. Aerosp. Electron. Syst., vol. AES-10, no. 6, pp. 853-863, 1974.
59. M. J. D. Rendas and J. M. F. Moura, "Ambiguity in radar and sonar," IEEE Trans. Signal Process., vol. 46, no. 2, pp. 294-305, Feb. 1998.
60. L. L. Scharf, Statistical Signal Processing: Detection, Estimation, anal Time Series Analysis. Reading, MA: Addison-Wesley, 1991.
61. L. L. Scharf and T. McWhorter, "Geometry of the Cramér-Rao bound," Signal Process., vol. 31, pp. 301-311, 1993.
62. J. W. Silverstein, "The smallest eigenvalue of a large-dimensional Wishart matrix," Ann. Prob., vol. 13, pp. 1364-1368, 1985.
63. J. W. Silverstein and Z. D. Bai, "On the empirical distribution of eigenvalues of a class of large dimensional random matrices," J. Multivariate Anal., vol. 54, pp. 175-192, 1995.
64. L. T. Skovgaard, "A Riemannian geometry of the multivariate normal model," Scand. J. Stat., vol. 11, pp. 211-223, 1984.
65. S. T. Smith, "Dynamical systems that perform the singular value decomposiiton," Syst. Control Lett., vol. 16, pp. 319-327, 1991.
66. S. T. Smith, "Subspace tracking with full rank updates," in Proc. 31st Asilomar Conf. Signals, Syst., Comput., vol. 1, 1997, pp. 793-797.
67. S. T. Smith, "Adaptive radar," in Wiley Encyclopedia of Electrical and Electronics Engineering, J. G. Webster, Ed. New York: Wiley, 1999, vol. 1, pp. 263-289.
68. S. T. Smith, "Intrinsic Cramér-Rao bounds and subspace estimation accuracy," in Proc. IEEE Sensor Array anal Multichannel Signal Processing Workshop, Cambridge, MA, 2000, pp. 489-493.
69. M. Spivak, A Comprehensive Introduction to Differential Geometry, Third ed. Houston, TX: Publish or Perish, 1999, vol. 1, 2.
70. A. Srivastava, "A Bayesian approach to geometric subspace estimation," IEEE Trans. Signal Process., vol. 48, no. 5, pp. 1390-1400, May 2002.
71. A. Srivastava and E. Klassen, "Monte Carlo extrinsic estimators for manifold-valued parameters," IEEE Trans. Signal Process., vol. 50, no. 2, pp. 299-308, Feb. 2002.
72. A. Srivastava and E. Klassen, "Bayesian, geometric subspace tracking," Advances Appl. Prob., to be published.
73. P. Stoica and A. Nehorai, "MUSIC, maximum likelihood, and Cramér-Rao bound," IEEE Trans. Acoust., Speech, Signal Process., vol. 37, no. 5, pp. 720-741, May 1989.
74. P. Stoica and A. Nehorai, "Performance study of conditional and unconditional direction-of-arrival estimation," IEEE Trans. Acoust., Speech, Signal Process., vol. 38, no. 10, pp. 1783-1795, Oct. 1990.
75. P. Stoica and B. C. Ng, "On the Cramér-Rao bound under parametric constraints," IEEE Signal Process. Lett., vol. 5, no. 7, pp. 177-179, Jul. 1998.
76. P. Stoica and T. Söderström, "Statistical analysis of MUSIC and subspace rotation estimates of sinusoidal frequencies," IEEE Trans. Signal Process., vol. 39, no. 8, pp. 1836-1847, Aug. 1991.
77. R. J. Vaccaro and Y. Ding, "Optimal subspace-based parameter estimation," in Proc. IEEE Conf. Acoust., Speech, Signal Process. , vol. 4, Minneapolis, MN, USA, 1993, pp. 368-371.
78. H. L. Van Trees, Detection, Estimation, and Modulation Theory. New York: Wiley, 1968, pt. 1.
79. J. Xavier, "Blind Identification of MIMO Channels Based on 2nd Order Statistics and Colored Inputs," Ph.D. dissertation, Instituto Superior Técnico, Lisbon, Portugal, 2002.
80. J. Xavier, "Geodesic lower bound for parametric estimation with constraints," in Proc. Fifth IEEE Workshop Signal Process. Adv. Wireless Commun., Lisbon, Portugal, Jul. 2004.
81. R. Yang and J. O. Berger, "Estimation of a covariance matrix using the reference prior," Ann. Stat., vol. 22, no. 3, pp. 1195-1211, 1994.
82. L. Zheng and D. N. C. Tse, "Communication on the Grassmann manifold: a geometric approach to the noncoherent multiple-antenna channel," IEEE Trans. Inf. Theory, vol. 48, no. 2, pp. 359-383, Feb. 2002.