Maximum-likelihood estimation, the Cramér-Rao bound, and the method of scoring with parameter constraints

IEEE Transactions on Signal Processing

Volumn 56, Issue 3, 2008, Pages 895-908

  Moore, Terrence J ^a   Sadler, Brian M ^a   Kozick, Richard J ^b  

^a U S ARMY RESEARCH LABORATORY   (United States)

^b BUCKNELL UNIVERSITY   (United States)

Author keywords

Asymptotic normality; Cram r Rao bound; Iterative methods; Maximum likelihood (ML); Method of scoring; Optimization; Parameter estimation; Parametric constraints

Indexed keywords

ASYMPTOTIC ANALYSIS; CONSTRAINT THEORY; CONVERGENCE OF NUMERICAL METHODS; CRAMER-RAO BOUNDS; ITERATIVE METHODS; MAXIMUM LIKELIHOOD ESTIMATION;

ASYMPTOTIC NORMALITY; METHOD OF SCORING; PARAMETRIC CONSTRAINTS;

SIGNAL PROCESSING;

EID: 40749153100     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2007.907814     Document Type: Article

References (33)

1. J. D. Gorman and A. O. Hero, "Lower bounds for parametric estimation with constraints," IEEE Trans. Inf. Theory, vol. 26, no. 6, pp. 1285-1301, Nov. 1990.
2. T. L. Marzetta, "A simple derivation of the constrained multiple parameter Cramer-Rao bound," IEEE Trans. Signal Process., vol. 41, no. 6, pp. 2247-2249, Jun. 1993.
3. P. Stoica and B. C. Ng, "On the Cramér-Rao bound under parametric constraints," IEEE Signal Process. Lett., vol. 5, no. 7, pp. 177-179, Jul. 1998.
4. S. M. Kay, Fundamentals of Statistical Signal Processing, Estimation Theory. Englewood Cliffs, NJ: Prentice-Hall, 1993.
5. M. Crowder, "On constrained maximum likelihood estimation with non-i.i.d. observations," Ann. Inst. Stat. Math., vol. 36, pt. A, pp. 239-249, 1984.
6. M. R. Osborne, "Scoring with constraints," ANZIAM J., vol. 42, no. 1, pp. 9-25, Jul. 2000.
7. M. R. Osborne, "Fisher's method of scoring," Int. Stat. Rev., vol. 60, pp. 99-117, 1992.
8. R. J. Kozick, B. M. Sadler, and T. J. Moore, "Performance of MIMO: CM and semi-Blind cases," in Proc. 2003 4th IEEE Workshop Signal Processing Advances in Wireless Communications, 2003, pp. 309-313.
9. A. Leshem, "Maximum likelihood separation of constant modulus signals," IEEE Trans. Signal Process., vol. 48, no. 10, pp. 2948-2952, Oct. 2000.
10. J. Aitchison and S. D. Silvey, "Maximum-likelihood estimation of parameters subject to restraints," Ann. Math. Statist., vol. 29, pp. 813-828, 1958.
11. J. Aitchison and S. D. Silvey, "Maximum-likelihood estimation procedures and associated tests of significance," J. R. Stat. Soc. B vol. 22, pp. 154-71, 1960.
12. S. P. Han, "A globally convergent method for nonlinear programming," J. Optim. Theory Appl., vol. 22, pp. 297-309, 1977.
13. Z. F. Li, M. R. Osborne, and T. Prvan, "Numerical algorithms for constrained maximum likelihood estimation," ANZIAM J., vol. 45, pp. 91-114, 2003.
14. M. Jamshidian, "On algorithms for restricted maximum likelihood estimation," Comput. Stat. Data Anal., vol. 45, pp. 137-157, 2004.
15. S. M. Kay, Fundamentals of Statistical Signal Processing, Detection Theory. Englewood Cliffs, NJ: Prentice-Hall, 1998.
16. B. M. Sadler, R. J. Kozick, and T. J. Moore, "On the performance of source separation with constant modulus signals," Army Research Lab, Adelphi, MD, Tech. Rep. ARL-TR-3462, Mar. 2005.
17. P. E. Gill, W. Murray, and M. H. Wright, Practical Optimization. New York: Academic, 1981.
18. D. G. Luenberger, Introduction to Linear and Nonlinear Programming Reading, MA: Addison-Wesley, 1973.
19. J. Shao, Mathematical Statistics. New York: Springer-Verlag, 2003.
20. B. Hochwald and A. Nehorai, "On identifiability and information-regularity in parameterized normal distributions," Circuits, Syst., Signal Process., vol. 16, no. 1, pp. 83-89, 1997.
21. P. Stoica and T. L. Marzetta, "Parameter estimation problems with singular information matrices," IEEE Trans. Signal Process., vol. 49, no. 1, pp. 87-90, Jan. 2001.
22. T. Moore, R. Kozick, and B. Sadler, "The constrained Cramer-Rao bound from the perspective of fitting a model," IEEE Signal Process. Lett. vol. 14, no. 8, pp. 564-567, Aug. 2007.
23. C. G. Khatri, "A note on a MANOVA model applied to problems in growth curve," Ann. Inst. Stat. Math., vol. 18, pp. 75-86, 1966.
24. T. Moore and B. Sadler, "Maximum-likelihood estimation and scoring under parametric constraints," Army Research Lab, Aldelphi, MD, Tech. Rep. ARL-TR-3805, May 2006.
25. R. T. Haftka and Z. Gürdal, Elements of Structural Optimization Norwell, MA: Kluwer Academic, 1992.
26. T. K. Moon and W. C. Stirling, Mathematical Methods and Algorithms for Signal Processing. Englewood Cliffs, NJ: Prentice-Hall, 2000.
27. D. P. Bertsekas, Nonlinear Programming. Belmont, MA: Athena Scientific, 1995.
28. A.-J. van der Veen, "Asymptotic properties of the algebraic constant modulus algorithm," IEEE Trans. Signal Process., vol. 49, no. 8, pp. 1796-1807, Aug. 2001.
29. S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge, U.K.: Cambridge Univ. Press, 2004.
30. A. A. Goldstein, "Convex programming in Hilbert space," Bull. Amer. Math. Soc., vol. 70, no. 5, pp. 709-710, 1964.
31. J. R. Kirkwood, An Introduction to Analysis. Boston, MA: PWS Kent, 1995.
32. J. N. Franklin, Matrix Theory. New York: Dover, 1993.
33. T. J. Moore, B. M. Sadler, and R. J. Kozick, "Regularity and strict identifiability in MIMO systems," IEEE Trans. Signal Process., vol. 50, no. 8, pp. 1831-1842, Aug. 2002.