Relative entropy and the multivariable multidimensional moment problem

IEEE Transactions on Information Theory

Volumn 52, Issue 3, 2006, Pages 1052-1066

  Georgiou, Tryphon T ^a,b  

^a IEEE   (United States)

^b University of Minnesota   (United States)

Author keywords

Covariance realization; Moment problem; Multidimensional; Multivariable; Quantum entropy; Spectral analysis

Indexed keywords

COVARIANCE REALIZATION; MULTIVARIABLE MULTIDIMENSIONAL MOMENT PROBLEM; QUANTUM ENTROPY; RELATIVE ENTROPY;

ALGEBRA; FUNCTION EVALUATION; INFORMATION THEORY; INVERSE PROBLEMS; OPTIMIZATION; SPECTRUM ANALYSIS;

PROBLEM SOLVING;

EID: 33746035133     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2005.864422     Document Type: Article

References (81)

1. N.I. Akhiezer, The Classical Moment Problem. London, U.K.: Hafner Publishing, Oliver and Boyd, 1965.
2. N. I. Akhiezer and M. G. Krein, Some Questions in the Theory of Moments. Providence, RI: Amer. Mach. Soc., 1962.
3. H. Araki, "Relative entropy of states of von Neumann algebras," Publ. RIMS, Kyoto Univ., vol. 11, pp. 809-833, 1976.
4. 0)-inner matrix-valued functions," Russian Acad. Sci. Izv. Math., vol. 42, no. 1, pp. 1-26, 1994.
5. G. A. Bliss, Lectures on the Calculus of Variations. Chicago, IL: Univ.Chicago Press, 1946.
6. A. Blomqvist, A. Lindquist, and R. Nagamune, "Matrix-valued Nevanlinna-pick interpolation with complexity constraint: An optimization approach," IEEE Trans. Autom. Control, vol. 48, no. 12, pp. 2172-2190, Dec. 2003.
7. S. Boyd, L. El Ghaoui, E. Feron, and V. Balakrishnan, Linear Matrix Inequalities in Systems and Control Theory. Philadelphia, PA: SIAM, June 1994, vol. 15, Studies in Applied Mathematics.
8. V. B. Braginsky and F. Ya. Khalili, Quantum Measurement. Cambridge, U.K.: Cambridge Univ. Press, 1992.
9. J. Burg, "Maximum Entropy Spectral Analysis," Ph.D. dissertation, Stanford Univ., Stanford, CA, 1975.
10. C. I. Byrnes and A. Lindquist, "A convex optimization approach to generalized moment problems," in Control and Modeling of Complex Systems: Cybernetics in the 21st Century. Boston, MA: Birhäuser, 2003, pp. 3-21.
11. _, "Interior Point Solutions of Variational Problems and Global Inverse Function Theorems," Roy. Inst. Technol. (KTH), Stockholm, Sweden, Rep. TRITA/MAT01-OS13, 2001.
12. C. Byrnes, T. T. Georgiou, and A. Lindquist, "A generalized entropy criterion for Nevanlinna-Pick interpolation: A convex optimization approach to certain problems in systems and control," IEEE Trans. Autom. Control, vol. 45, no. 6, pp. 822-839, Jun. 2001.
13. _, "A new approach to spectral estimation: A tunable high-resolution spectral estimator," IEEE Trans. Signal Process., vol. 48, no. 11, pp. 3189-3206, Nov. 2000.
14. C. I. Byrnes, S. V. Gusev, and A. Lindquist, "A convex optimization approach to the rational covariance extension problem," SIAM Rev., vol. 43, no. 4, pp. 645-675, 2001.
15. _, "From finite covariance windows to modeling filters: A convex optimization approach," SIAM J. Control and Opt., vol. 37, pp. 211-229, 1999.
16. ∞ with a complexity constraint," Trans. Amer. Math. Soc., to be published.
17. D. Dacunha-Castelle and F. Gamboa, "Maximization de l'entropie sous contraintes non linèaires," Annales de l'Inst. H. Poincarre, vol. 4, pp. 567-596, 1990.
18. I. Csiszár, "I-divergence geometry of probability distributions and minimization problems," Ann. Probab., vol. 3, no. 1, pp. 146-158, 1975.
19. _, "Why least squares and maximum entropy? An axiomatic approach to inference for linear inverse problems," Ann. Probab., vol. 19, no. 4, pp. 2032-2066, 1991.
20. _, "Sanov property, generalized I-projection and a conditional limit theorem," Ann. Probab., vol. 3, pp. 146-158, 1984.
21. B. W. Dickinson, "Two dimensional Markov spectrum estimates need not exist," IEEE Trans. Inf. Theory, vol. IT-26, no. 1, pp. 120-121, Jan. 1980.
22. P. Delsarte, Y. Genin, and Y. Kamp, "Half-plane Toeplitz systems," IEEE Trans. Inf. Theory, vol. IT-26, no. 4, pp. 465-474, Jul. 1980.
23. P. Dewilde and H. Dym, "Lossless chain scattering matrices and optimum linear prediction: The vector case," Circuit Theory and Applications, vol. 9, pp. 135-175, 1981.
24. W. F. Donoghue Jr., Monotone Matrix Functions and Analytic Continuation. New York: Springer-Verlag, 1974.
25. P. Enqvist, "A homotopy approach to rational covariance extension with degree constraint," Int. J. Appl. Math. Comput. Sci., vol. 11, no. 5, pp. 1173-1201, 2001.
26. R. P. Feynman, "An operator calculus having applications in quantum electrodynamics," Phys. Rev., vol. 84, no. 1, pp. 108-128, 1951.
27. F. Gamboa and E. Gassiat (Orsay), "Maximum d'entropie et problème des moments: cas multidimensionnel," Probab. Math. Statist., vol. 12, pp. 67-83, 1991.
28. T. T. Georgiou, "Partial Realization of Covariance Sequences," Ph.D. dissertation, Elec. Eng., Univ. Florida, Gainesville, 1983. Available:. [Online] at www.ece.umn.edu/users/georgiou/papers/dissertation.pdf.
29. _, "Realization of power spectra from partial covariance sequences," IEEE Trans. Acous., Speech, Signal Process., vol. AASP-35, no. 4, pp. 438-449, Apr. 1987.
30. _, "Signal estimation via selective harmonic amplification: MUSIC, redux," IEEE Trans. Signal Process., vol. 48, no. 3, pp. 780-790, Mar. 2000.
31. _, "A topological approach to Nevanlinna-Pick interpolation," SIAM J. Math. Anal., vol. 18, no. 5, pp. 1248-1260, Sep. 1987.
32. _, "The interpolation problem with a degree constraint," IEEE Trans. Autom. Control, vol. 44, no. 3, pp. 631-635, Mar. 1999.
33. _, "The structure of state covariances and its relation to the power spectrum of the input," IEEE Trans. Autom. Control, vol. 47, no. 7, pp. 1056-1066, Jul. 2002.
34. _, "Spectral analysis based on the state covariance: The maximum entropy spectrum and linear fractional parameterization," IEEE Trans. Autom. Control, vol. 47, no. 11, pp. 1811-1823, Nov. 2002.
35. _, "Solution of the general moment problem via a one-parameter imbedding," IEEE Trans. Autom. Control, vol. 50, no. 6, pp. 811-826, Jun. 2005.
36. _, "Toeplitz covariance matrices and the von Neumann relative entropy," in Control and Modeling of Complex Systems: Cybernetics in the 21st Century. Boston, MA: Birkhauser, 2003, pp. 23-30.
37. _, Structured Covariances and Related Approximation Questions (Lecture Notes in Control and Information Sciences). New York: Springer-Verlag, 2003, vol. 286, pp. 135-140.
38. T. T. Georgiou and A. Lindquist, "Kullback-Leibler approximation of spectral density functions," IEEE Trans.Inf. Theory, vol. 49, no. 11, pp. 2910-2917, Nov. 2003.
39. _, "Two alternative views on control design with degree constraint," IEEE Trans. Autom. Control, submitted for publication.
40. A. Graham, Kronecker Products and Matrix Calculus With Applications. New York: Halsted Press/Wiley, 1981.
41. S. Haykin, Nonlinear Methods of Spectral Analysis. New York: Springer-Verlag, 1979.
42. S. Haykin, J. H. Justice, N. L. Owsley, J. L. Yen, and A. C. Kak, Array Signal Processing. Englewood Cliffs, NJ: Prentice-Hall, 1985.
43. J. J. Halliwell, J. Pérez-Mercader, and W. H. Zurek, Physical Origins of Time Asymetry. Cambridge, U.K.: Cambridge Univ. Press, 1994.
44. S. P. Heims and E. T. Jaynes, "Theory of gyromagnetic effects and some related magnetic phenomena," Rev. Mod. Phys., vol. 34, no. 2, pp. 143-165, 1962.
45. D. Jackson, "Formal properties of orthogonal polynomials in two variables," Duke Math. J., vol. 2, pp. 423-434, 1936.
46. E. T. Jaynes, "On the rationale of maximum entropy methods," Proc. IEEE, vol. 70, no. 9, pp. 939-952, Sep. 1982.
47. D. H. Johnson and D. E. Dudgeon, Array Signal Processing: Concepts and Techniques. Englewood Cliffs, NJ: Prentice-Hall, 1993.
48. M. Junk, "Maximum entropy for reduced moment problems," Math. Models and Methods in Appl. Sci., vol. 10, no. 7, pp. 1001-1025, 2000.
49. R. E. Kalman, "Realization of covariance sequences," in Toeplitz Centennial, I. C. Gohberg, Ed. Basel, Switzerland: Birkhauser, 1982, pp. 331-342.
50. C. Kittel, Thermal Physics. New York: Wiley, 1969.
51. M. G. Krein and A. A. Nudel'man, The Markov Moment Problem and Extremal Problems. Providence, RI: Amer. Math. Soc., 1977.
52. S. Kullback and R. A. Leibler, "On information and sufficiency," Ann. Math. Statist., vol. 22, no. 1, p. 7986, Mar. 1951.
53. S. W. Lang and J. H. McClellan, "Multidimensional MEM spectral estimatiion," IEEE Trans. Acoust., Speech, Signal Process., vol. ASSP-30, no. 6, pp. 880-887, Dec. 1982.
54. R. D. Levine and M. Tribus, Eds., The Maximum Entropy Formalism. Cambridge, MA: MIT Press, 1979.
55. G. Le Besnerais, J.-F. Bercher, and G. Demoment, "A new look at entropy for solving linear inverse problems," IEEE Trans. Inf. Theory, vol. 45, no. 5, pp. 1565-1578, Jul. 1999.
56. A. S. Lewis, "Consistency of moment systems," Can. J. Math., vol. 47, no. 5, pp. 995-1066, 1995.
57. E. H. Lieb, "Convex trace functions and Wigner-Yanase-Dyson conjecture," Adv. Math., vol. 11, pp. 267-288, 1973.
58. A. Lesniewski and M. B. Ruskai, "Relative entropy and monotone Riemannian metrics on noncommutative probability spaces," J. Math. Phys., vol. 40, pp. 5702-5724, 1999.
59. T. L. Marzetta, "Two-dimensional linear prediction: Autocorrelation arrays, minimum-phase predition error filters, and reflection coefficient arrays," IEEE Trans. Acoust., Speech Signal Process., vol. ASSP-28, no. 6. pp. 725-733, Dec. 1980.
60. P. Masani, "Recent trends in multivariate prediction theory," in Multivariate Analysis, P. R. Krishnaiah, Ed., New York, 1966, pp. 351-382.
61. M. I. Miller and D. L. Snyder, "The role of likelihood and entropy in incomplete-data problems: Applications to estimating poin-process intensities and Toeplitz constrained covariances," Proc. IEEE, vol. 75, no. 7. pp. 892-907, Jul. 1987.
62. I. Najfeld and T. F. Havel, "Derivatives of the matrix exponential and their computation," Adv. Appl. Math., vol. 16, pp. 321-375, 1995.
63. Y. Nesterov and A. Nemirovskii, Interior-Point Polynomial Algorithms in Convex Programming. Philadelphia, PA: SIAM, 1994.
64. J. von Neumann, "Thermodynamik quantenmechanischer Gesamtheiten," Nachr. der Gesellschaft der Wiss. Gött., pp. 273-291, 1927.
65. M. A. Nielsen and I. L. Chuang, Quantum Computation and Quantum Information. Cambridge, U.K.: Cambridge Univ. Press, 2000.
66. M. Ohya and D. Petz, Relative Entropy and Its Use. Berlin, Germany: Springer-Verlag, 1993.
67. D. Petz and Cs. Sudár, "Extending the Fisher metric to density matrices," in Geometry in Present Days Science. Singapore: World Scientific, 1999, pp. 21-34.
68. D. Petz, "Geometry of canonical correlation on the state space of a quantum system," J. Math. Phys., vol. 35, no. 2, pp. 780-795, 1994.
69. _, "Quasi-entropies for finite quantum systems," Rep. Math. Phys., vol. 23, no. 1, pp.57-65, 1986.
70. F. K. Bose, Ed., "Special Issue on Multidimensional Systems," Proc. IEEE, vol. 65, no. 6, Jun. 1977.
71. M. Rosenblum and J. Rovnyak, Hardy Classes and Operator Theory. Oxford, U.K.: Oxford Univ. Press, 1985.
72. W. Rudin, Real and Complex Analysis, 3rd ed. New York: McGraw-Hill, 1987.
73. _, "The extension problem for positive-definite functions," Illinois J. Math., vol. 7, pp. 532-539, 1963.
74. 1-pK and Remarks on Related Inequalities," arXiv: quant-ph/0404126v1, Aug. 30, 2004.
75. J. A. Shohat and J. D. Tamarkin, The Problem of Moments. New York: Amer. Math. Society, 1943.
76. P. Stoica and R. Moses, Introduction to Spectral Analysis. Englewood Cliffs, NJ: Prentice-Hall, 1997.
77. A. Sudbery, Quantum Mechanics and the Particles of Nature. Cambridge, U.K.: Cambridge Univ. Press, 1986.
78. H. Umegaki, "Conditional expectation in an operator algebra IV. Entropy and Information," Kodai Math. Sem. Rep., vol. 14, pp. 59-85, 1962.
79. H. L. Van Trees, Optimum Array Processing. New York: Wiley-Interscience, 2002, Part IV of Detection, Estimation and Modulation Theory.
80. J. W. Woods, "Two-dimensional Markov spectral estimation," IEEE Trans. Inf. Theory, vol. IT-22, no. 5, pp. 552-559, Sep. 1976.
81. H. P. Yuen and M. Lax, "Multiple-parameter quantum estimation and measurement of nonselfadjoint observables," IEEE Trans. Inf. Theory, vol. IT-19, no. 6, pp. 740-750, Nov. 1973.