Parameter identifiability with kullback-leibler information divergence criterion

International Journal of Adaptive Control and Signal Processing

Volumn 23, Issue 10, 2009, Pages 940-960

  Chen, Badong ^a   Hu, Jinchun ^a   Zhu, Yu ^a   Sun, Zengqi ^a  

^a TSINGHUA UNIVERSITY   (China)

Author keywords

Consistency in probability; Fisher's information matrix (FIM); Kullback leibler information divergence (KLID); Parameter estimation; System identification

Indexed keywords

CONSISTENCY IN PROBABILITY; CONSISTENCY PROBLEMS; FISHER'S INFORMATION; FISHER'S INFORMATION MATRIX (FIM); GAUSSIAN PROCESSES; IDENTIFIABILITY; KULLBACK-LEIBLER INFORMATION; KULLBACK-LEIBLER INFORMATION DIVERGENCE (KLID); LEAST SQUARE; PARAMETER ESTIMATION METHOD; PARAMETER IDENTIFIABILITY; SYSTEM IDENTIFICATION; UPPER BOUND;

ASYMPTOTIC ANALYSIS; MARKOV PROCESSES; METAL ANALYSIS; PARAMETER ESTIMATION; SPECTRAL DENSITY;

FISHER INFORMATION MATRIX;

EID: 70349423630     PISSN: 08906327     EISSN: 10991115     Source Type: Journal    
DOI: 10.1002/acs.1078     Document Type: Article

References (28)

1. Cover TM, Thomas JA. Element of Information Theory. Wiley: Chichester, 1991.
2. Liou CY, Musicus BR. A separable cross-entropy approach to power spectral estimation. IEEE Transactions on Acoustics, Speech, and Signal Processing 1990; 38(1):105-113.
3. Ramirez J, Segura JC, Benitez C, de la Torre A, Rubio AJ. A new Kullback-Leibler VAD for speech recognition in noise. IEEE Signal Processing Letters 2004; 11(2):266-269.
4. Karny M, Guy TV. Fully probabilistic control design. Systems and Control Letters 2006; 55:259-265.
5. Akaike H. A new look at the statistical model identification. IEEE Transactions on Automatic Control 1974; 19:716-723.
6. Rissanen J. Modeling by shortest data description. Automatica 1978; 14:465-471.
7. Baram Y, Sandell NR., An information theoretic approach to dynamical systems modelling and identification. IEEE Transactions on Automatic Control 1978; 23(1):61-66.
8. Baram Y, Beeri Y. Stochastic model simplification. IEEE Transactions on Automatic Control 1981; 26(2):379-390.
9. Weinstein E, Feder M, Oppenheim AV. Sequential algorithms for parameter estimation based on the Kullback-Leibler information measure. IEEE Transactions on Acoustics, Speech, and Signal Processing 1990; 38(9): 1652-1654.
10. Krishnamurthy V. On-line estimation of dynamic shock-error models based on the Kullback-Leibler information measure. IEEE Transactions on Automatic Control 1994; 39(5):1129-1135.
11. Stoorvogel AA, Van Schuppen JH. System identification with information theoretic criteria. In Identification, Adaptation, Learning, Bittanti S, Picc G (eds). Springer: Berlin, 1996; 289-338.
12. Stoorvogel AA, Van Schuppen JH. Approximation problems with the divergence criterion for Gaussian variables and Gaussian process. Systems and Control Letters 1998; 35:207-218.
13. Pardo MC. Estimation of parameters for a mixture of normal distributions on the basis of the Cressie and Read divergence. Communications in Statistics (Simulation and Computation) 1999; 28(1):115-130.
14. Pardo L. Statistical Inference Based on Divergence Measures. Chapman & Hall/CRC: London, 2006.
15. Ljung L. System Identification: Theory for the User (2nd edn). Prentice-Hall PTR: Englewood Cliffs, NJ, 1999.
16. Nguyen VV, Wood EF. Review and unification of linear identifiability concepts. SIAM Reviews 1982; 24(1):34-51.
17. Hall P. On Kullback-Leibler loss and density estimation. The Annals of Statistics 1987; 15(4):1491-1519.
18. Grewal MS, Glover K. Identifiability of linear and nonlinear dynamic systems. IEEE Transactions on Automatic Control 1976; 21(6):833-837.
19. Glover K, Willems JC. Parameterizations of linear dynamical systems: canonical forms and identifiability. IEEE Transactions on Automatic Control 1974; 19(6):640-646.
20. Tse E, Anton JJ. On the identifiability of parameters. IEEE Transactions on Automatic Control 1972; 17(5): 637-646.
21. van der Vaart AW. Asymptotic Statistics. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press: Cambridge, 1998.
22. Rothenberg TJ. Identification in parametric models. Econometrica 1971; 39:577-591.
23. Kazakos D, Kazakos PP. Spectral distance measure between Gaussian processes. IEEE Transactions on Automatic Control 1980; 25(5):950-959.
24. Whittle P. The analysis of multiple stationary time series. Journal ofthe Royal Statistical Society, Series B 1953; 15:125-139.
25. Doob JL. Stochastic Processes. Wiley: New York, 1953.
26. van de Geer SA. Empirical Processes in M-Estimation. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press: Cambridge, 2000.
27. Barron AR, Gyorfi L, van der Meulen EC. Distribution estimation consistent in total variation and in two types of information divergence. IEEE Transactions on Information Theory 1992; 38(5):1437-1454.
28. Devroye L, Lugosi G. Combinatorial Methods in Density Estimation. Springer: Berlin, 2000.