Nonlinear dynamical factor analysis for state change detection

IEEE Transactions on Neural Networks

Volumn 15, Issue 3, 2004, Pages 559-575

  Ilin, Alexander ^a   Valpola, Harri ^a   Oja, Erkki ^a  

^a AALTO UNIVERSITY   (Finland)

Author keywords

Change detection; Independent component analysis; Nonlinear state space model; Variational Bayesian learning

Indexed keywords

ALGORITHMS; COMPUTER SIMULATION; DIFFERENTIAL EQUATIONS; INDEPENDENT COMPONENT ANALYSIS; NEURAL NETWORKS; STATE SPACE METHODS;

NONLINEAR DYNAMICAL FACTOR ANALYSIS; NONLINEAR STATES-SPACE MODEL; STATE CHANGE DETECTION; VARIATIONAL BAYESIAN LEARNING;

LEARNING SYSTEMS;

ARTICLE; FACTORIAL ANALYSIS; NONLINEAR SYSTEM;

FACTOR ANALYSIS, STATISTICAL; NONLINEAR DYNAMICS;

EID: 2542624430     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2004.826129     Document Type: Article

References (45)

1. S. Amari and H. Nagaoka, Methods of Information Geometry: American Mathematical Society, Providence, 2000.
2. H. Attias, "ICA, graphical models and variational methods," in Independent Component Analysis: Principles and Practice, S. Roberts and R. Everson, Eds: Cambridge Univ. Press, 2001, pp. 95-112.
3. B. Azimi-Sadjadi and P. S. Krishnaprasad, "Change detection for nonlinear systems; a particle filtering approach," in Proc. Amer. Contr. Conf. (ACC2002), 2002.
4. M. Basseville and I. Nikiforov, Detection of Abrupt Changes: Theory and Application, ser. Information and system science. Englewood Cliffs, NJ: Prentice-Hall, 1993.
5. T. Briegel and V. Tresp, "Fisher scoring and a mixture of modes approach for approximate inference and learning in nonlinear state space models," in Advances in Neural Information Processing Systems 11, M. Kearns, S. Solla, and D. Cohn, Eds. Cambridge, MA: MIT Press, pp. 403-409.
6. J. Chen and R. J. Patton, Robust Model-Based Fault Diagnosis for Dynamic Systems, MA: Kluwer Academic, 1999.
7. L. H. Chiang, E. L. Russell, and R. D. Braatz, Fault Detection and Diagnosis in Industrial Systems, London: Springer-Verlag, 2001.
8. R. A. Choudrey and S. J. Roberts, "Flexible Bayesian independent component analysis for blind source separation," in Proc. Int. Conf. Independent Component Analysis and Signal Separation (ICA2001), San Diego, CA, 2001, pp. 90-95.
9. T. M. Cover and J. A. Thomas, Elements of Information Theory. New York: Wiley, 1991.
10. G. Dorffner, "Neural networks for time series processing," Neural Network World, vol. 6, no. 4, pp. 447-468, 1996.
11. F. D. Foresee and M. T. Hagan, "Gauss-Newton approximation to Bayesian regularization," in Proc. 1997 Int. Joint Conf. Neural Networks, 1997, pp. 1930-1935.
12. E. A. García and P. M. Frank, "Deterministic nonlinear observer-based approaches to fault diagnosis: A survey," Contr. Eng. Practice, vol. 5, no. 5, pp. 663-670, 1997.
13. Z. Ghahramani and S. Roweis, "Learning nonlinear dynamical systems using an EM algorithm," in Advances in Neural Information Processing Systems 11, M. Kearns, S. Solla, and D. Cohn, Eds. Cambridge, MA: The MIT Press, 1999, pp. 431-437.
14. M. S. Grewal and A. P. Andrews, Kalman Filtering: Theory and Practice, ser. Information and system science. Englewood Cliffs, NJ: Prentice-Hall, 1993.
15. F. Gustafsson, Adaptive Filtering and Change Detection: Wiley, 2000.
16. S. Haykin, Neural Networks - A Comprehensive Foundation, 2nd ed. Englewood Cliffs, NJ: Prentice-Hall, 1999.
17. ____, Adaptive Filter Theory, 4th ed. Englewood Cliffs, NJ: Prentice-Hall, 2001.
18. S. Haykin and J. Principe, "Making sense of a complex world," IEEE Signal Processing Mag., vol. 15, pp. 66-81, May 1998.
19. G. E. Hinton and D. van Camp, "Keeping neural networks simple by minimizing the description length of the weights," in Proc. 6th Ann. ACM Conf. Computational Learning Theory, Santa Cruz, CA, 1993, pp. 5-13.
20. P. Højen-Sørensen, L. K. Hansen, and O. Winther, "Mean field implementation of Bayesian ICA," in Proc. Int. Conf. Independent Component Analysis and Signal Separation (ICA2001), San Diego, CA, 2001, pp. 439-444.
21. A. Honkela, H. Valpola, and J. Karhunen, "Accelerating cyclic update algorithms for parameter estimation by pattern searches," Neural Processing Lett., vol. 17, no. 2, pp. 191-203, 2003.
22. A. Hyvärinen, J. Karhunen, and E. Oja, Independent Component Analysis. New York: Wiley, 2001.
23. A. Ilin and H. Valpola, "On the effect of the form of the posterior approximation in variational learning of ICA models," in Proc. 4th Int. Symp. Independent Component Analysis and Blind Signal Separation (ICA2003), Nara, Japan, 2003, pp. 915-920.
24. A. Iline, H. Valpola, and E. Oja, "Detecting process state changes by nonlinear blind source separation," in Proc. 3rd Int. Workshop on Independent Component Analysis and Signal Separation (ICA2001), San Diego, CA, Dec. 2001, pp. 704-709.
25. H. Lappalainen and A. Honkela, "Bayesian nonlinear independent component analysis by multi-layer perceptrons," in Advances in Independent Component Analysis, M. Girolami, Ed, Berlin: Springer-Verlag, 2000, pp. 93-121.
26. H. Lappalainen and J. Miskin, "Ensemble learning," in Advances in Independent Component Analysis, M. Girolami, Ed, Berlin: Springer-Verlag, 2000, pp. 75-92.
27. D. J. C. MacKay, "Bayesian interpolation," Neural Comput., vol. 4, no. 3, pp. 415-447, 1992.
28. ____, "A practical Bayesian framework for backpropagation networks," Neural Comput., vol. 4, no. 3, pp. 448-472, 1992.
29. J. Miskin and D. J. C. MacKay, "Ensemble learning for blind source separation," in Independent Component Analysis: Principles and Practice, S. Roberts and R. Everson, Eds: Cambridge Univ. Press, 2001, pp. 209-233.
30. R. J. Patton, J. Chen, and T. M. Siew, "Fault diagnosis in nonlinear dynamic systems via neural networks," in Proc. IEE Int. Conf. Contr. '94, Conf. Pub. 389, Warwick, U.K., 1994, pp. 1346-1351.
31. S. Roweis and Z. Ghahramani, "An EM algorithm for identification of nonlinear dynamical systems," in Kalman Filtering and Neural Networks, S. Haykin, Ed. New York: Wiley, 2001, pp. 175-220.
32. J. Särelä, H. Valpola, R. Vigário, and E. Oja, "Dynamical factor analysis of rhythmic magnetoencephalographic activity," in Proc. Int. Conf. Independent Component Analysis and Signal Separation (ICA2001), San Diego, CA, 2001, pp. 451-456.
33. J. Segen and A. C. Sanderson, "Detecting change in a time-series," IEEE Trans. Inform. Theory, vol. IT-26, pp. 249-255, 1980.
34. H. Valpola, Unsupervised Learning of Nonlinear Dynamic State-Space Models, Espoo, Finland: Publications in Computer and Information Science A59, Lab. Computer and Information Science, Helsinki Univ. Technol., 2000.
35. H. Valpola, A. Honkela, and X. Giannakopoulos, (2002) Matlab Codes for the NFA and NDFA Algorithms [Online]. Available: http://www.cis.hut.fi/projects/bayes/
36. H. Valpola, A. Honkela, and J. Karhunen, "Nonlinear static and dynamic blind source separation using ensemble learning," in Proc. Int. Joint Conf. Neural Networks (IJCNN'01), Washington, D.C., 2001, pp. 2750-2755.
37. H. Valpola and J. Karhunen, "An unsupervised ensemble learning method for nonlinear dynamic state-space models," Neural Comput., vol. 14, no. 11, pp. 2647-2692, 2002.
38. H. Valpola, E. Oja and A. Ilin, A. Honkela, and J. Karhunen, "Nonlinear blind source separation by variational Bayesian learning," IEICE Trans. Fund. Electron., Commun. Comput. Sci., vol. E86-A, no. 3, Mar. 2003.
39. H. Valpola, T. Raiko, and J. Karhunen. Building blocks for hierarchical latent variable models. presented at Proc. 3rd Int. Workshop on Independent Component Analysis and Signal Separation (ICA2001). [Online]. Available: http://www.cis.hut.fi/harri/
40. H. Valpola and M. Harva, and J. Karhunen, "Hierarchical models of variance sources," Signal Process., vol. 84, no. 2, pp. 267-282, 2004.
41. H. Valpola and T. Östman, and J. Karhunen, "Nonlinear independent factor analysis by hierarchical models," in Proc. 4th Int. Symp. Independent Component Analysis and Blind Signal Separation (ICA2003), Nara, Japan, 2003, pp. 257-262.
42. P. van Overschee and B. de Moor, Subspace Identification for Linear Systems. Boston, MA: Kluwer Academic, 1996.
43. E. Wan and A. Nelson, "Dual extended Kalman filter methods," in Kalman Filtering and Neural Networks, S. Haykin, Ed. New York: Wiley, 2001, pp. 123-173.
44. E. A. Wan and R. van der Merwe, "The unscented Kalman filter," in Kalman Filtering and Neural Networks, S. Haykin, Ed. New York: Wiley, 2001, pp. 221-280.
45. J. Zhang, E. B. Martin, and A. J. Morris, "Process monitoring using nonlinear statistical techniques," Chem. Eng. J., vol. 67, no. 3, pp. 181-189, 1997.