Predictive modular neural networks for unsupervised segmentation of switching time series: The data allocation problem

IEEE Transactions on Neural Networks

Volumn 13, Issue 6, 2002, Pages 1432-1449

  Kehagias, Athanasios ^a   Petridis, Vassilios ^a  

^a ARISTOTLE UNIVERSITY OF THESSALONIKI   (Greece)

Author keywords

Adaptive systems; Competitive learning; Data allocation (DA); Modularity; Switching; Time series; Unsupervised learning

Indexed keywords

ADAPTIVE CONTROL SYSTEMS; DATA ACQUISITION; ERRORS; ITERATED SWITCHING NETWORKS; TIME SERIES ANALYSIS;

DATA ALLOCATION PROBLEM;

NEURAL NETWORKS;

EID: 0036857915     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2002.804288     Document Type: Article

References (60)

1. G.A. Ackerson and K.S. Fu, "On state estimation in switching environments," IEEE Trans. Automat. Contr., vol. AC-15, pp. 10-17, Jan. 1970.
2. P. Baldi and Y. Chauvin, "Smooth on-line learning algorithms for hidden Markov models," Neural Comput., vol. 6, pp. 305-316, 1994.
3. P. Baldi, S. Brunak, P. Frasconi, G. Pollastri, and G. Soda, "Bidirectional dynamics for protein secondary structure prediction," in Sequence Learning: Paradigms, Algorithms, and Applications, R. Sun and C.L. Giles, Eds. New York: Springer-Verlag, 2001, pp. 80-104.
4. Y. Bar-Shalom and X.R. Li, Estimation and Tracking. Norwood, MA: Artech House, 1993.
5. Y. Bengio and P. Frasconi, "Input/output HMMs for sequence processing," IEEE Trans. Neural Networks, vol. 7, pp. 1231-1249, Sept. 1996.
6. Y. Bengio, "Markovian models for sequential data," Neural Comp. Surveys, vol. 2, pp. 129-162, 1998.
7. P. Billingsley, Probability and Measure. New York: Wiley, 1985.
8. L. Bottou and Y. Bengio et al., "Convergence properties of the k-means algorithms," in Advantages in Neural Information Processing Systems 7, G. Tesauro et al., Eds. Cambridge, MA: MIT Press, 1995, pp. 585-592.
9. X. Boyen and D. Koller, "Approximate learning of dynamic models," in Proc. 12th Annu. Conf. Neural Inform. Processing Syst., Denver, CO, 1998, pp. 396-402.
10. T.W. Cacciatore and S.J. Nowlan, "Mixtures of controllers for jump linear and nonlinear plants," in Advances in Neural Information Processing Systems 6, J.D. Cowen, G. Tesauro, and J. Alspector, Eds. San Mateo, CA: Morgan Kaufmann, 1994, pp. 719-726.
11. M.J. Caputi, "A necessary condition for effective performance of the multiple model adaptive estimator," IEEE Trans. Aerosp. Electron. Syst., vol. 31, pp. 1132-1138, Sept. 1995.
12. C.K. Carter and R. Kohn, "On Gibbs sampling for state space models," Biometrika, vol. 81, pp. 541-553, 1994.
13. M.P. Clements and H.M. Krolzig, "A comparison of the forecast performance of Markov-switching and threshold autoregressive models of U.S. GNP," Econom. J., vol. 1, pp. 47-75, 1998.
14. A.P. Dempster, N.M. Laird, and D.B. Rubin, "Maximum likelihood from incomplete data via the EM algorithm," J. Royal Statist. Soc. B., vol. 39, pp. 1-38, 1977.
15. L. Deng, "A generalized hidden Markov model with state-conditioned trend functions of time for the speech signal," Signal Processing, vol. 27, pp. 65-78, 1992.
16. L. Deng, P. Kenny, M. Lennig, and P. Mermelstein, "Modeling acoustic transitions in speech by state interpolation HMM," IEEE Trans. Signal Processing, vol. 40, pp. 265-271, Feb. 1992.
17. R.O. Duda, P.E. Hart, and D.G. Stork, Pattern Classifications, 2 ed. New York: Wiley, 2001.
18. F. Dufour and P. Bertrand, "The filtering problem for continuous-time linear systems with Markovian switching coefficients," Syst. Contr. Lett., vol. 23, pp. 453-461, 1994.
19. _, "Stabilizing control for hybrid models," IEEE Trans. Automat. Contr., vol. 39, pp. 2354-2357, Dec. 1993.
20. R.J. Elliot, L. Aggoun, and J.B. Moore, Hidden Markov Models. New York: Springer-Verlag, 1995.
21. P.H. Franses, Time Series Models for Business and Economic Forecasting. Cambridge, U.K.: Cambridge Univ. Press, 1998.
22. Z. Ghahramani and G.E. Hinton, "Variational learning for switching state-space models," Neural Comput., vol. 12, pp. 963-996, 1998.
23. Z. Ghahramani, "Learning dynamic Bayesian networks," in Adaptive Processing of Sequences and Data Structures, C.L. Giles and M. Gori, Eds. New York: Springer-Verlag, 1998, Lecture Notes in Artificial Intelligence, pp. 168-197.
24. Z. Ghahramani and S. Roweis, "Learning nonlinear dynamical systems using an EM algorithm," in Advances in Neural Information Processing Systems 11, M.S. Kearns, S.A. Solla, and D.A. Cohn, Eds. Cambridge, MA: MIT Press, 1999, pp. 599-605.
25. J.D. Hamilton, "Analysis of times series subject to changes in regime," J. Econom., vol. 45, pp. 39-70, 1990.
26. _, Time Series Analysis. Princeton, NJ: Princeton Univ. Press, 1994.
27. R.A. Jacobs, M.I. Jordan, S.J. Nowlan, and G.E. Hinton, "Adaptive mixtures of local experts," Neural Comput., vol. 3, pp. 79-87, 1991.
28. T.A. Johansen and B.A. Foss, "Identification of nonlinear system structure and parameters using regime decomposition," Automatica, vol. 31, pp. 321-326, 1995.
29. M.I. Jordan and R.A. Jacobs, "Hierarchical mixtures of experts and the EM algorithm," Neural Comput., vol. 6, pp. 181-214, 1994.
30. B.H. Juang, "Maximum-likelihood estimation for mixture multivariate stochastic observations of Markov chains," AT & T Tech. J., vol. 64, 1985.
31. A. Kehagias and V. Petridis, "Predictive modular neural networks for time series classification," Neural Networks, vol. 10, pp. 31-49, 1996.
32. _, "Time series segmentation using predictive modular neural networks," Neural Comput., vol. 9, pp. 1691-1710, 1997.
33. P. Kenny, M. Lennig, and P. Mermelstein, "A linear predictive HMM for vector-valued observations with applications to speech recognition," IEEE Trans. Acoust., Speech, Signal Processing, vol. 8, pp. 220-225, Mar. 1990.
34. J. Kohlmorgen, K.R. Mueller, J. Rittweger, and K. Pawelzik, "Identification of nonstationary dynamics in physiological recordings," Biol. Cybern., vol. 83, pp. 73-84, 2000.
35. A. Krogh et al., "Hidden Markov models in computational biology: Applications to protein modeling," J. Mol. Biol., vol. 235, pp. 1501-1531, 1994.
36. H.M. Krolzig, Markov-Switching Vector Autoregressions. New York: Springer-Verlag, 1996.
37. _, (1998) Econometric modeling of Markov switching vector autoregressions using MSVAR for OX [Online]http://www.econ.ox.ac.uk/re-search.hendry.krolzig/ox/MSVAR130a.ZIP.
38. C.G. Hilborn and D.G. Lainiotis, "Optimal estimation in the presence of unknown parameters," IEEE Trans. Syst. Sci. Cybern., vol. SMC-5, pp. 38-43, Feb. 1969.
39. D.G. Lainiotis, "Optimal adaptive estimation: Structure and parameter adaptation," IEEE Trans. Automat Contr., vol. AC-16, pp. 160-170, Jan. 1971.
40. M.C. Medeiros, M.G.C. Resende, and A. Veiga, "Piecewise Linear Time Series Estimation With GRASP,", AT & T Rep. 99.3.1, 1999.
41. K. Narendra, "Adaptation and learning using multiple models, switching and tuning," IEEE Contr. Syst. Mag., pp. 37-51, Jan. 1995.
42. K. Narendra and J. Balakhrishnan, "Adaptive control using multiple models," IEEE Trans. Automat. Contr., vol. 42, pp. 171-187, Jan. 1997.
43. R.M. Neal and G. Hinton, "A new view of the EM algorithm that justifies incremental, sparse and other variants," in Learning in Graphical Models, M.I. Jordan, Ed. Boston, MA: Kluwer, 1998, pp. 355-368.
44. M. Ostendorf, V. Digalakis, and O. Kimball, "From HMMs to segment models: A unified view of stochastic modeling for speech recognition," IEEE Trans. Speech Audio Processing, pp. 360-378, Mar. 1996.
45. K. Pawelzik, J. Kohlmorgen, and K.R. Muller, "Annealed competition of experts for a segmentation and classification of switching dynamics," Neural Comput., vol. 8, pp. 357-372, 1996.
46. V. Petridis and A. Kehagias, "A recurrent network implementation of time series classification," Neural Comput., vol. 8, pp. 357-372, 1996.
47. _, "Modular neural networks for MAP classification of time series and the partition algorithm," IEEE Trans. Neural Networks, vol. 7, pp. 73-86, Jan. 1996.
48. _, Predictive Modular Neural Networks: Applications to Time Series. Boston, MA: Kluwer, 1998.
49. L.R. Rabiner, "A tutorial on hidden Markov models and selected applications in speech recognition," Proc. IEEE, vol. 77, pp. 257-285, Feb. 1989.
50. M. Sato, "OnLine model selection based on the variational Bayes," Neural Comput., vol. 13, pp. 1649-1681, 2001.
51. B. Stenger et al., "Topology free hidden Markov models: Application to background modeling," in Proc. 8th IEEE Int. Conf. Comput. Vision, vol. I, 2001, pp. 294-301.
52. H. Tong, Nonlinear Time Series: A Dynamical System Approach. Oxford, U.K.: Clarendon, 1990.
53. R.S. Tsay, "Testing and modeling multivariate threshold models," J. Amer. Statist. Assoc., vol. 93, pp. 1188-1202, 1998.
54. N. Ueda, R. Nakano, Z. Ghahramani, and G.E. Hinton, "SMEM algorithm for mixture models," Neural Comput., vol. 12, pp. 2109-2128, 2000.
55. D. van Dijk, T. Terasvirta, and P.H. Franses, "Smooth transition autoregressive models-A survey of recent developments,", Econometric Rev., vol. 21, pp. 1-47, 2002.
56. A. Weigend and N. Gershenfeld, Eds., Time Series Prediction: Forecasting the Future and Understanding the Past. Reading, MA: Addison-Wesley, 1992.
57. A. Weigend and S. Shi, "Predicting daily probability distributions of S & P500 returns." J. Forecasting, vol. 19, pp. 375-392, 1998.
58. L. Xu, "RBF nets, mixture experts, and Bayesian Ying-Yang learning," Neurocomput., vol. 19, pp. 223-257, 1998.
59. _, "Temporal BYY learning for state space approach, hidden Markov model, and blind source separation," IEEE Trans. Signal Processing, vol. 48, pp. 2132-2144, Nov. 2000.
60. W. Yoshida, S. Ishii, and M. Sato, "Reconstruction of chaotic dynamics and robustness to noise with on-line EM algorithm," presented at the IEEE SMC Conference, 1999.