A recurrent log-linearized Gaussian mixture network

IEEE Transactions on Neural Networks

Volumn 14, Issue 2, 2003, Pages 304-316

  Tsuji, Toshio ^a   Bu, Nan ^a   Fukuda, Osamu ^b   Kaneko, Makoto ^a  

^a HIROSHIMA UNIVERSITY   (Japan)

^b NATIONAL INSTITUTE OF ADVANCED INDUSTRIAL SCIENCE AND TECHNOLOGY AIST   (Japan)

Author keywords

EEG; Gaussian mixture model; Hidden Markov model (HMM); Log linearized model; Neural networks (NNs); Pattern classification; Recurrent neural networks (RNNs)

Indexed keywords

COMPUTER SIMULATION; ELECTROENCEPHALOGRAPHY; LEARNING ALGORITHMS; LEARNING SYSTEMS; MARKOV PROCESSES; MATHEMATICAL MODELS; PROBABILITY; SPEECH RECOGNITION;

HIDDEN MARKOV MODEL; PROBABILISTIC NEURAL NETWORK; RECURRENT LOG-LINEARIZED GAUSSIAN MIXTURE NETWORK;

RECURRENT NEURAL NETWORKS;

EID: 0037361044     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2003.809403     Document Type: Article

References (39)

1. D. E. Rumelhart and J. L. McClelland, "Learning internal representations by error propagation," in Parallel Distributed Processing, Explorations in the Microstructure of Cognition, D. E. Rumelhart and J. L. McClelland, Eds. Cambridge, MA: MIT Press, 1986, vol. 1, pp. 318-362.
2. T. Caelli, D. M. Squire, and T. P. J. Wild, "Model-based neural networks," Neural Networks, vol. 6, no. 5, pp. 613-625, 1993.
3. M. D. Richard and R. P. Lippmann, "Neural network classifiers estimate Bayesian a posteriori probabilities," Neural Comput., vol. 3, pp. 461-483, 1991.
4. D. F. Specht, "Probabilistic neural networks," Neural Networks, vol. 3, no. 1, pp. 109-118, 1990.
5. J. S. Bridle, "Probabilistic interpretation of feedforward classification network outputs, with relationships to statistical pattern recognition," in Neurocomputing: Algorithms, Architectures, and Applications, S. F. Fogelman and J. Hault, Eds. New York: Springer-Verlag, 1989, pp. 227-236.
6. H. G. C. Tråvén, "A neural network approach to statistical pattern classification by 'semiparametric' estimation of probability density functions," IEEE Trans. Neural Networks, vol. 2, pp. 366-377, May 1991.
7. L. I. Perlovsky and M. M. McManus, "Maximum likelihood neural networks for sensor fusion and adaptive classification," Neural Networks, vol. 4, no. 1, pp. 89-102, 1991.
8. T. Tsuji, D. Mori, and T. Ito, "Motion discrimination method from EMG signal using statistically structured neural networks" (in Japanese), Trans. Inst. Elect. Eng. Japan, vol. 112-C, no. 8, pp. 465-473, 1992.
9. T. Tsuji, O. Fukuda, H. Ichinobe, and M. Kaneko, "A log-linearized Gaussian mixture network and its application to EEG pattern classification," IEEE Trans. Syst., Man, Cybern, C, vol. 29, pp. 60-72, Feb. 1999.
10. S. Lee and S. Shimoji, "Self-organization of Gaussian mixture model for learning class pdfs in pattern classification," in Proc. IEEE Int. Joint Conf. Neural Networks, vol. III, 1993, pp. 2492-2495.
11. R. L. Streit and T. E. Luginbuhl, "Maximum likelihood training of probabilistic neural networks," IEEE Trans. Neural Networks, vol. 5, pp. 764-783, Sept. 1994.
12. C. Bishop, Neural Network for Pattern Recognition. Oxford, U.K.: Clarendon, 1995.
13. O. Fukuda, T. Tsuji, A. Ohtsuka, and M. Kaneko, "EMG-based human-robot interface for rehabilitation aid," in Proc. IEEE Int. Conf. Robotics Automation, Leuven, Belgium, 1998, pp. 3492-3497.
14. R. J. Williams and D. Zipser, "A learning algorithm for continually running fully recurrent neural networks," Neural Comput., vol. 1, pp. 270-280, 1989.
15. H. Bersini, M. Searens, and L. G. Sotelino, "Hopfield net generation, encoding and classification of temporal trajectories," IEEE Trans. Neural Networks, vol. 5, pp. 945-953, Nov. 1994.
16. J. J. Hopfield, "Neural Networks and physical systems with emergent collective computational abilities," Proc. Nat. Academy Sci. USA, vol. 79, pp. 2554-2558, 1982.
17. N. G. Bourbakis, C. Koutsougeras, and A. Jameel, "Handwritten character recognition using low resolutions," Eng. Applicat. Artif. Intell., vol. 12, pp. 139-147, 1999.
18. T. Lin, B. G. Horne, and C. L. Giles, "How embedded memory in recurrent neural network architectures helps learning long-term temporal dependencies," Neural Networks, vol. 11, no. 5, pp. 861-868, 1998.
19. A. Petrosian, D. Prokhorov, R. Homan, R. Dasheiff, and D. Wunsch, II, "Recurrent neural network based prediction of epileptic seizures in intra- and extracranial EEG," Neurocomputing, vol. 30, pp. 201-218, 2000.
20. A. Aussem, "Dynamical recurrent neural networks toward prediction and modeling of dynamical systems," Neurocomputing, vol. 28, pp. 207-232, 1999.
21. J. Zhang, A. J. Morris, and E. B. Martin, "Long-term prediction models based on mixed order locally recurrent neural networks," Comput. Chem. Eng., vol. 22, no. 7-8, pp. 1051-1063, 1998.
22. C. Schittenkopf, G. Dorffner, and E. Dockner, "Fat tails and nonlinearity in volatility models: What is more important?,", Tech. Rep., OEFAI-99-05, 1999.
23. P. J. Werbos, "Backpropagation through time: What it does and how to do it," Proc. IEEE, vol. 78, pp. 1550-1560, Oct. 1990.
24. M. Zak, "Terminal attractors for addressable memory in neural networks," Phys. Lett. A, vol. 133, no. 1,2, pp. 18-22, 1988.
25. T. Tsuji, O. Fukuda, M. Kaneko, and K. Ito, "Pattern classification of time-series EMG signals using neural networks," Int. J. Adaptive Contr. Signal Processing, vol. 14, no. 8, pp. 829-848, 2000.
26. L. E. Baum and T. Petrie, "Statistical inference for probabilistic function of finite state Markov chains," Ann. Math. Stat., vol. 37, no. 6, pp. 1554-1563, 1966.
27. L. R. Rabiner, "A tutorial on hidden Markov model and selected applications in speech recognition," Proc. IEEE, vol. 77, pp. 257-286, Feb. 1989.
28. A. P. Dempster, N. M. Larid, and D. B. Rubin, "Maximum likelihood from incomplete data via the EM algorithm," J. Roy. Stat. Soc. Ser. B, vol. 39, pp. 1-38, 1977.
29. L. E. Baum, T. Petrie, G. Soules, and N. Weiss, "A minimization technique occurring in the statistical analysis of probabilistic functions of Markov chains," Ann. Math. Statist., vol. 41, pp. 164-171, 1970.
30. B. H. Juang, S. E. Levinson, and M. M. Sondhi, "Maximum likelihood estimation for multivariate mixture observations of Markov chains," IEEE Trans. Inform. Theory, vol. IT-32, pp. 307-309, Mar. 1986.
31. L. R. Rabiner, J. G. Wilpon, and B. H. Juang, "A segmental k-means training procedure for connected word recognition," AT&T Tech. J., vol. 65, no. 3, pp. 21-32, 1986.
32. H. Bourlard and C. J. Wellekens, "Links between Markov models and multilayer perceptions," in Advances in Neural Information Processing Systems I, D. S. Touretzky, Ed. San Mateo, CA: Morgan Kaufmann, 1989, pp. 502-510.
33. M. Cohen, H. Franco, N. Morgan, D. Rumelhart, and V. Abrash, "Hybrid neural network/hidden Markov model continuous speech recognition," in Proc. Int. Conf. Spoken Language Processing 1992, Banff, AB, Canada, 1992, pp. 915-918.
34. A. J. Robinson, "An application of recurrent nets to phone probability estimation," IEEE Trans. Neural Networks, vol. 5, pp. 298-305, Mar. 1994.
35. C. D., Mitchell, M. P. Harper, and L. H. Jamieson, "Stochastic observation hidden Markov models," in Proc. IEEE ICASSP'96, 1996, pp. 617-620.
36. N. Ström, "Phoneme probability estimation with dynamic sparsely connected artificial neural networks," Free Speech J., vol. 1, no. 5, 1997.
37. J. S. Bridle, "Alpha-nets: A recurrent 'Neural' network architecture with a hidden Markov model interpretation," Speech Commun., vol. 9, no. 1, pp. 83-92, 1990.
38. M. I. Jordan, "Serial order: A parallel distributed processing approach," Institute for Cognitive Science, University of California, San Diego, Tech. Rep. 8604, 1986.
39. J. L. Elman, "Finding structure in time," Cognitive Sci., vol. 14, pp. 179-211, 1990.