Anticipation-based temporal sequences learning in hierarchical structure

IEEE Transactions on Neural Networks

Volumn 18, Issue 2, 2007, Pages 344-358

  Starzyk, Janusz A ^a   He, Haibo ^b  

^a Ohio University   (United States)

^b Stevens Institute of Technology   (United States)

Author keywords

Hierarchical structure; Input anticipation; Temporal sequence learning; Winner take all (WTA)

Indexed keywords

FORECASTING; LEARNING SYSTEMS; MATHEMATICAL MODELS; PATTERN RECOGNITION;

HEBBIAN LEARNING MECHANISM; INPUT ANTICIPATION; TEMPORAL SEQUENCE LEARNING; WINNER-TAKE-ALL;

HIERARCHICAL SYSTEMS;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; AUTOMATED PATTERN RECOGNITION; INFORMATION RETRIEVAL; METHODOLOGY; NATURAL LANGUAGE PROCESSING; TIME;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; INFORMATION STORAGE AND RETRIEVAL; NATURAL LANGUAGE PROCESSING; NEURAL NETWORKS (COMPUTER); PATTERN RECOGNITION, AUTOMATED; TIME FACTORS;

EID: 34047102036     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2006.884681     Document Type: Article

References (34)

1. R. Sun and C. L. Giles, "Sequence learning: From recognition and prediction to sequential decision making," IEEE Intell. Syst., vol. 16, no. 4, pp. 67-70, Jul./Aug. 2001.
2. J. H. Wang, M. C. Tsai, and W. S. Su, "Learning temporal sequences using dual-weight neurons," J. Chinese Inst. Eng., vol. 24, pp. 329-344, 2001.
3. N. Y. Liang, G. B. Huang, P. Saratchandran, and N. Sundararajan, "A fast and accurate online sequential learning algorithm for feedforward networks," IEEE Trans. Neural Netw., vol. 17, no. 6, pp. 1411-1423, Nov. 2006.
4. A. G. Tijsseling, "Sequential information processing using time-delay connections in ontogenic CALM networks," IEEE Trans. Neural Netw., vol. 16, no. 1, pp. 145-159, Jan. 2005.
5. J. L. Elman, "Finding structure in time," Cognitive Sci., vol. 14, pp. 179-211, 1990.
6. M. I. Jordan, "Attractor dynamics and parallelism in a connectionist sequential machine," in Proc. 8th Annu. Conf. Cognitive Sci. Soc., 1986, pp. 531-546.
7. J. B. Pollack, "The induction of dynamical recognizers," Mach. Learn., vol. 7, pp. 227-252, 1991.
8. J. Tani, "Learning to generate articulated behavior through the bottom-up and the top-down interaction process," Neural Netw., vol. 16, pp. 11-23, 2003.
9. J. Tani and S. Nolfi, "Learning to perceive the world as articulated: An approach for hierarchical learning in sensory-motor systems," Neural Netw., vol. 12, pp. 1131-1141, 1999.
10. S. Hochreiter and J. Schmidhuber, "Long short-term memory," Neural Comput., vol. 9, pp. 1735-1780, 1997.
11. D. Wang and M. A. Arbib, "Complex temporal sequence learning based on short-term memory," Proc. IEEE, vol. 78, no. 9, pp. 1536-1543, Sep. 1990.
12. D. Wang and M. A. Arbib, "Timing and chunking in processing temporal order," IEEE Trans. Syst., Man, Cybern., vol. 23, no. 4, pp. 993-1009, July/Aug. 1993.
13. D. Wang and B. Yuwono, "Anticipation-based temporal pattern generation," IEEE Trans. Syst., Man, Cybern., vol. 25, no. 4, pp. 615-628, Apr. 1995.
14. D. Wang and B. Yuwono, "Incremental learning of complex temporal patterns," IEEE Trans. Neural Netw., vol. 7, no. 6, pp. 1465-1481, Nov. 1996.
15. L. Wang, "Learning and retrieving spatio-temporal sequences with any static associative neural network," IEEE Trans. Circuits Syst. II, Analog Digit. Signal Process., vol. 45, no. 6, pp. 729-739, Jun. 1998.
16. L. Wang, "Multi-associative neural networks and their applications to learning and retrieving complex spatio-temporal sequences," IEEE Trans. Syst., Man, Cybern B, Cybern., vol. 29, no. 1, pp. 73-82, Feb. 1999.
17. S. Chartier and M. Boukadoum, "A sequential dynamic heteroassociative memory for multistep pattern recognition and one-to-many association," IEEE Trans. Neural Netw., vol. 17, no. 1, pp. 59-68, Jan. 2006.
18. J. Bose, S. B. Furber, and J. L. Shapiro, "An associative memory for the on-line recognition and prediction of temporal sequences," in Proc. Int. Joint Conf. Neural Netw., Montreal, QC, Canada, Jul. 31-Aug. 4 2005, pp. 1223-1228.
19. C. G. N. Manning and I. H. Witten, "Identifying hierarchical structure in sequences: A linear-time algorithm," J. Artif. Intell. Res., vol. 7, pp. 67-82, 1997.
20. D. George and J. Hawkins, "Invariant pattern recognition using Bayesian inference on hierarchical sequences" [Online]. Available: http://www.stanford.edu/~dil/RNI/DilJeffTechReport.pdf
21. S. Grossberg, "Competitive learning: From interactive activation to adaptive resonance," Cognitive Sci., vol. 11, pp. 23-63, 1987.
22. M. McCloskey and N. J. Cohen, "Catastrophic interference in connectionist networks: The sequential learning problem," Phys. Learn. Motivation, vol. 24, pp. 109-165, 1989.
23. N. E. Sharkey and A. J. C. Sharkey, "Understanding catastrophic interference in neural nets," Dept. Comput. Sci., Univ. Sheffield, U.K., Tech. Rep. CS-94-4, 1994.
24. C. A. Kortge, "Episodic memory in connectionist networks," in Proc. 12th Annu. Conf. Cognitive Sci. Soc., 1990, pp. 764-771.
25. R. M. French, "Using semidistributed representations to overcome catastrophic forgetting in connectionist networks," in Proc. 13th Annu. Conf. Cognitive Sci. Soc., 1991, pp. 173-178.
26. S. A. Sloman and D. E. Rumelhart, "Reducing interference in distributed memories through episodic gating," in From Learning Theory to Connectionist Theory: Essays in Honor of William K. Estes, A. F. Healy, S. M. Kosslyn, and R. M. Shiffrin, Eds. Hillsdale, NJ: Lawrence Erlbaum, 1992, pp. 227-248.
27. G. H. Bower, S. Thompson-Schill, and E. Tulving, "Reducing retroactive interference: An interference analysis," J. Exp. Psych: Learning, Memory, Cogn., vol. 20, pp. 51-66, 1994.
28. K. Doya and S. Yoshizawa, "Memorizing hierarchical temporal Patterns in analog neuron networks," in Proc. Int. Joint. Conf. Neural Netw., San Diego, CA, 1990, vol. 3, pp. 299-304.
29. P. J. Jennings and S. W. Keele, "A computational model of attentional requirements in sequence learning," in Proc. 12th Ann. Conf. Cog. Sci. Soc., 1990, pp. 876-883.
30. D. Wang, J. Buhman, and C. von der Malsburg, "Pattern segmentation in associative memory," Neural Comput., vol. 2, pp. 94-106, 1990.
31. J. Lücke and C. von der Malsburg, "Rapid processing and unsupervised learning in a model of the cortical macrocolumn," Neural Comput., vol. 16, pp. 501-33, 2004.
32. D. Vogel, "A biologically plausible model of associative memory which uses disinhibition rather than long term potentiation," Brain Cogn., vol. 45, pp. 212-228, 2001.
33. D. Vogel, "A neural network model of memory and higher cognitive functions," Int. J. Psychophysiol., vol. 55, pp. 3-21, 2005.
34. D. Vogel, Personal Communication.