Incremental learning of complex temporal patterns

IEEE Transactions on Neural Networks

Volumn 7, Issue 6, 1996, Pages 1465-1481

  Wang, DeLiang ^a,b   Yuwono, Budi ^b  

^a IEEE

^b OHIO STATE UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 0001637805     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/72.548174     Document Type: Article

References (44)

1. Y. Baram, "Memorizing binary vector sequences by a sparsely encoded network," IEEE Trans. Neural Networks, vol. 5, pp. 974-981, 1994.
2. J. M. Barnes and B. J. Underwood, "Fate of first-list associations in transfer theory," J. Exp. Psych.., vol. 58, pp. 97-105, 1959.
3. Y. Bengio, P. Simard, and P. Frasconi, "Learning long-term dependencies with gradient descent is difficult," IEEE Trans. Neural Networks, vol. 5, pp. 157-166, 1994.
4. G. H. Bower, S. Thompson-Schill, and E. Tulving, "Reducing retroactive interference: An interference analysis," J. Exp. Psych.: Learning, Memory, Cognition, vol. 20, pp. 51-66, 1994.
5. G. Bradski, G. A. Carpenter, and S. Grossberg, "STORE working memory networks for storage and recall of arbitrary temporal sequences," Biol. Cybern., vol. 71, pp. 469-480, 1994.
6. J. Buhmann and K. Schulten, "Noise-driven temporal association in neural networks," Europhys. Lett., vol. 4, pp. 1205-1209, 1987.
7. G. A. Carpenter and S. Grossberg, "A massively parallel architecture for a self-organizing neural pattern recognition machine," Comput. Vision, Graphs, Image Processing, vol. 37, pp. 54-115, 1987.
8. C. C. Chandler, "Accessing related events increases retroactive interference in a matching test," J. Exp. Psych.: Learning, Memory, Cognition, vol. 19, pp. 967-974, 1993.
9. R. L. Crooks and J. Stein, Psychology: Science, Behavior, and Life. Fort Worth, TX: Holt, Rinehart, and Winston, 1991.
10. S. Diederich and M. Opper, "Learning of correlated patterns in spin-like glass networks by local learning rules," Phys. Rev. Lett., vol. 58, pp. 949-952, 1987.
11. J. L. Elman, "Finding structure in time," Cognitive Sci., vol. 14, pp. 179-211, 1990.
12. R. Elmasri and S. B. Navathe, Fundamentals of Database Systems. Redwood City, CA: Benjamin/Cummings, 1994.
13. R. M. French, "Using semidistributed representations to overcome catastrophic forgetting in connectionist networks," in Proc. 13th Annu. Conf. Cognitive Sci. Soc., 1991, pp. 173-178.
14. _, "Dynamically contraining connectionist networks to produce distributed, orthogonal representations to reduce catastrophic interference," in Proc. 16th Annu. Conf. Cognitive Sci. Soc., 1994, pp. 335-340.
15. R. Granger, J. Whitson, J. Larson, and G. Lynch, "Non-Hebbian properties of long-term potentiation enable high-capacity encoding of temporal sequences," in Proc. Nat. Academy Sci. USA, vol. 91, 1994, pp. 10104-10108.
16. S. Grossberg, "Some networks that can learn, remember, and reproduce any number of complicated space-time patterns, 1," J. Math. Mechan., vol. 19, pp. 53-91, 1969.
17. _, "Adaptive pattern classification and universal receding I: Parallel development and coding of neural feature detectors," Biol. Cybern., vol. 23, pp. 121-134, 1976.
18. _, "Competitive learning: From interactive activation to adaptive resonance," Cognitive Sci., vol. 11, pp. 23-63, 1987.
19. T. Guyon, L. Personnaz, J. P. Nadal, and G. Dreyfus, "Storage and retrieval of complex sequences in neural networks," Phys. Rev. A, vol. 38, pp. 6365-6372, 1988.
20. H. Hertz, A. Krogh, and R. G. Palmer, Introduction to the Theory of Neural Computation. Redwood City, CA: Addison-Wesley, 1991.
21. T. M. Heskes and S. Gielen, "Retrieval of pattern sequences at variable speeds in a neural network with delays," Neural Networks, vol. 5, pp. 145-152, 1992.
22. M. I. Jordan, "Attractor dynamics and parallelism in a connectionist sequential machine," in Proc. 8th Annu. Conf. Cognitive Sci. Soc., 1986, pp. 531-546.
23. P. Kanerva, Sparse Distributed Memory. Cambridge, MA: MIT Press, 1988.
24. I. Kantor and H. Sompolinsky, "Associative recall of memory without errors," Phys. Rev. A, vol. 35, pp. 380-392, 1987.
25. C. A. Kortge, "Episodic memory in connectionist networks," in Proc. 12th Annu. Conf. Cognitive Sci. Soc., 1990, pp. 764-771.
26. J. K. Kruschke, "ALCOVE: An exemplar-based model of category learning," Psych. Rev., vol. 99, pp. 22-44, 1992.
27. K. S. Lashley, "The problem of serial order in behavior," in Cerebral Mechanisms in Behavior, L. A. Jeffress, Ed. New York: Wiley, 1951, pp. 112-146.
28. R. P. Lippmann, "Review of neural networks for speech recognition," Neural Computa., vol. 1, pp. 1-38, 1989.
29. M. McCloskey and N. J. Cohen, "Catastrophic interference in connectionist networks: The sequential learning problem," Psych. Learning Motivat., vol. 24, pp. 109-165, 1989.
30. G. A. Miller, "The magical number seven, plus or minus two: Some limits on our capacity for processing information," Psych. Rev., vol. 63, pp. 81-97, 1956.
31. M. C. Mozer, "Neural-net architectures for temporal sequence processing," in Predicting the Future and Understanding the Past, A. Weigend and N. Gershenfeld, Eds. Redwood City, CA: Addison-Wesley, 1993, pp. 243-264.
32. R. Ratcliff, "Connectionist models of recognition memory: Constraints imposed by learning and forgetting function," Psych. Rev., vol. 97, pp. 285-308, 1990.
33. G. J. Rinkus, "TEMECOR: An associative, spatio-temporal pattern memory for complex state sequences," in Proc. World Congr. Neural Networks, Washington, D.C., 1995, pp. I.442-I.448.
34. W. A. Rodriguez, L. S. Borbely, and R. S. Garcia, "Attenuation by contextual cues of retroactive interference of a conditional discrimination in rats," Animal Learning Behavior, vol. 21, pp. 101-105, 1993.
35. V. Ruiz de Angulo and C. Torras, "On-line learning with minimal degradation in feedforward networks," IEEE Trans. Neural Networks, vol. 6, pp. 657-668, 1995.
36. N. E. Sharkey and A. J. C. Sharkey, "Understanding catastrophic interference in neural nets," Dep. Comput. Sci., Univ. Sheffield, U.K., Tech. Rep. CS-94-4, 1994.
37. H. A. Simon, "How big is a chunk?" Sci., vol. 183, pp. 482-488, 1974.
38. S. A. Sloman and D. E. Rumelhart, "Reducing interference in distributed memories through episodic gating," in From Learning Theory to Connectionist Theory: Essays in Honor of William K. Estes, A. F. Healy, S. M. Kosslyn, and R. M. Shiffrin, Eds. Hillsdale, NJ: Lawrence Erlbaum, 1992, pp. 227-248.
39. H. Sompolinsky and I. Kanter, "Temporal association in asymmetric neural networks," Phys. Rev. Lett., vol. 57, pp. 2861-2864, 1986.
40. D. L. Wang, "Temporal pattern processing," in Handbook of Brain Theory and Neural Networks, M. A. Arbib, Ed. Cambridge, MA: MIT Press, 1995, pp. 967-971.
41. D. L. Wang and M. A. Arbib, "Complex temporal sequence learning based on short-term memory," Proc. IEEE, vol. 78, pp. 1536-1543, 1990.
42. _, "Timing and chunking in processing temporal order," IEEE Trans. Syst., Man. Cybern., vol. 23, pp. 993-1009, 1993.
43. D. L. Wang and B. Yuwono, "Anticipation-based temporal pattern generation," IEEE Trans. Syst., Man, Cybern., vol. 25, pp. 615-628, 1995.
44. D. J. Willshaw, O. P. Buneman, and H. C. Longuet-Higgins, "Nonholographic associative memory," Nature, vol. 222, pp. 960-962, 1969.