Timing and Chunking in Processing Temporal Order

IEEE Transactions on Systems, Man and Cybernetics

Volumn 23, Issue 4, 1993, Pages 993-1009

  Wang, DeLiang ^a   Arbib, Michael A ^b  

^a OHIO STATE UNIVERSITY   (United States)

^b UNIVERSITY OF SOUTHERN CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATION THEORY; COMPUTER SIMULATION; CYBERNETICS; HIERARCHICAL SYSTEMS; LEARNING SYSTEMS; PATTERN RECOGNITION;

CHUNKING; HEBBIAN LEARNING; HUMAN INTELLIGENCE; INTERFERENCE THEORY OF FORGETTING; NEURON POPULATIONS; SEQUENCE REPRODUCTION; SEQUENTIAL SYSTEM ACTIVATION; SHORT-TERM MEMORY MODEL; TEMPORAL ORDER; TIMING;

NEURAL NETWORKS;

EID: 0027632575     PISSN: 00189472     EISSN: 21682909     Source Type: Journal    
DOI: 10.1109/21.247884     Document Type: Article

References (66)

1. S. Amari and M. A. Arbib, “Competition and cooperation in neural nets,” in Systems neuroscience, J. Metzler, Ed. New York: Academic, 1977, pp. 119–165.
2. S. Anderson, R. F. Port, and J. D. McAuley, “Dynamic memory: a model for auditory pattern recognition,” preprint.
3. M. A. Arbib, “Programs, schemas, and neural networks for control of hand movements: Beyond the RS framework,” in Attention and performance XIII. Motor representation and control, M. Jeannerod, Ed. Hillsdale, NJ: Erlbaum, 1990, pp. 111–138.
4. B. T. Baiteli, G. W. Cottrell, and J. L. Elman, “The role of input and target similarity in assimilation,” in Proc. 13th Ann. Conf. Cog. Sci. Soc. Hillsdale, NJ: Erlbaum, pp. 322–327, 1991.
5. J. Buhmann and K. Schulten, “Noise-driven temporal association in neural networks,” Europhys. Lett., vol. 4, pp. 1205–1209, 1987.
6. G. A. Carpenter, and S. Grossberg, “A massively parallel architecture for a self-organizing neural pattern recognition machine,” Computer Vision, Graphs, and Image Processing, vol. 37, pp. 54–115, 1987.
7. P. A. Carpenter and M. A. Just, “The role of working memory in language comprehension,” in Complex information processing: The impact of Herbert A. Simon, D. Klahr and K. Kotovsky, Eds. Hillsdale, NJ: Erlbaum, 1989.
8. A. Cohen, R. I. Ivry, and S. W. Keele, “Attention and structure in sequence learning,” J. Exp. Psychol, vol. 16, pp. 17–30, 1990.
9. S. R. Colman and I. Gormezano, “Classical conditioning of the rabbit's (Oryctolagus cuniculus) nictitating membrane response under symmetrical CS-US interval shifts,” J. Comp. Physiol. Psychol, vol. 77, pp. 447–455, 1971.
10. R. Conrad, “Decay theory of immediate memory,” Nature, vol. 179, pp. 831–832, 1957.
11. T. Dehaene, J. P. Changeux, and J. P. Nadal, “Neural networks that learn temporal sequences by selection,” in Proc. Nat. Acad. Sci. USA, vol. 84, 1987, pp. 2727–2731.
12. R. L. Didday, “The simulation and modeling of distributed information processing in the frog visual system,” Ph.D dissertion, Stanford University, 1970.
13. R. L. Didday and M. A. Arbib, “Eye movements and visual perception: ‘Two visual systems’ model,” Int. J. Man-Machine Stud, vol. 7, pp. 547–569, 1975.
14. K. Doya and S. Yoshizawa, “Adaptive neural oscillator using continuous-time Backpropagation learning,” Neur. Netw., vol. 2, pp. 375–385, 1989.
15. K. Doya and S. Yoshizawa, “Memorizing hierarchical temporal patterns in analog neuron networks,” in Proc. Int. Joint Conf. Neur. Netw., vol. 3, San Diego, CA, pp. 299–304, 1990.
16. R. O. Duda and P. E. Hart, Pattern Classification and Scene Analysis. New York: Wiley, 1973.
17. H. C. Ebel and W. F. Prokasy, “Classical eyelid conditioning as a function of sustained and shifted interstimulus intervals,” J. Exp. Psychol., vol. 65, pp. 52–58, 1963.
18. J. L. Elman, “Finding structure in time,” Cog. Sci., vol. 14, pp. 179–211, 1990.
19. W. J. Freeman, Y. Yao, and B. Burke, “Central pattern generating and recognizing in olfactory bulb: A correlation learning rule,” Neur. Netw., vol. 1, pp. 277–288, 1988.
20. S. Grossberg, “Some networks that can learn, remember, and reproduce any number of complicated space-time patterns—Part I,” J. Math. Mechan, vol. 19, pp. 53–91, 1969.
21. S. Grossberg, “Adaptive pattern classification and universal recoding: I. Parallel development and coding of neural feature detectors,” Biol. Cybern., vol. 23, pp. 121–134, 1976.
22. H. Gutfreund and M. Mezard, “Processing of temporal sequences in neural networks,” Phys. Rev. Lett., vol. 61, pp. 235–238, 1988.
23. I. Guyon, L. Personnaz, J. P. Nadal, and G. Dreyfus, “Storage and retrieval of complex sequences in neural networks,” Phys. Rev. A, vol. 38, pp. 6365–6372, 1988.
24. D. O. Hebb, The Organization of Behavior. New York: Wiley, 1949.
25. T. M. Heskes and S. Gielen, “Retrieval of pattern sequences at variable speeds in a neural network with delays,” Neur. Netw., vol. 5, pp. 145–152, 1992.
26. F. K. Hoehler and R. F. Thompson, “Effect of the interstimulus (CSUCS) interval on hippocampal unit activity during classical conditioning of the nictitating membrane response of the rabbit (Oryctolagus cuniculus),” J. Comp. Physiol. Psychol, vol. 94, pp. 201–215, 1980.
27. P. J. Jennings and S. W. Keele, “A computational model of attentional requirements in sequence learning,” in Proc. 12th Ann. Conf. Cog. Sci. Soc. Hillsdale, NJ: Erlbaum, 1990, pp. 876–883.
28. M. I. Jordan, “Attractor dynamics and parallelism in a connectionist sequential machine,” in Proc. 8th Ann. Conf. Cog. Sci. Soc. Hillsdale, NJ: Erlbaum, 1986, pp. 531–546.
29. M. I. Jordan, “Learning to articulate: Sequential networks and distal constraints,” in Attention and Performance XIII. Motor representation and control, M. Jeannerod, Ed. Hillsdale, NJ: Erlbaum, 1990.
30. D. Kleinfeld, “Sequential state generation by model neural networks,” Proc. Nat. Acad. Sci. USA, vol. 83, 1986, pp. 9469–9473.
31. C. Koch and S. Ullman, “Shifts in selective visual attention: towards the underlying neural circuitry,” Human Neurobiol., vol. 4, pp. 219–227, 1985.
32. T. Kohonen, “Self-organized formation of topologically correct feature maps,” Biol. Cybern, vol. 43, pp. 59–69, 1982.
33. T. Kohonen, “Dynamically expanding context, with application to the correction of symbol strings in the recognition of continuous speech,” in Proc. Int. Conf. Neur. Netw., vol. 2, San Diego, CA, 1987, pp. 3–9.
34. T. Kohonen, “A self-learning musical grammar, or ‘associative memory of the second kind’,” in Proc. Int. Joint Conf. Neur. Netw., vol. 1, Washington, DC, 1989, pp. 1–5.
35. S. Kojima and P. S. Goldman-Rakic, “Delay-related activity of prefrontal neurons in rhesus monkeys performing delayed response,” Brain Res., vol. 248, pp. 43–49, 1982.
36. B. Kosko, “Bidirectional associative memory,” IEEE Trans. Syst., Man, Cybern., vol. SMC-18, pp. 49–60, 1988.
37. R. Kühn, J. L. van Hemmen, and U. Riedel, “Complex temporal association in neural networks,” J. Phys. A, vol. 22, pp. 3123–3135, 1989.
38. S. Kurogi, “A model of neural network for spatiotemporal pattern recognition,” Biol. Cybern., vol. 57, pp. 103–114, 1987.
39. K. J. Lang, A. H. Waibel, and G. E. Hinton, “A time-delay neural network architecture for isolated word recognition,” Neur. Netw., vol. 3, pp. 23–43, 1990.
40. K. S. Lashley, “The problem of serial order in behavior,” in Cerebral mechanisms in behavior, L. A. Jeffress, Ed. New York: Wiley, 1951, pp. 112–146.
41. D. W. Leonard and J. Theios, “Effect of CS-US interval shift on classical conditioning of the nictitating membrane in the rabbit,” J. Comp. Physiol. Psychol, vol. 63, pp. 355–358, 1967.
42. W. J. M. Levelt, Speaking: From intention to articulation. Cambridge, MA: MIT Press, 1989.
43. C. V. D. Malsburg, “Self-organization of orientation sensitive cells in the striate cortex,” Kybernetik, vol. 14, pp. 85–100, 1973.
44. G. A. Miller, “The magical number seven, plus or minus two: some limits on our capacity for processing information,” Psychol. Rev., vol. 63, pp. 81–97, 1956.
45. B. B. Murdock, Jr., “Serial-order effects in a distributed-memory model,” in Memory and learning: The Ebbinghaus centennial conference, D. S. Gorfein and R. R. Hoffman, Eds. Hillsdale, NJ: Erlbaum, 1987, pp. 227–310.
46. M. J. Nissen and P. Bullemer, “Attentional requirements of learning: Evidence from performance measures,” Cog. Psychol, vol. 19, pp. 1–32, 1987.
47. L. R. Peterson and J. P. Peterson, “Short-term retention of individual verbal items,” J. Exp. Psychol, vol. 58, pp. 193–198, 1959.
48. J. B. Pollack, “The induction of dynamic recognizers,” Machine Learning, vol. 7, pp. 227–252, 1991.
49. S. K. Reed, Cognition: Theory and Applications. Monterey, CA: Brooks/Cole, 1982.
50. D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning internal representations by error propagation,” in Parallel Distributed Processing, D. E. Rumelhart and J. L. McClelland, Eds. Cambridge, MA: The MIT Press, 1986, vol. 1, pp. 318–362.
51. D. E. Rumelhart and D. Zipser, “Feature discovery by competitive learning,” in Parallel Distributed Processing, D. E. Rumelhart and J. L. McClelland, Eds. Cambridge, MA: The MIT Press, 1986, vol. 1, pp. 151–193.
52. D. Servan-Schreiber, A. Cleeremans, and J. L. McClelland, “Learning sequential structure in simple recurrent networks,” in Advances in Neural Information Processing Systems I, D. S. Touretzky, Ed. San Mateo, CA: Morgan Kaufmann, 1989, pp. 643–652.
53. H. A. Simon, “How big is a chunk?” Science, vol. 183, pp. 482–488, 1974.
54. M. C. Smith, “CS-US interval and US intensity in classical conditioning of the rabbit's nictitating membrane response,” J. Comp. Physiol Psychol, vol. 65, pp. 679–687, 1965.
55. H. Sompolinsky and I. Kanter, “Temporal association in asymmetric neural networks,” Phys. Rev. Lett., vol. 57, pp. 2861–2864, 1986.
56. O. Sporns, G. Tononi, and G. M. Edelman, “Modeling perceptual grouping and figure-ground segregation by means of active re-entrant connections,” Proc. Nat. Acad. Sci. USA, vol. 88, pp. 129–133, 1991.
57. J. C. Stanley and W. L. Kilmer, “A wave model of temporal sequence learning,” Int. J. Man-Machine Stud., vol. 7, pp. 397–412, 1975.
58. S. Sternberg, “highspeed scanning in human memory,” Science, vol. 153, pp. 652–654, 1966.
59. D. W. Tank and J. J. Hopfield, “Neural computation by concentrating information in time,” Proc. Natl. Acad. Sci. USA, vol. 84, pp. 1896–1900, 1987.
60. K. P. Unnikrishnan, J. J. Hopfield, and D. W. Tank, “Speaker-independent digit recognition using a neural network with time-delayed connections,” Neur. Comp., vol. 4, pp. 108–119, 1992.
61. A. Waibel, “Modular construction of time-delay neural networks for speech recognition,” Neur. Comp., vol. 1, pp. 39–46, 1989.
62. A. Waibel, T. Hanazawa, G. E. Hinton, K. Shikano, And K. J. Lang, “Phoneme recognition using time-delay neural networks,” IEEE Trans. Acoust. Speech and Signal Processing, vol. ASSP-37, pp. 328–339, 1989.
63. D. L. Wang and M. A. Arbib, “Complex temporal sequence learning based on short-term memory,” Proc. IEEE, vol. 78, pp. 1536–1543, 1990.
64. D. L. Wang, J. Buhmann, and C.v.d. Malsburg, “Pattern segmentation in associative memory,” Neur. Comp., vol. 2, pp. 94–106, 1990.
65. N. C. Waugh and D. A. Norman, “Primary memory,” Psychol. Rev., vol. 72, pp. 89–104, 1965.
66. R. J. Williams and D. Zipser, “A learning algorithm for continually running fully recurrent neural networks,” Neur. Comp., vol. 1, pp. 270–280, 1989.