State-dependent computation using coupled recurrent networks

Neural Computation

Volumn 21, Issue 2, 2009, Pages 478-509

  Rutishauser, Ueli ^a   Douglas, Rodney J ^b  

^a CALIFORNIA INSTITUTE OF TECHNOLOGY   (United States)

^b UNIVERSITY OF ZURICH   (Switzerland)

Author keywords

[No Author keywords available]

Indexed keywords

ANIMAL; ARTIFICIAL NEURAL NETWORK; BRAIN CORTEX; COMPUTER SIMULATION; CYTOLOGY; INFORMATION PROCESSING; LETTER; NERVE CELL; NERVE CELL NETWORK; NERVE TRACT; NONLINEAR SYSTEM; PHYSIOLOGY; REACTION TIME; SYNAPSE;

ANIMALS; AUTOMATIC DATA PROCESSING; CEREBRAL CORTEX; COMPUTER SIMULATION; NERVE NET; NEURAL NETWORKS (COMPUTER); NEURAL PATHWAYS; NEURONS; NONLINEAR DYNAMICS; REACTION TIME; SYNAPSES;

EID: 67650295931     PISSN: 08997667     EISSN: 1530888X     Source Type: Journal    
DOI: 10.1162/neco.2008.03-08-734     Document Type: Letter

References (59)

1. Abbott, L. (1994). Decoding neuronal firing and modelling neural networks. Q. Rev. Biophys., 27, 291-331.
2. Abeles, M. (1991). Corticonics: Neural circuits of the cerebral cortex. Cambridge: Cambridge University Press.
3. Baca, S., Marin-Burgin, A., Wagenaar, D., & Kristan, W. (2008).Widespread inhibition proportional to excitation controls the gain of a leech behavioral circuit. Neuron, 57(2), 276-289.
4. Barone, P., & Joseph, J. (1989). Prefrontal cortex and spatial sequencing in macaque monkey. Experimental Brain Research, 78, 447-464.
5. Ben-Yishai, R., Bar-Or, R., & Sompolinsky, H. (1995). Theory of orientation tuning in visual cortex. Proceedings of the National Academy of Sciences of the USA-Biological Sciences, 92(9), 3844-3848.
6. Binzegger, T., Douglas, R. J., & Martin, K. A. (2004). A quantitative map of the circuit of cat primary visual cortex. J. Neurosci., 24(39), 8441-8453.
7. Bongard, J., & Lipson, H. (2005). Active coevolutionary learning of deterministic finite automata. Journal of Machine Learning Research, 6, 1651-1678.
8. Carrasco, R., Forcada, M., Valdés-Muñoz, M. A., & Neco, R. P. (2000). Stable encoding of finite-statemachines in discrete-time recurrent neural nets with sigmoid units. Neural Computation, 12(9), 2129-2174.
9. Cohen, M. A., & Grossberg, S. (1983). Absolute stability of global pattern-formation and parallel memory storage by competitive neural networks. IEEE Transactions on Systems, Man, and Cybernetics, 13(5), 815-826.
10. Coultrip, R., Granger, R., & Lynch, G. (1992). A cortical model of winner-take-all competition via lateral inhibition. Neural Networks, 5, 47-54.
11. Dayan, P., & Abbott, L. (2001). Theoretical neuroscience. Cambridge, MA: MIT Press.
12. Diesmann, M., Gewaltig, M., & Aertsen, A. (1999). Stable propagation of synchronous spiking in cortical neural networks. Nature, 402, 529-533.
13. Douglas, R., Koch, C., Mahowald, M., Martin, K., & Suarez, H. (1995). Recurrent excitation in neocortical circuits. Science, 269(5226), 981-985.
14. Douglas, R., & Martin, K. (2004). Neuronal circuits of the neocortex. Annu. Rev. Neurosci., 27, 419-451.
15. Douglas, R., & Martin, K. (2007). Recurrent neuronal circuits in the neocortex. Curr. Biol., 17(13), R496-R500.
16. Elman, J. (1991). Distributed representations, simple recurrent networks, and grammatical structure. Machine Learning, 7(2), 195-225.
17. Fujii, N., & Graybiel, A. (2003). Representation of action sequence boundaries by macaque prefrontal cortical neurons. Science, 301, 1246-1249.
18. Giles, C., Miller, C., Chen, D., Chen, H., Sun, G., & Lee, Y. (1992). Learning and extracting finite state automata with second-order recurrent neural networks. Neural Computation, 4, 393-405.
19. Grossberg, S. (1973). Contour enhancement, short term memory, and constancies in reverberating neural networks. Studies in Applied Mathematics, 52, 213-257.
20. Hahnloser, R. (1998). On the piecewise analysis of networks of linear threshold neurons. Neural Networks, 11(4), 691-697.
21. Hahnloser, R., Douglas, R., Mahowald, M., & Hepp, K. (1999). Feedback interactions between neuronal pointers and maps for attentional processing. Nat. Neurosci., 2(8), 746-752.
22. Hahnloser, R., Sarpeshkar, R., Mahowald, M., Douglas, R., & Seung, H. S. (2000). Digital selection and analogue amplification coexist in a cortex-inspired silicon circuit. Nature, 405(6789), 947-951.
23. Hahnloser, R., Seung, H., & Slotine, J. (2003). Permitted and forbidden sets in symmetric threshold-linear networks. Neural Comput., 15(3), 621-638.
24. Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735-1780.
25. Hopcroft, J. (1971). An n log n algorithm for minimizing states in a finite automaton (Tech. Rep. CS-TR-71-190). Stanford, CA: Stanford University.
26. Hopcroft, J., Motwani, R., & Ullman, J. (2000). Introduction to automata theory, languages, and computation. Menlo Park, CA: Addison-Wesley.
27. Hopfield, J. J. (1984). Neurons with graded response have collective computational properties like those of 2-state neurons. Proceedings of the National Academy of Sciences of the USA-Biological Sciences, 81(10), 3088-3092.
28. Jaeger, H., & Haas, H. (2004). Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless communication. Science, 304, 78-80.
29. Kalitzin, S., van Dijk, B.W., & Spekreijse, H. (2000). Self-organized dynamics in plastic neural networks: Bistability and coherence. Biological Cybernetics, 83, 139-150.
30. Kremer, S.C. (1995). On the computational power of Elman-style recurrent networks. IEEE Transactions on Neural Networks, 6(4), 1000-1004.
31. Kurt, S., Deutscher, A., Crook, J., Ohl, F., Budinger, E., Moeller, C., et al. (2008). Auditory cortical contrast enhancing by globalwinner-take-all inhibitory interactions. PLoS ONE, 3(3), e1735.
32. Maass, W. (2000). On the computational power of winner-take-all. Neural Computation, 12, 2519-2536.
33. Maass, W., Joshi, P., & Sontag, E. (2007). Computational aspects of feedback in neural circuits. PLOS Computational Biology, 15-34.
34. Machens, C., Romo, R., & Brody, C. (2005). Flexible control of mutual inhibition: A neural model of two-interval discrimination. Science, 307(5712), 1121-1124.
35. McCulloch, W., & Pitts, W. (1943). A logical calculus of the ideas immanent in nervous activity. Bulletin of Mathematical Biology, 5(4), 115-133.
36. Minksy, L. (1967). Computation: Finite and infinite machines. Englewood Cliffs, NJ: Prentice Hall.
37. Mushiake, H., Inase, M., & Tanji, J. (1990). Selective coding of motor sequence in the supplementary motor area of the monkey cerebral cortex. Experimental Brain Research, 82, 208-210.
38. Natschlager, T., & Maass, W. (2002). Spiking neurons and the induction of finite state machines. Theoretical Computer Science, 287, 251-265.
39. Neftci, E., Chicca, E., Indiveri, G., Slotine, J., & Douglas, R. (2008). Contraction properties of VLSI cooperative competitive neural networks of spiking neurons. In J. Platt, D. Koller, Y. Singer, & S. Roweis, (Eds.), Advances in neural information processing systems, 20 (pp. 1073-1080). Cambridge, MA: MIT Press.
40. Omlin, C. W., & Giles, C. L. (1996). Constructing deterministic finite-state automata in recurrent neural networks. Journal of the ACM, 43(6), 937-972.
41. Pavasovic, A., Andreou, A., & Westgate, C. (1994). Characterization of subthreshold MOS mismatch in transistors for VLSI systems. Journal of VLSI Signal Processing, 8(1), 75-85.
42. Pouget, A., Zhang, K., Deneve, S., & Latham, P. (1998). Statistically efficient estimation using population coding. Neural Computation, 15(10), 373-401.
43. Rabinovich, M., Volkovskii, A., Lecanda, P., Huerta, R., Abarbanel, H. D. I., & Laurent, G. (2001). Dynamical encoding by networks of competing neuron groups: Winnerless competition. Phys. Rev. Lett., 87(6), 068102.
44. Rao, P. (2004). Bayesian computation in recurrent neural circuits. Neural Computation, 16, 1-38.
45. Rodger, S., & Finley, T. (2006). JFLAP-An interactive formal languages and automata package. Sudbury, MA: Jones and Bartlett.
46. Seliger, P., Tsimring, L., & Rabinovich, M. (2003). Dynamics-based sequential memory: Winnerless competition of patterns. Phys. Rev. E, 67(1), 011905.
47. Serrano-Gotarredona, T., & Linares-Barranco, B. (1999). Systematic width-and-length dependent CMOS transistor mismatch characterization and simulation. Analog Integrated Circuits and Signal Processing, 21(3), 271-296.
48. Seung, H. S., Richardson, T. J., Lagarias, J. C., & Hopfield, J. J. (1998). Minimax and Hamiltonian dynamics of excitatory-inhibitory networks. In M. Jordan, M. Kearns, & S. Solla (Eds.), Advances in neural information processing systems, 10. Cambridge, MA: MIT Press.
49. Shepherd, G. M., Stepanyants, A., Bureau, I., Chklovskii, D., & Svoboda, K. (2005). Geometric and functional organization of cortical circuits. Nat. Neurosci., 8(6), 782-790.
50. Shima, K., & Tanji, J. (2000). Neuronal activity in the supplementary and presupplementary motor areas of temporal organization of multiple movements. Journal of Neurophysiology, 84, 2148-2160.
51. Strogatz, S. (1994). Nonlinear dynamics and chaos. Boulder, CO: Westview Press.
52. Swadlow, H. (2002). Thalamocortical control of feed-forward inhibition in awake somatosensory barrel cortex. Philos. Trans. R. Soc. Lond. B Biol. Sci., 357, 1717-1727.
53. Tomioka, R., Okamoto, K., Furuta, T., Fujiyama, F., Iwasato, T., Yanagawa, Y., et al. (2005). Demonstration of long-range GABAergic connections distributed throughout the mouse neocortex. European Journal of Neuroscience, 21(6), 1587-1600.
54. von Neumann, J. (1958). The computer and the brain. New Haven, CT: Yale University Press.
55. Wang, X. (2002). Probabilistic decision making by slow reverberation in cortical circuits. Neuron, 36, 955-968.
56. Wersing, H., Beyn, W. J., & Ritter, H. (2001). Dynamical stability conditions for recurrent neural networks with unsaturating piecewise linear transfer functions. Neural Computation, 13(8), 1811-1825.
57. Wolfram, S. (1984). Universality and complexity in cellular automata. Physica D, 10(1-2), 1-35.
58. Xie, X., Hahnloser, R., & Seung, S. (2002). Double-ring network model of the head-direction system. Phys. Rev. E Stat. Nonlin. Soft Matter Phys., 66(4 Pt. 1), 041902.
59. Yuille, A., & Grzywacz, N. (1989). A winner-take-all mechanism based on presynaptic inhibition feedback. Neural Computation, 1(3), 334-347.