Internal-time temporal difference model for neural value-based decision making

Neural Computation

Volumn 22, Issue 12, 2010, Pages 3062-3106

  Nakahara, Hiroyuki ^a,b   Kaveri, Sivaramakrishnan ^a,b  

^a RIKEN BRAIN SCIENCE INSTITUTE   (Japan)

^b TOKYO INSTITUTE OF TECHNOLOGY   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

BIOLOGICAL MODEL; BRAIN; DECISION MAKING; LETTER; PHYSIOLOGY; TIME;

BRAIN; DECISION MAKING; MODELS, NEUROLOGICAL; TIME;

EID: 78649667137     PISSN: 08997667     EISSN: 1530888X     Source Type: Journal    
DOI: 10.1162/NECO_a_00049     Document Type: Letter

References (135)

1. Acebron, J. A., Bonilla, L. L., Vicente, C. J. P., Ritort, F., & Spigler, R. (2005). The Kuramoto model: A simple paradigm for synchronization phenomena. Rev. Mod. Phys., 77(1), 137-185.
2. Ainslie, G. (1975). Specious reward: A behavioral theory of impulsiveness and impulse control. Psychol. Bull., 82(4), 463-496.
3. Arzy, S., Molnar-Szakacs, I., & Blanke, O. (2008). Self in time: Imagined self-location influences neural activity related to mental time travel. J. Neurosci., 28(25), 6502-6507.
4. Bayer, H. M., & Glimcher, P. W. (2005). Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron., 47(1), 129-141.
5. Berns, G. S., Laibson, D., & Loewenstein, G. (2007). Intertemporal choice-toward an integrative framework. Trends Cogn. Sci., 11(11), 482-488.
6. Boyer, P. (2008). Evolutionary economics of mental time travel? Trends Cogn. Sci., 12(6), 219-224.
7. Bradtke, S. J., & Duff, M. O. (1995). Reinforcement learning methods for continuous-time Markov decision problems. In G. Tesauro, D.S. Touretzky, & T.K. Leen (Eds.), Advances in neural information processing systems, 7 (pp. 393-400). San Francisco: Morgan Kaufmann.
8. Brown, J., Bullock, D., & Grossberg, S. (1999). How the basal ganglia use parallel excitatory and inhibitory learning pathways to selectively respond to unexpected rewarding cues. J. Neurosci., 19(23), 10502-10511.
9. Buckner, R. L., & Carroll, D. C. (2007). Self-projection and the brain. Trends Cogn. Sci., 11(2), 49-57.
10. Buhot, M. C. (1997). Serotonin receptors in cognitive behaviors. Curr. Opin. Neurobiol., 7(2), 243-254.
11. Buhusi, C. V., & Meck, W. H. (2005). What makes us tick? Functional and neural mechanisms of interval timing. Nat. Rev. Neurosci., 6(10), 755-765.
12. Buonomano, D. V. (2007). The biology of time across different scales. Nat. Chem. Biol., 3(10), 594-597.
13. Cardinal, R. N. (2006). Neural systems implicated in delayed and probabilistic reinforcement. Neural Netw., 19(8), 1277-1301.
14. Cerutti, D. T., & Staddon, J. E. (2004). Immediacy versus anticipated delay in the time-left experiment: A test of the cognitive hypothesis. J. Exp. Psychol. Anim. Behav. Process., 30(1), 45-57.
15. Church, R. M. (1984). Properties of the internal clock. Ann. N.Y. Acad. Sci., 423, 566-582.
16. Cools, R., Roberts, A. C., & Robbins, T. W. (2008). Serotoninergic regulation of emotional and behavioural control processes. Trends Cogn. Sci., 12(1), 31-40.
17. Daw, N. D., Courville, A. C., & Tourtezky, D. S. (2006). Representation and timing in theories of the dopamine system. Neural Comput., 18(7), 1637-1677.
18. Daw, N.D., Kakade, S., & Dayan, P.(2002). Opponent interactions between serotonin and dopamine. Neural Netw., 15(4-6), 603-616.
19. Daw, N. D., Niv, Y., & Dayan, P. (2005). Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat. Neurosci., 8(12), 1704-1711.
20. Daw, N. D., & Touretzky, D. S. (2002). Long-term reward prediction in TD models of the dopamine system. Neural Comput., 14(11), 2567-2583.
21. Dayan, P. (2009). Prospective and retrospective temporal difference learning. Network, 20(1), 32-46.
22. Dayan, P., & Huys, Q. J. (2008). Serotonin, inhibition, and negative mood. PLoS. Comput. Biol., 4(2), e4.
23. Dayan, P., & Niv, Y. (2008). Reinforcement learning: The good, the bad and the ugly. Curr. Opin. Neurobiol., 18(2), 185-196.
24. Delamater, A. R., & Oakeshott, S. (2007). Learning about multiple attributes of reward in Pavlovian conditioning. In B. Balleine, K. Doya, J. O'Doherty, & M. Sakagami (Eds.), Reward and decision making in corticobasal ganglia networks (pp. 1-20). New York: Wiley-Blackwell.
25. Denk, F., Walton, M. E., Jennings, K. A., Sharp, T., Rushworth, M. F., & Bannerman, D. M. (2005). Differential involvement of serotonin and dopamine systems in cost-benefit decisions about delay or effort. Psychopharmacology (Berl.), 179(3), 587-596.
26. Desmond, J. E., & Moore, J. W. (1988). Adaptive timing in neural networks: The conditioned response. Biol. Cybern., 58(6), 405-415.
27. Doya, K. (2000). Reinforcement learning in continuous time and space. Neural Comput., 12(1), 219-245.
28. Doya, K. (2002). Metalearning and neuromodulation. Neural Netw., 15(4-6), 495-506.
29. Doya, K. (2008). Modulators of decision making. Nat. Neurosci., 11(4), 410-416.
30. Doya, K., Samejima, K., Katagiri, K., & Kawato, M. (2002). Multiple model-based reinforcement learning. Neural Comput., 14(6), 1347-1369.
31. Dragoi, V., Staddon, J. E., Palmer, R. G., & Buhusi, C. V. (2003). Interval timing as an emergent learning property. Psychological Review, 110(1), 126-44.
32. Eagleman, D. M. (2008). Human time perception and its illusions. Curr. Opin. Neurobiol., 18(2), 131-136.
33. Fiorillo, C. D., Newsome, W. T., & Schultz, W. (2008). The temporal precision of reward prediction in dopamine neurons. Nat. Neurosci., 11(8), 966-973.
34. Frederick, S., Loewenstein, G., & O'Donoghue, T. (2002). Time discounting and time preference: A critical review. Journal of Economic Literature, 40(2), 351-401.
35. Fudenberg, D., & Levine, D. K. (2006). A dual-self model of impulse control. Am. Econ. Rev., 96(5), 1449-1476.
36. Gallistel, C. R., & Gibbon, J. (2000). Time, rate, and conditioning. Psychol. Rev., 107(2), 289-344.
37. Gallistel, C. R., & Gibbon, J. (2002). The symbolic foundations of conditioned behavior. London: Routledge.
38. Genovesio, A., Tsujimoto, S., & Wise, S. P. (2006). Neuronal activity related to elapsed time in prefrontal cortex. J. Neurophysiol., 95(5), 3281-3285.
39. Ghose, G. M., & Maunsell, J. H. (2002). Attentional modulation in visual cortex depends on task timing. Nature, 419(6907), 616-620.
40. Gibbon, J. (1977).Scalar expectancy theory and Weber's law in animal timing. Psychol. Rev., 84(3), 279-325.
41. Gibbon, J., & Church, R. M. (1981). Time left: Linear versus logarithmic subjective time. J. Exp. Psychol. Anim. Behav. Process, 7(2), 87-107.
42. Gibbon, J., Malapani, C., Dale, C. L., & Gallistel, C. (1997). Toward a neurobiology of temporal cognition: Advances and challenges. Curr. Opin. Neurobiol., 7(2), 170-184.
43. Gilbert, D. T., & Wilson, T. D. (2007). Prospection: Experiencing the future. Science, 317(5843), 1351-1354.
44. Glascher, J., & Buchel, C. (2005). Formal learning theory dissociates brain regions with different temporal integration. Neuron, 47(2), 295-306.
45. Glimcher, P. W., Kable, J., & Louie, K. (2007). Neuroeconomic studies of impulsivity: Now or just as soon as possible? Am. Econ. Rev., 97(2), 142-147.
46. Gold, J. I., & Shadlen, M. N. (2007). The neural basis of decision making. Annu. Rev. Neurosci., 30, 535-574.
47. Grondin, S. (2001). From physical time to the first and second moments of psychological time. Psychological Bulletin, 127(1), 22-44.
48. Grossberg, S., & Schmajuk, N. (1989). Neural dynamics of adaptive timing and temporal discrimination during associative leaning. Neural Netw., 2(2), 79-102.
49. Hare, T.A., O'Doherty, J., Camerer, C.F., Schultz, W., & Rangel, A. (2008). Dissociating the role of the orbitofrontal cortex and the striatum in the computation of goal values and prediction errors. J. Neurosci., 28(22), 5623-5630.
50. Haruno, M., & Kawato, M. (2006). Different neural correlates of reward expectation and reward expectation error in the putamen and caudate nucleus during stimulus-action-reward association learning. J. Neurophysiol., 95(2), 948-959.
51. Hazy, T. E., Frank, M. J., & O'Reilly, R. C. (2010). Neural mechanisms of acquired phasic dopamine responses in learning. Neurosci. Biobehav. Rev., 34(5), 701-720.
52. Hikosaka, O., Nakamura, K., & Nakahara, H. (2006). Basal ganglia orient eyes to reward. J. Neurophysiol., 95(2), 567-584.
53. Ho, M. Y., Velázquez-Martínez, D. N., Bradshaw, C. M., and Szabadi, E. (2002). 5-hydroxytryptamine and interval timing behaviour. Pharmacol. Biochem. Behav., 71(4), 773-785.
54. Houk, J. C., Adams, J. L., & Barto, A. G. (1995). A model of how basal ganglia generate and use neural signals that predict reinforcement. In J. C. Houk, J. L., Davis, & D. G. Beiser (Eds.), Models of information processing in the basal ganglia (pp. 249-270). Cambridge, MA: MIT Press.
55. Ivry, R. B., & Spencer, R. M. (2004). The neural representation of time. Curr. Opin. Neurobiol., 14(2), 225-232.
56. Izhikevich, E. M. (2007). Solving the distal reward problem through linkage of STDP and dopamine signaling. Cereb. Cortex., 17(10), 2443-2452.
57. Jacobs, B. L., & Azmitia, E. C. (1992). Structure and function of the brain serotonin system. Physiol. Rev., 72(1), 165-229.
58. Janssen, P., & Shadlen, M. N. (2005). A representation of the hazard rate of elapsed time in macaque area LIP. Nat. Neurosci., 8(2), 234-241.
59. Kable, J. W., & Glimcher, P. W. (2007). The neural correlates of subjective value during intertemporal choice. Nat. Neurosci., 10(12), 1625-1633.
60. Karmarkar, U. R., & Buonomano, D. V. (2007). Timing in the absence of clocks: Encoding time in neural network states. Neuron. 53(3), 427-438.
61. Killeen, P. R., & Fetterman, J. G. (1988). A behavioral theory of timing. Psychological Review, 95(2), 274-295.
62. Killeen, P. R., Fetterman, J. G., & Bizo, L. A. (1997). Time's causes. In C. M. Bradshaw & E. Szabadi (Eds.), Time and behaviour: Psychological and neurobehavioural analyses (pp. 79-132). Burlington, MA: Elsevier.
63. Kobayashi, S., & Schultz, W. (2008). Influence of reward delays on responses of dopamine neurons. J. Neurosci., 28(31), 7837-7846.
64. Kubota, Y., Liu, J., Hu, D., DeCoteau, W. E., Eden, U. T., Smith, A. C., et al. (2009). Stable encoding of task structure coexists with flexible coding of task events in sensorimotor striatum. J. Neurophysiol., 102(4), 2142-2160.
65. La Camera, G., & Richmond, B. J. (2008). Modeling the violation of reward maximization and invariance in reinforcement schedules. PLoS. Comput. Biol., 4(8), e1000131.
66. Laibson, D. (1997). Golden eggs and hyperbolic discounting. Q. J. Econ., 112(2), 443-477.
67. Leon, M. I., & Shadlen, M. N. (2003). Representation of time by neurons in the posterior parietal cortex of the macaque. Neuron 38(2), 317-327.
68. Lewis, P. A., & Miall, R. C. (2003). Distinct systems for automatic and cognitively controlled time measurement: Evidence from neuroimaging. Curr. Opin. Neurobiol., 13(2), 250-255.
69. Liberman, N., & Trope, Y. (2008). The psychology of transcending the here and now. Science, 322(5905), 1201-1205.
70. Loewenstein, G., & Prelec, D. (1992). Anomalies in intertemporal choice: Evidence and an interpretation. Q. J. Econ., 107(2), 573-597.
71. Lohrenz, T., McCabe, K., Camerer, C. F., & Montague, P. R. (2007). Neural signature of fictive learning signals in a sequential investment task. Proc. Natl. Acad. Sci. U.S.A., 104(22), 9493-9498.
72. Ludvig, E. A., Sutton, R. S., & Kehoe, E. J. (2008). Stimulus representation and the timing of reward-prediction errors in models of the dopamine system. Neural Comput., 20(12), 3034-3054.
73. Luhmann, C. C., Chun, M. M., Yi, D. J., Lee, D., & Wang, X. J. (2008). Neural dissociation of delay and uncertainty in intertemporal choice. J Neurosci., 28(53), 14459-14466.
74. Mahadevan, S. (1996). Average reward reinforcement learning: Foundations, algorithms, and empirical results. Machine Learning, 22(1-3), 159-195.
75. Matell, M., Bateson, M., & Meck, W. (2006). Single-trial analyses demonstrate that increases in clock speed contribute to the methamphetamine-induced horizontal shifts in peak-interval timing functions. Psychopharmacology (Berl.), 188(2), 201-212.
76. Matell, M. S., & Meck, W. H. (2000). Neuropsychological mechanisms of interval timing behavior. Bioessays, 22(1), 94-103.
77. Mauk, M. D., & Buonomano, D. V. (2004). The neural basis of temporal processing. Annu. Rev. Neurosci., 27, 307-340.
78. Mazur, J. E. (2006). Mathematical models and the experimental analysis of behavior. J. Exp. Anal. Behav., 85(2), 275-291.
79. McClure, S. M., Ericson, K. M., Laibson, D. I., Loewenstein, G., & Cohen, J. D. (2007). Time discounting for primary rewards. J. Neurosci., 27(21), 5796-5804.
80. McClure, S.M., Laibson, D. I., Loewenstein, G., & Cohen, J.D. (2004). Separate neural systems value immediate and delayed monetary rewards. Science, 306(5695), 503-507.
81. Meck, W. H. (1996). Neuropharmacology of timing and time perception. Cogn. Brain Res., 3(3-4), 227-242.
82. Miyazaki, K., Miyazaki, K. W., & Doya, K. (2007). Increased serotonin efflux in the dorsal raphe of rats working for delayed rewards. Presented at the 37th Annual Meeting of the Society for Neuroscience, San Diego, CA.
83. Mizuhiki, K., Inaba, K., Toda, S., Ozaki, S., & Shidara, M. (2008). Single neurons in monkey dorsal raphe nucleus respond to reward schedules. Poster session presented at the 38th Annual Meeting of the Society for Neuroscience, Washington, DC.
84. Mobini, S., Chiang, T. J., Al-Ruwaitea, A. S., Ho, M. Y., Bradshaw, C. M., & Szabadi, E. (2000). Effect of central 5-hydroxytryptamine depletion on inter-temporal choice: A quantitative analysis. Psychopharmacology (Berl.), 149(3), 313-318.
85. Montague, P. R., & Berns, G. S. (2002). Neural economics and the biological substrates of valuation. Neuron, 36(2), 265-284.
86. Montague, P. R., Dayan, P., & Sejnowski, T. J. (1996). A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J. Neurosci., 16(5), 1936-1947.
87. Montague, P. R., Hyman, S. E., & Cohen, J. D. (2004). Computational roles for dopamine in behavioural control. Nature, 431(7010), 760-767.
88. Montague, P. R., King-Casas, B., & Cohen, J. D. (2006). Imaging valuation models in human choice. Annu. Rev. Neurosci., 29, 417-448.
89. Morris, G., Nevet, A., Arkadir, D., Vaadia, E., & Bergman, H. (2006). Midbrain dopamine neurons encode decisions for future action. Nat. Neurosci., 9(8), 1057-1063.
90. Nakahara, H., Doya, K., & Hikosaka, O. (2001). Parallel cortico-basal ganglia mechanisms for acquisition and execution of visuomotor sequences-a computational approach. J. Cogn. Neurosci., 13(5), 626-647.
91. Nakahara, H., Itoh, H., Kawagoe, R., Takikawa, Y., & Hikosaka, O. (2004). Dopamine neurons can represent context-dependent prediction error. Neuron, 41(2), 269-280.
92. Nakahara, H., Nakamura, K., & Hikosaka, O. (2006). Extended LATER model can account for trial-by-trial variability of both pre- and post-processes. Neural Netw., 19(8), 1027-1046.
93. Nakamura, K., Matsumoto, M., & Hikosaka, O. (2008). Reward-dependent modulation of neuronal activity in the primate dorsal raphe nucleus. J. Neurosci., 28(20), 5331-5343.
94. Niv, Y., Daw, N., & Dayan, P. (2006). Tonic dopamine: Opportunity costs and the control of response vigor. Psychopharmacology (Berl.), 191(3), 507-520.
95. Nobre, A., Correa, A., & Coull, J. (2007). The hazards of time. Curr. Opin. Neurobiol., 17(4), 465-470.
96. O'Doherty, J. P., Dayan, P., Friston, K., Critchley, H., & Dolan, R. J. (2003). Temporal difference models and reward-related learning in the human brain. Neuron, 38(2), 329-337.
97. O'Reilly, R. C., Frank, M. J., Hazy, T. E., & Watz, B. (2007). PVLV: The primary value and learned value Pavlovian learning algorithm. Behav. Neurosci., 121(1), 31-49.
98. Pan, W. X., Schmidt, R., Wickens, J. R., & Hyland, B. I. (2005). Dopamine cells respond to predicted events during classical conditioning: Evidence for eligibility traces in the reward-learning network. J. Neurosci., 25(26), 6235-6242.
99. Potjans, W., Morrison, A., & Diesmann, M. (2009). A spiking neural network model of an actor-critic learning agent. Neural Comput., 21(2), 301-339.
100. Preuschoff, K., Bossaerts, P., & Quartz, S. R. (2006). Neural differentiation of expected reward and risk in human subcortical structures. Neuron, 51(3), 381-390.
101. Puterman, M. L. (1994). Markov decision processes: Discrete stochastic dynamic programming. Hoboken, NJ: Wiley.
102. Rammsayer, T. H. (1999). Neuropharmacological evidence for different timing mechanisms in humans. Q. J. Exp. Psychol. B, 52(3), 273-286.
103. Ranade, S. P., & Mainen, Z. F. (2009). Transient firing of dorsal raphe neurons encodes diverse and specific sensory, motor, and reward events. J. Neurophysiol., 102(5), 3026-3037.
104. Redgrave, P., & Gurney, K. (2006). The short-latency dopamine signal: A role in discovering novel actions? Nat. Rev. Neurosci., 7(12), 967-975.
105. Roesch, M. R., & Olson, C. R. (2005). Neuronal activity dependent on anticipated and elapsed delay in macaque prefrontal cortex, frontal and supplementary eye fields, and premotor cortex. J. Neurophysiol., 94(2), 1469-1497.
106. Rubinstein, A. (2003). Economics and psychology? The case of hyperbolic discounting. Int. Econ. Rev., 44(4), 1207-1216.
107. Samuelson, P. A. (1937). A note on measurement of utility. Rev. Econ. Studies, 4(2), 155-161.
108. Schacter, D. L., Addis, D. R., & Buckner, R. L. (2007). Remembering the past to imagine the future: The prospective brain. Nat. Rev. Neurosci., 8(9), 657-661.
109. Schultz, W. (1998). Predictive reward signal of dopamine neurons. J. Neurophysiol., 80(1), 1-27.
110. Schultz, W., Dayan, P., & Montague, P. R. (1997). A neural substrate of prediction and reward. Science, 275(5306), 1593-1599.
111. Schwartz, A. (1993). A reinforcement learning method for maximizing undiscounted rewards. In Proceedings of the Tenth International Conference in Machine Learning (pp. 298-305). San Francisco: Morgan Kaufmann.
112. Schweighofer, N., Bertin, M., Shishida, K., Okamoto, Y., Tanaka, S. C., Yamawaki, S., et al. (2008). Low-serotonin levels increase delayed reward discounting in humans. J. Neurosci., 28(17), 4528-4532.
113. Schweighofer, N., Shishida, K., Han, C. E., Okamoto, Y., Tanaka, S. C., Yamawaki, S., et al. (2006). Humans can adopt optimal discounting strategy under real-time constraints. PLoS Comput. Biol., 2(11), e152.
114. Sershen, H., Hashim, A., & Lajtha, A. (2000). Serotonin-mediated striatal dopamine release involves the dopamine uptake site and the serotonin receptor. Brain Res. Bull., 53(3), 353-357.
115. Seymour, B., Daw, N. D., Dayan, P., Singer, T., & Dolan, R. J. (2007). Differential encoding of losses and gains in the human striatum. J. Neurosci., 27(18), 4826-4831.
116. Solomon, R. B., Conover, K. L., & Shizgal, P. (2007). Estimation of subjective opportunity costs in rats working for rewarding brain stimulation: Further progress. Presented at the 37th annual meeting of the Society for Neuroscience, San Diego, CA.
117. Soltani, A., & Wang, X. J. (2010). Synaptic computation underlying probabilistic inference. Nat. Neurosci., 13(1), 112-119.
118. Staddon, J. E., & Higa, J. J. (1999). Time and memory: Towards a pacemaker-free theory of interval timing. J. Exp. Anal. Behav., 71(2), 215-251.
119. Suri, R. (2001). Anticipatory responses of dopamine neurons and cortical neurons reproduced by internal model. Exp. Brain Res., 140(2), 234-240.
120. Suri, R. E., & Schultz, W. (2001). Temporal difference model reproduces anticipatory neural activity. Neural Comput., 13(4), 841-862.
121. Sutton, R. S. (1995). TD models: Modeling the world at a mixture of time scales. In A. Prieditis & S. Russell (Eds.) Proceedings of the Twelfth International Conference in Machine Learning (pp. 531-539). San Francisco: Morgan Kaufmann.
122. Sutton, R. S., & Barto, A. G. (1990). Time-derivative models of Pavlovian reinforcement. In M. Gabriel & J. Moore (Eds.), Learning and computational neuroscience: Foundations of adaptive networks (pp. 497-537). Cambridge, MA: MIT Press.
123. Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction. Cambridge, MA: MIT Press.
124. Szpunar, K. K., Watson, J. M., & McDermott, K. B. (2007). Neural substrates of envisioning the future. Proc. Natl. Acad. Sci. U.S.A., 104(2), 642-647.
125. Takahashi, T. (2005). Loss of self-control in intertemporal choice may be attributable to logarithmic time-perception. Med. Hypotheses, 65(4), 691-693.
126. Tan, C. O., & Bullock, D. (2008). A dopamine-acetylcholine cascade: Simulating learned and lesion-induced behavior of striatal cholinergic interneurons. J. Neurophysiol., 100(4), 2409-2421.
127. Tanaka, S. C., Doya, K., Okada, G., Ueda, K., Okamoto, Y., & Yamawaki, S. (2004). Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops. Nat. Neurosci., 7(8), 887-893.
128. Thaler, R. H., & Shefrin, H. M. (1981). An economic theory of self-control. J. Polit. Econ., 89(2), 392-406.
129. Tsitsiklis, J. N., & Van Roy, B. (2002). On average versus discounted reward temporal-difference learning. Machine Learning, 49(2-3), 179-191.
130. Winstanley, C. A., Eagle, D. M., & Robbins, T. W. (2006). Behavioral models of impulsivity in relation to ADHD: Translation between clinical and preclinical studies. Clin. Psychol. Rev., 26(4), 379-395.
131. Wise, R. A. (2004). Dopamine, learning and motivation. Nat. Rev. Neurosci., 5(6), 483-494.
132. Wittmann, M., & Paulus, M. P. (2008). Decision making, impulsivity and time perception. Trends Cogn. Sci., 12(1), 7-12.
133. Yamada, H., Matsumoto, N., & Kimura, M. (2007). History- and current instruction-based coding of forthcoming behavioral outcomes in the striatum. J. Neurophysiol., 98(6), 3557-3567.
134. Zeiler, M. D. (1998). On sundials, springs, and atoms. Behav. Processes., 44(2), 89-99.
135. Zhang, K. (1996). Representation of spatial orientation by the intrinsic dynamics of the head-direction cell ensemble: A theory. J. Neurosci., 16(6), 2112-2126.