Sensorimotor Learning Biases Choice Behavior: A Learning Neural Field Model for Decision Making

PLoS Computational Biology

Volumn 8, Issue 11, 2012, Pages

  Klaes, Christian ^a,b   Schneegans, Sebastian ^c   Schöner, Gregor ^c   Gail, Alexander ^a  

^a GERMAN PRIMATE CENTER   (Germany)

^b CALIFORNIA INSTITUTE OF TECHNOLOGY   (United States)

^c RUHR UNIVERSITY BOCHUM   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

LEARNING ALGORITHMS; MAMMALS; NEURAL NETWORKS; SPECIFICATIONS;

ACTION SELECTION; CHOICE BEHAVIORS; DECISIONS MAKINGS; ENCODINGS; MOVEMENT PLANNING; NEURAL FIELD MODELS; NEURAL SUBSTRATES; SENSORIMOTORS; SENSORY INPUT; SEQUENTIAL OPERATIONS;

DECISION MAKING;

ACCURACY; ANIMAL BEHAVIOR; ANIMAL EXPERIMENT; ARTICLE; ASSOCIATION; BODY WEIGHT; CONTROLLED STUDY; DECISION MAKING; EXPERIMENTAL MONKEY; HAPLORHINI; LEARNING; LEARNING ALGORITHM; MEMORY; MOTOR PERFORMANCE; NERVE CELL PLASTICITY; NEUROPHYSIOLOGY; NONHUMAN; PREDICTION; REWARD; SENSORIMOTOR FUNCTION; SENSORIMOTOR INTEGRATION; STIMULUS RESPONSE; TASK PERFORMANCE; WORKING MEMORY;

EID: 84870660025     PISSN: 1553734X     EISSN: 15537358     Source Type: Journal    
DOI: 10.1371/journal.pcbi.1002774     Document Type: Article

References (78)

1. Tversky A, Kahneman D, (1981) The Framing of Decisions and the Psychology of Choice. Science 211: 453-458 doi:10.1126/science.7455683.
2. Cisek P, (2006) Integrated neural processes for defining potential actions and deciding between them: a computational model. J Neurosci 26: 9761-9770.
3. Cisek P, Kalaska JF, (2010) Neural mechanisms for interacting with a world full of action choices. Annu Rev Neurosci 33: 269-298.
4. Andersen RA, Cui H, (2009) Intention, Action Planning, and Decision Making in Parietal-Frontal Circuits. Neuron 63: 568-583 doi:10.1016/j.neuron.2009.08.028.
5. Klaes C, Westendorff S, Chakrabarti S, Gail A, (2011) Choosing Goals, Not Rules: Deciding among Rule-Based Action Plans. Neuron 70: 536-548 doi:10.1016/j.neuron.2011.02.053.
6. Wise SP, Boussaoud D, Johnson PB, Caminiti R, (1997) Premotor and parietal cortex: Corticocortical connectivity and combinatorial computations 1. Annu Rev Neurosci 20: 25-42.
7. Burnod Y, Baraduc P, Battaglia-Mayer A, Guigon E, Koechlin E, et al. (1999) Parieto-frontal coding of reaching: an integrated framework. Exp Brain Res 129: 325-346.
8. Andersen RA, Buneo CA, (2002) Intentional Maps in Posterior Parietal Cortex. Annu Rev Neurosci 25: 189-220 doi:10.1146/annurev.neuro.25.112701.142922.
9. Battaglia-Mayer A, Caminiti R, Lacquaniti F, Zago M, (2003) Multiple levels of representation of reaching in the parieto-frontal network. Cereb Cortex 13: 1009-1022.
10. Cisek P, Kalaska JF, (2005) Neural Correlates of Reaching Decisions in Dorsal Premotor Cortex: Specification of Multiple Direction Choices and Final Selection of Action. Neuron 45: 801-814 doi:10.1016/j.neuron.2005.01.027.
11. Sugrue LP, Corrado GS, Newsome WT, (2004) Matching behavior and the representation of value in the parietal cortex. Science 304: 1782-1787.
12. Gold JI, Shadlen MN, (2000) Representation of a perceptual decision in developing oculomotor commands. Nature 404: 390-394 doi:10.1038/35006062.
13. Platt ML, Glimcher PW, (1999) Neural correlates of decision variables in parietal cortex. Nature 400: 233-238.
14. Musallam S, (2004) Cognitive Control Signals for Neural Prosthetics. Science 305: 258-262 doi:10.1126/science.1097938.
15. Pastor-Bernier A, Cisek P, (2011) Neural correlates of biased competition in premotor cortex. J Neurosci 31: 7083-7088.
16. Scherberger H, Andersen RA, (2007) Target selection signals for arm reaching in the posterior parietal cortex. J Neurosci 27: 2001-2012.
17. Pesaran B, Nelson MJ, Andersen RA, (2008) Free choice activates a decision circuit between frontal and parietal cortex. Nature 453: 406-409.
18. Kable JW, Glimcher PW, (2009) The Neurobiology of Decision: Consensus and Controversy. Neuron 63: 733-745 doi:10.1016/j.neuron.2009.09.003.
19. Kiani R, Shadlen MN, (2009) Representation of Confidence Associated with a Decision by Neurons in the Parietal Cortex. Science 324: 759-764 doi:10.1126/science.1169405.
20. Erlhagen W, Schöner G, (2002) Dynamic field theory of movement preparation. Psychol Rev 109: 545.
21. Gold JI, Shadlen MN, (2007) The neural basis of decision making. Annu Rev Neurosci 30: 535-574.
22. Brozović M, Gail A, Andersen RA, (2007) Gain mechanisms for contextually guided visuomotor transformations. J Neurosci 27: 10588-10596.
23. Salinas E, (2004) Fast remapping of sensory stimuli onto motor actions on the basis of contextual modulation. J Neurosci 24: 1113-1118.
24. Fusi S, Asaad WF, Miller EK, Wang X-J, (2007) A Neural Circuit Model of Flexible Sensorimotor Mapping: Learning and Forgetting on Multiple Timescales. Neuron 54: 319-333 doi:10.1016/j.neuron.2007.03.017.
25. Gail A, Andersen RA, (2006) Neural dynamics in monkey parietal reach region reflect context-specific sensorimotor transformations. J Neurosci 26: 9376-9384.
26. Gail A, Klaes C, Westendorff S, (2009) Implementation of spatial transformation rules for goal-directed reaching via gain modulation in monkey parietal and premotor cortex. J Neurosci 29: 9490-9499.
27. Westendorff S, Klaes C, Gail A, (2010) The cortical timeline for deciding on reach motor goals. J Neurosci 30: 5426-5436.
28. Van Zoest W, Donk M, (2006) Saccadic target selection as a function of time. Spat Vis 19: 61-76.
29. Schneegans S, Schöner G (2008) Dynamic field theory as a framework for understanding embodied cognition. Handbook of cognitive science: An embodied approach. Amsterdam: Elsevier Science. p 241-271.
30. Averbeck BB, Latham PE, Pouget A, (2006) Neural correlations, population coding and computation. Nat Rev Neurosci 7: 358-366.
31. Decharms RC, Zador A, (2000) Neural representation and the cortical code. Annu Rev Neurosci 23: 613-647.
32. Wilson HR, Cowan JD, (1973) A mathematical theory of the functional dynamics of cortical and thalamic nervous tissue. Biol Cybern 13: 55-80.
33. Erlhagen W, Bastian A, Jancke D, Riehle A, Schöner G, (1999) The distribution of neuronal population activation (DPA) as a tool to study interaction and integration in cortical representations. J Neurosci Methods 94: 53-66.
34. Deco G, Jirsa VK, Robinson PA, Breakspear M, Friston K, (2008) The Dynamic Brain: From Spiking Neurons to Neural Masses and Cortical Fields. PLoS Comput Biol 4: e1000092 doi:10.1371/journal.pcbi.1000092.
35. Amari S, (1977) Dynamics of pattern formation in lateral-inhibition type neural fields. Biol Cybern 27: 77-87.
36. Wang X-J, (2001) Synaptic reverberation underlying mnemonic persistent activity. Trends Neurosci 24: 455-463 doi:10.1016/S0166-2236(00)01868-3.
37. Wang XJ, Tegnér J, Constantinidis C, Goldman-Rakic PS, (2004) Division of labor among distinct subtypes of inhibitory neurons in a cortical microcircuit of working memory. Proc Natl Acad Sci U S A 101: 1368.
38. Kobayashi Y, Isa T, (2002) Sensory-motor gating and cognitive control by the brainstem cholinergic system. Neural Netw 15: 731-741.
39. Suri RE, Schultz W, (1998) Learning of sequential movements by neural network model with dopamine-like reinforcement signal. Exp Brain Res 121: 350-354.
40. Vasilaki E, Fusi S, Wang X-J, Senn W, (2009) Learning flexible sensori-motor mappings in a complex network. Biol Cybern 100: 147-158 doi:10.1007/s00422-008-0288-z.
41. Stringer SM, Rolls ET, Taylor P, (2007) Learning movement sequences with a delayed reward signal in a hierarchical model of motor function. Neural Netw 20: 172-181.
42. Marshall JA, (1995) Adaptive perceptual pattern recognition by self-organizing neural networks: Context, uncertainty, multiplicity, and scale. Neural Netw 8: 335-362 doi:10.1016/0893-6080(94)00099-8.
43. Grossberg S, Williamson JR, (2001) A Neural Model of How Horizontal and Interlaminar Connections of Visual Cortex Develop into Adult Circuits That Carry Out Perceptual Grouping and Learning. Cereb Cortex 11: 37-58 doi:10.1093/cercor/11.1.37.
44. Kalarickal GJ, Marshall JA, (2002) Rearrangement of receptive field topography after intracortical and peripheral stimulation: the role of plasticity in inhibitory pathways. Network 13: 1-40.
45. Schultz W, (2001) Book Review: Reward Signaling by Dopamine Neurons. Neuroscientist 7: 293-302 doi:10.1177/107385840100700406.
46. Schultz W, Dayan P, Montague PR, (1997) A neural substrate of prediction and reward. Science 275: 1593-1599.
47. Körding KP, Wolpert DM, (2004) Bayesian integration in sensorimotor learning. Nature 427: 244-247 doi:10.1038/nature02169.
48. Mitz AR, Godschalk M, Wise SP, (1991) Learning-dependent neuronal activity in the premotor cortex: activity during the acquisition of conditional motor associations. J Neurosci 11: 1855-1872.
49. Chen LL, Wise SP, (1995) Neuronal activity in the supplementary eye field during acquisition of conditional oculomotor associations. J Neurophysiol 73: 1101-1121.
50. Chen LL, Wise SP, (1995) Supplementary eye field contrasted with the frontal eye field during acquisition of conditional oculomotor associations. J Neurophysiol 73: 1122-1134.
51. Chen LL, Wise SP, (1996) Evolution of directional preferences in the supplementary eye field during acquisition of conditional oculomotor associations. J Neurosci 16: 3067-3081.
52. Asaad WF, Rainer G, Miller EK, (1998) Neural activity in the primate prefrontal cortex during associative learning. Neuron 21: 1399-1407.
53. Glimcher PW, (2003) The Neurobiology of Visual-Saccadic Decision Making. Annu Rev Neurosci 26: 133-179 doi:10.1146/annurev.neuro.26.010302.081134.
54. Shadlen MN, Newsome WT, (2001) Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey. J Neurophysiol 86: 1916-1936.
55. Hanks TD, Ditterich J, Shadlen MN, (2006) Microstimulation of macaque area LIP affects decision-making in a motion discrimination task. Nat Neurosci 9: 682-689 doi:10.1038/nn1683.
56. Galletti C, Fattori P, Kutz DF, Gamberini M, (1999) Brain location and visual topography of cortical area V6A in the macaque monkey. Eur J Neurosci 11: 575-582 doi:10.1046/j.1460-9568.1999.00467.x.
57. Galletti C, Fattori P, Gamberini M, Kutz DF, (1999) The cortical visual area V6: brain location and visual topography. Eur J Neurosci 11: 3922-3936 doi:10.1046/j.1460-9568.1999.00817.x.
58. Wallis JD, Anderson KC, Miller EK, (2001) Single neurons in prefrontal cortex encode abstract rules. Nature 411: 953-956 doi:10.1038/35082081.
59. Wallis JD, Miller EK, (2003) Neuronal activity in primate dorsolateral and orbital prefrontal cortex during performance of a reward preference task. Eur J Neurosci 18: 2069-2081.
60. Georgopoulos A, Schwartz A, Kettner R, (1986) Neuronal population coding of movement direction. Science 233: 1416-1419 doi:10.1126/science.3749885.
61. Bastian A, Schöner G, Riehle A, (2003) Preshaping and continuous evolution of motor cortical representations during movement preparation. Eur J Neurosci 18: 2047-2058.
62. Fuster JM, (2001) The Prefrontal Cortex-An UpdateTime Is of the Essence. Neuron 30: 319-333.
63. Munoz DP, Everling S, (2004) Look away: the anti-saccade task and the voluntary control of eye movement. Nat Rev Neurosci 5: 218-228 doi:10.1038/nrn1345.
64. Alexander GE, Crutcher MD, (1990) Functional architecture of basal ganglia circuits: neural substrates of parallel processing. Trends Neurosci 13: 266-271.
65. Pasupathy A, Miller EK, (2005) Different time courses of learning-related activity in the prefrontal cortex and striatum. Nature 433: 873-876 doi:10.1038/nature03287.
66. Brasted PJ, Wise SP, (2004) Comparison of learning-related neuronal activity in the dorsal premotor cortex and striatum. Eur J Neurosci 19: 721-740.
67. Schultz W, Tremblay L, Hollerman JR, (2000) Reward Processing in Primate Orbitofrontal Cortex and Basal Ganglia. Cereb Cortex 10: 272-283 doi:10.1093/cercor/10.3.272.
68. Toni I, Passingham RE, (1999) Prefrontal-basal ganglia pathways are involved in the learning of arbitrary visuomotor associations: a PET study. Exp Brain Res 127: 19-32.
69. Buss A, Spencer JP (2008) The emergence of rule-use: A dynamic neural field model of the DCCS. Paper to appear in the Twenty-ninth Annual Conference of the Cognitive Science Society. Available:http://csjarchive.cogsci.rpi.edu/Proceedings/2008/pdfs/p463.pdf. Accessed 17 August 2012.
70. Urbanczik R, Senn W, (2009) Reinforcement learning in populations of spiking neurons. Nat Neurosci 12: 250-252 doi:10.1038/nn.2264.
71. Rigotti M, Rubin DBD, Wang XJ, Fusi S, (2010) Internal representation of task rules by recurrent dynamics: the importance of the diversity of neural responses. Front Comput Neurosci 4: 24.
72. Soltani A, Wang X-J, (2009) Synaptic computation underlying probabilistic inference. Nat Neurosci 13: 112-119 doi:10.1038/nn.2450.
73. Herrnstein RJ, Loveland DH, (1975) Maximizing and matching on concurrent ratio schedules. J Exp Anal Behav 24: 107.
74. Soltani A, Wang XJ, (2006) A biophysically based neural model of matching law behavior: melioration by stochastic synapses. J Neurosci 26: 3731-3744.
75. Barraclough DJ, Conroy ML, Lee D, (2004) Prefrontal cortex and decision making in a mixed-strategy game. Nat Neurosci 7: 404-410 doi:10.1038/nn1209.
76. Carpenter RHS, Williams MLL, (1995) Neural computation of log likelihood in control of saccadic eye movements. Nature 377: 59-62 doi:10.1038/377059a0.
77. Redding GM, Wallace B, (2000) Prism Exposure Aftereffects and Direct Effects for Different Movement and Feedback Times. J Mot Behav 32: 83-99 doi:10.1080/00222890009601362.
78. Sharma J, (2003) V1 Neurons Signal Acquisition of an Internal Representation of Stimulus Location. Science 300: 1758-1763 doi:10.1126/science.1081721.