A computational framework for understanding decision making through integration of basic learning rules

Journal of Neuroscience

Volumn 33, Issue 13, 2013, Pages 5686-5697

  Bazhenov, Maxim ^a   Huerta, Ramon ^b   Smith, Brian H ^c  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b UNIVERSITY OF CALIFORNIA SAN DIEGO   (United States)

^c ARIZONA STATE UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ACCURACY; ARTICLE; BEHAVIOR; CONTROLLED STUDY; DECISION MAKING; DEFENSE MECHANISM; DISCRIMINATION LEARNING; FEMALE; HONEYBEE; LATENT INHIBITION; LEARNING; NERVE CELL; NERVE CELL PLASTICITY; NONHUMAN; OLFACTORY DISCRIMINATION; OLFACTORY MEMORY; OLFACTORY SYSTEM; PERFORMANCE; PRIORITY JOURNAL; RECALL; SENSOR; STATE DEPENDENT LEARNING;

ANIMALS; BEES; COMPUTER SIMULATION; CONDITIONING (PSYCHOLOGY); DECISION MAKING; DISCRIMINATION (PSYCHOLOGY); MENTAL RECALL; MODELS, BIOLOGICAL; NEURAL INHIBITION; NEURONS; ODORS; OLFACTORY PATHWAYS;

EID: 84875470293     PISSN: 02706474     EISSN: 15292401     Source Type: Journal    
DOI: 10.1523/JNEUROSCI.4145-12.2013     Document Type: Article

References (80)

1. Abel R, Rybak J, Menzel R (2001) Structure and response patterns of olfactory interneurons in the honeybee, Apis mellifera. J Comp Neurol 437: 363-383. CrossRef Medline
2. Ament SA, Wang Y, Robinson GE (2010) Nutritional regulation of division of labor in honeybees: toward a systems biology perspective. Wiley Interdiscip Rev Syst Biol Med 2:566-576. CrossRef Medline
3. Bazhenov M, Stopfer M, Rabinovich M, Huerta R, Abarbanel HD, Sejnowski TJ, Laurent G (2001) Model of transient oscillatory synchronization in the locust antennal lobe. Neuron 30:553-567.
4. Bishop MC (2006) Pattern recognition and machine learning (information science and statistics). New York: Springer.
5. Bitterman ME, Menzel R, Fietz A, Schäfer S (1983) Classical conditioning of proboscis extension in honeybees (Apis mellifera). J Comp Psychol 97: 107-119. CrossRef Medline
6. Brennan PA, Keverne EB (1997) Neural mechanisms of mammalian olfactory learning. Prog Neurobiol 51:457-481. CrossRef Medline
7. Burrell BD, Li Q (2008) Co-induction of long-term potentiation and longterm depression at a central synapse in the leech. Neurobiol Learn Mem 90:275-279. CrossRef Medline
8. Cassenaer S, Laurent G (2007) Hebbian STDP in mushroom bodies facilitates the synchronous flow of olfactory information in locusts. Nature 448:709-713. CrossRef Medline
9. Cassenaer S, Laurent G (2012) Conditional modulation of spike-timingdependent plasticity for olfactory learning. Nature 482:47-52. CrossRef Medline
10. Chandra SB, Wright GA, Smith BH (2010) Latent inhibition in the in the honeybee, Apis mellifera: is it a unitary phenomenon? Anim Cogn 13: 805-815. CrossRef Medline
11. Dacher M, Smith BH (2008) Olfactory interference during inhibitory backward pairing in honeybees. PloS One 3:e3513. CrossRef Medline
12. Dehaene S, Changeux JP (2000) Reward-dependent learning in neuronal networks for planning and decision making. Prog Brain Res 126:217-229. CrossRef Medline
13. Drezner-Levy T, Shafir S (2007) Parameters of variable reward distributions that affect risk sensitivity of honeybees. J Exp Biol 210:269-277. CrossRef Medline
14. Faber T, Joerges J, Menzel R (1999) Associative learning modifies neural representations of odors in the insect brain. Nat Neurosci 2:74-78. CrossRef Medline
15. Farooqui T, Robinson K, Vaessin H, Smith BH (2003) Modulation of early olfactory processing by an octopaminergic reinforcement pathway in the honeybee. J Neurosci 23:5370-5380. Medline
16. Fernandez PC, Locatelli FF, Person-Rennell N, Deleo G, Smith BH (2009) Associative conditioning tunes transient dynamics of early olfactory processing. J Neurosci 29:10191-10202. CrossRef Medline
17. Frey U, Morris RG (1997) Synaptic tagging and long-term potentiation. Nature 385:533-536. CrossRef Medline
18. Gallistel CR, Fairhurst S, Balsam P (2004) The learning curve: implications of a quantitative analysis. Proc Natl Acad Sci U S A 101:13124-13131. CrossRef Medline
19. García-Sanchez M, Huerta R (2003) Design parameters of the fan-out phase of sensory systems. J Comput Neurosci 15:5-17. CrossRef Medline
20. Hammer M (1993) An identified neuron mediates the unconditioned stimulus in associative olfactory learning in honeybees. Nature 366:59-63. CrossRef
21. Hammer M (1997) The neural basis of associative reward learning in honeybees. Trends Neurosci 20:245-252. CrossRef Medline
22. Hebb DO (1949) The organization of behavior. New York: Wiley
23. Heisenberg M (2003) Mushroom body memoir: from maps to models. Nat Rev Neurosci 4:266-275. CrossRef Medline
24. Hildebrand JG, Shepherd GM (1997) Mechanisms of olfactory discrimination: converging evidence for common principles across phyla. Annu Rev Neurosci 20:595-631. CrossRef Medline
25. Houk JC, Adams JL, and Barto AG (1995) A model of how the basal ganglia generates and uses neural signals that predict reinforcement. In: Models of information processing in the basal ganglia (Houk J, Davis J, Beiser D, eds), pp 249-270. Cambridge, MA: MIT.
26. Hripcsak G, Rothschild AS (2005) Agreement, the f-measure, and reliability in information retrieval. JAMIA 12:296-298. Medline
27. Huerta R (2013) Learning pattern recognition and decision making in the insect brain. AIP Conference Proceedings 1510:101.
28. Huerta R, Nowotny T (2009) Fast and robust learning by reinforcement signals: explorations in the insect brain. Neural Comput 21:2123-2151. CrossRef Medline
29. Huerta R, Nowotny T, García-Sanchez M, Abarbanel HD, Rabinovich MI (2004) Learning classification in the olfactory system of insects. Neural Comput 16:1601-1640. CrossRef Medline
30. Huerta R, Vembu S, Amigó JM, Nowotny T, Elkan C (2012) Inhibition in multiclass classification. Neural Comput 24:2473-2507. CrossRef Medline
31. Izhikevich EM (2007) Solving the distal reward problem through linkage of STDP and dopamine signaling. Cereb Cortex 17:2443-2452. Medline
32. Kirschner S, Kleineidam CJ, Zube C, Rybak J, Grünewald B, Rössler W (2006) Dual olfactory pathway in the honeybee, Apis mellifera. J Comp Neurol 499:933-952. CrossRef Medline
33. Linster C, Menon AV, Singh CY, Wilson DA (2009) Odor-specific habituation arises from interaction of afferent synaptic adaptation and intrinsic synaptic potentiation in olfactory cortex. Learn Mem 16:452-459. CrossRef Medline
34. Locatelli FF, Fernandez PC, Villareal F, Muezzinoglu K, Huerta R, Galizia CG, Smith BH (2013) Nonassociative plasticity alters competitive interactions among mixture components in early olfactory processing. Eur J Neurosci 37:63-79. CrossRef Medline
35. Lubow RE (1973) Latent inhibition. Psychol Bull 79:398-407. CrossRef Medline
36. Malenka RC, Bear MF (2004) LTP and LTD: an embarrassment of riches. Neuron 44:5-21. CrossRef Medline
37. Maleszka J, Barron AB, Helliwell PG, Maleszka R (2009) Effect of age, behaviour and social environment on honeybee brain plasticity. J Comp Physiol A Neuroethol Sens Neural Behav Physiol 195:733-740. CrossRef Medline
38. Mauelshagen J (1993) Neural correlates of olfactory learning paradigms in an identified neuron in the honeybee brain. J Neurophysiol 69:609-625. Medline
39. Mazor O, Laurent G (2005) Transient dynamics versus fixed points in odor representations by locust antennal lobe projection neurons. Neuron 48: 661-673. CrossRef Medline
40. McCulloch WS, Pitts W (1990) A logical calculus of the ideas immanent in nervous activity. 1943. Bull Math Biol 52:99-115; discussion 73-97. Medline
41. Menzel R (1990) Learning, memory, and 'cognition' in honeybees. In: Neurobiology of comparative cognition (Kesner RP, Olton DS, eds), pp 237-292. Hillsdale, N.J.: Lawrence Erlbaum.
42. Menzel R, Manz G (2005) Neural plasticity of mushroom body-extrinsic neurons in the honeybee brain. J Exp Biol 208:4317-4332. CrossRef Medline
43. Mobbs P (1982) The brain of the honeybee Apis mellifera I. The connections and spatial organization of the mushroom bodies. Philos Trans R Soc Lond B 298:309-354. CrossRef
44. Muezzinoglu MK, Vergara A, Huerta R, Nowotny T, Rulkov N, Abarbanel HDI, Selverston AI, Rabinovich MI (2008) Artificial olfactory brain for mixture identification. NIPS:1121-1128.
45. Muezzinoglu MK, Huerta R, Abarbanel HD, Ryan MA, Rabinovich MI (2009a) Chemosensor-driven artificial antennal lobe transient dynamics enable fast recognition and working memory. Neural Comput 21:1018-1037. CrossRef Medline
46. Muezzinoglu MK, Vergara A, Huerta R, Rulkov N, Rabinovich MI, Selverston A, Abarbanel HDI (2009b) Acceleration of chemo-sensory information processing using transient features. Sensors Actuat B Chem 137:507-512. CrossRef
47. Müller U (2002) Learning in honeybees: from molecules to behaviour. Zoology 105:313-320. CrossRef Medline
48. Nowotny T, Huerta R (2012) On the equivalence of Hebbian learning and the SVM formalism. Information Sciences and Systems (CISS), 2012 46th Annual Conference on March 21-23, 2012, pp 1-4.
49. Nowotny T, Huerta R, Abarbanel HD, Rabinovich MI (2005) Selforganization in the olfactory system: one shot odor recognition in insects. Biol Cybern 93:436-446. CrossRef Medline
50. Okada R, Rybak J, Manz G, Menzel R (2007) Learning-related plasticity in PE1 and other mushroom body-extrinsic neurons in the honeybee brain. J Neurosci 27:11736-11747. CrossRef Medline
51. Olson DL, Delen D (2008) Advanced data mining techniques: New York: Springer.
52. Pearce JM, Hall G (1980) A model for Pavlovian learning: variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychological review 87:532-552. CrossRef Medline
53. Perez-Orive J, Mazor O, Turner GC, Cassenaer S, Wilson RI, Laurent G (2002) Oscillations and sparsening of odor representations in the mushroom body. Science 297:359-365. CrossRef Medline
54. Perez-Orive J, Bazhenov M, Laurent G (2004) Intrinsic and circuit properties favor coincidence detection for decoding oscillatory input. J Neurosci 24:6037-6047.
55. Rescorla RA (1969) Conditioned inhibition of fear resulting from negative CS-US contingencies. J Comp Physiol Psychol 67:504-509. CrossRef Medline
56. Rescorla RA (1988) Behavioral studies of Pavlovian conditioning. Annu Rev Neurosci 11:329-352. CrossRef Medline
57. Rescorla RA, Wagner AR (1972) A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement. In: Classical conditioning II: current research and theory (Black AH, Prokasy WF, eds), pp. 64-99. New York: Appleton-Century-Crofts.
58. Rock I, Steinfeld G (1963) Methodological questions in the study of onetrial learning. Science 140:822-824. CrossRef Medline
59. Rössler W, Zube C (2011) Dual olfactory pathway in Hymenoptera: evolutionary insights from comparative studies. Arthropod Struct Dev 40:349-357. CrossRef Medline
60. Rybak J, Menzel R (1993) Anatomy of the mushroom bodies in the honey beeb rain: The neuronal connections of the alpha-lobe. J Comp Neurol 334:444-465. CrossRef Medline
61. Schröter U, Malun D, Menzel R (2007) Innervation pattern of suboesophageal ventral unpaired median neurones in the honeybee brain. Cell Tissue Res 327:647-667. CrossRef Medline
62. Schultz W (2010) Multiple functions of dopamine neurons. F1000 Biol Rep 2: pii:2. CrossRef Medline
63. Sinakevitch I, Strausfeld NJ (2006) Comparison of octopamine-like immunoreactivity in the brains of the fruit fly and blow fly. J Comp Neurol 494:460-475. CrossRef Medline
64. Sinakevitch I, Grau Y, Strausfeld NJ, Birman S (2010) Dynamics of glutamatergic signaling in the mushroom body of young adult Drosophila. Neural Dev 5:10. CrossRef Medline
65. Sinakevitch I, Mustard JA, Smith BH (2011) Distribution of the octopamine receptor AmOA1 in the honeybee brain. PloS One 6:e14536. CrossRef Medline
66. Smith BH, Abramson CI, Tobin TR (1991) Conditional withholding of proboscis extension in honeybees (Apis mellifera) during discriminative punishment. J Comp Psychol 105:345-356. CrossRef Medline
67. Smith BH, Wright GA, Daly KS (2006) Learning-based recognition and discrimination of floral odors. In: The biology of floral scents (Dudareva N, Pichersky E, eds), pp 263-295. Boca Raton, FL: CRC.
68. Stopfer M, Laurent G (2000) Analysis of odor concentration coding in the locust antennal lobe. Soc Neurosci Abstr 28:451.6.
69. Stopfer M, Bhagavan S, Smith BH, Laurent G (1997) Impaired odour discrimination on desynchronization of odour-encoding neural assemblies. Nature 390:70-74. CrossRef Medline
70. Strausfeld NJ (2002) Organization of the honeybee mushroom body: representation of the calyx within the vertical and gamma lobes. J Comp Neurol 450:4-33. CrossRef Medline
71. Strausfeld NJ, Sinakevitch I, Brown SM, Farris SM (2009) Ground plan of the insect mushroom body: functional and evolutionary implications. J Comp Neurol 513:265-291. CrossRef Medline
72. Strube-Bloss MF, Nawrot MP, Menzel R (2011) Mushroom body output neurons encode odor-reward associations. J Neurosci 31:3129-3140. CrossRef Medline
73. Strube-Bloss MF, Herrera-Valdez MA, Smith BH (2012) Ensemble response in mushroom body output neurons of the honeybee outpaces spatiotemporal odor processing two synapses earlier in the antennal lobe. PLoS One 7:e50322. CrossRef Medline
74. Szyszka P, Ditzen M, Galkin A, Galizia CG, Menzel R (2005) Sparsening and temporal sharpening of olfactory representations in the honeybee mushroom bodies. J Neurophysiol 94:3303-3313. CrossRef Medline
75. Szyszka P, Galkin A, Menzel R (2008) Associative and non-associative plasticity in kenyon cells of the honeybee mushroom body. Front Syst Neurosci 2:3. Medline
76. Van Slyke Peeke H, Petrinovich LF (1984) Habituation, sensitization, and behavior (behavioral biology): San Diego, CA: Academic.
77. Vergara A, Vembu S, Ayhan T, Ryan MA, Homer ML, Huerta R (2012) Chemical gas sensor drift compensation using classifier ensembles. Sensors Actuat B Chem 166:320-329. CrossRef
78. Vergara A, Muezzinoglu MK, Rulkov N, Huerta R (2010) Informationtheoretic optimization of chemical sensors. Sensor Actuat B Chem 148: 298-306. CrossRef
79. Wilson DA, Linster C (2008) Neurobiology of a simple memory. J Neurophysiol 100:2-7. CrossRef Medline
80. Withers GS, Fahrbach SE, Robinson GE (1995) Effects of experience and juvenile hormone on the organization of the mushroom bodies of honeybees. J Neurobiol 26:130-144. CrossRef Medline