A model of food reward learning with dynamic reward exposure

Frontiers in Computational Neuroscience

Volumn , Issue SEPTEMBER, 2012, Pages

  Hammond, Ross A ^a   Ornstein, Joseph T ^a   Fellows, Lesley ^b   Dube, Laurette ^c   Levitan, Robert ^d   Dagher, Alain ^b  

^a BROOKINGS INSTITUTION   (United States)

^b MCGILL UNIVERSITY   (Canada)

^c MCGILL UNIVERSITY   (Canada)

^d UNIVERSITY OF TORONTO   (Canada)

Author keywords

Computational modeling; Food choice; Reward learning; Temporal difference learning

Indexed keywords

COMPUTATIONAL MODELING; COMPUTATIONAL TECHNIQUE; COMPUTATIONAL WORK; ENVIRONMENTAL EXPOSURE; EXTENDED MODEL; INDIVIDUAL BEHAVIOR; LOCK-IN; REWARD LEARNING; STANDARD MODEL; TEMPORAL DIFFERENCE LEARNING;

MEDICAL COMPUTING; NEUROLOGY;

NUTRITION;

ARTICLE; ENVIRONMENTAL EXPOSURE; FEEDING BEHAVIOR; FOOD INTAKE; FOOD PREFERENCE; LEARNING; MATHEMATICAL COMPUTING; MATHEMATICAL MODEL; NEUROBIOLOGY; OBESITY; REWARD; SIMULATION; SOCIAL INTERACTION; TEMPORAL CORTEX; VALIDATION PROCESS;

EID: 84866752963     PISSN: 16625188     EISSN: None     Source Type: Journal    
DOI: 10.3389/fncom.2012.00082     Document Type: Article

References (55)

1. Auchincloss, A.H., Diez Roux, A.V. (2008). A new tool for epidemiology: the usefulness of dynamic-agent models in understanding place effects on health. Am J Epidemiol, 168, 1-8.
2. Axelrod, R. (2006). Simulation in the social sciences. In Rennard, J.P., Ed. Handbook of research on nature-inspired computing for economics and management. IGI Global. 391
3. Berridge, K.C., Robinson, T.E., Aldridge, J.W. (2009). Dissecting components of reward: 'liking', 'wanting', and learning. Current Opinion in Pharmacology, 9, 65-73.
4. Berthoud, H-R. and Morrison, C. (2008). 395. The brain, appetite, and obesity. Ann Rev Psych, 59, 55-92.
5. Blundell, J. E., Stubbs, R. J., Golding, C., Croden, F., Alam, R., Whybrow, S., Le Noury, J., Lawton, C. L. (2005). Resistance and susceptibility to weight gain: individual variability in response to a high-fat diet. Physiology & Behavior, 86:5, 614-622.
6. Caspi, C.E., Sorenson, G., Subramanian, S.V., Kawachi, I. (2012). The local food environment and diet: A systematic review. Health & Place, 18, 1172-1187.
7. Champagne, F., Meaney, M.J. (2001). Like mother, like daughter: evidence for non-genomic transmission of parental behavior and stress responsivity. Prog Brain Res, 133, 287-302.
8. Dagher, A. (2012). Functional brain imaging of appetite. Trends in endocrinology and metabolism, 23, 250-260.
9. Dalley, J.W., Laane, K., Theobald, D.E., Armstrong, H.C., Corlett, P.R., Chudasama, Y., Robbins, T.W. (2005). Time-limited modulation of appetitive Pavlovian memory by D1 and NMDA receptors in the nucleus accumbens. Proc Natl Acad Sci U S A, 102, 6189-6194.
10. Dalley, J.W., Fryer, T.D., Brichard, L., Robinson, E.S., Theobald, D.E., Laane, K., Pena, Y., Murphy, E.R., Shah, Y., Probst, K., Abakumova, I., Aigbirhio, F.I., Richards, H.K., Hong, Y., Baron, J.C., Everitt, B.J., Robbins, T.W. (2007). Nucleus accumbens D2/3 receptors predict trait impulsivity and cocaine reinforcement. Science, 315, 1267-1270.
11. Davis, C., Patte, K., Levitan, R., Reid, C., Tweed, S., Curtis, C. (2007). From motivation to behaviour: a model of reward sensitivity, overeating, and food preferences in the risk profile for obesity. Appetite, 48:1, 12-19.
12. Dube, L., Bechara, A., Dagher, A., Drewnowski, A., LeBel, J., James, P., Yada, R. Y, Eds. (2010). Obesity Prevention: The Role of Brain and Society on Individual Behavior. Elsevier.
13. Epstein, L. H., Leddy, J. J., Temple, J. L., and Faith, M. S. (2007). Food reinforcement and eating: a multilevel analysis. Psychological Bulletin, 133:5, 884-906.
14. Flagel, S.B., Watson, S.J., Akil, H., Robinson, T.E. (2008). Individual differences in the attribution of incentive salience to a reward-related cue: Influence on cocaine sensitization. Behavioural brain research, 186, 48-56.
15. Flagel, 434 S.B., Akil, H., Robinson, T.E. (2009). Individual differences in the attribution of incentive salience to reward-related cues: Implications for addiction. Neuropharmacology, 56: Suppl 1, 139-148.
16. Frank, M. J. and Claus, E. D. (2006). Anatomy of a decision: striato-orbitofrontal interactions in reinforcement learning, decision making and reversal. Psychological Review, 113:2, 300-326.
17. Gillman, M.W. (2005). Developmental origins of health and disease. New England Journal of Medicine, 353:17, 1848-1850.
18. Grigson, P. S. (2002). Like drugs for chocolate: separate rewards modulated by common mechanisms? Physiology & Behavior, 76:3, 389-395.
19. Guerrieri, R., Nederkoorn, C., and Jansen, A. (2008). The effect of impulsive personality on overeating and obesity: current state of affairs. Psych Top, 17:2, 265-286.
20. Hammond, R. A., Axelrod, R. (2006). The evolution of ethnocentrism. J Conflict Resolution, 50:6, 926-936.
21. Hammond, R. A. (2009). Complex systems modeling for obesity research. Prev Chronic Dis, 6:3, A97.
22. Hammond, R.A., and Dube, L. (2012). A systems science perspective and transdisciplinary models for food and nutrition security. Proc Natl Acad Sci U S A, 109:31, 12356-12363.
23. Herman, C.P., and Polivy, J. (2008) External cues in the control of food intake in humans: the sensory-normative distinction. Physiol Behav 94:722-728.
24. Huang, T. T.-K. and Glass, T. A. (2008). Transforming research strategies for understanding and preventing obesity. JAMA, 300:15, 1811-1813.
25. Huang, T. T.-K., Drewnowski, A., Kumanyika, S.K., Glass, T.A. (2009). A systems-oriented multilevel framework for addressing obesity in the 21st century. Prev Chron Dis, 6:3, A82
26. IOM (Institute of Medicine). (2010). Bridging the Evidence Gap in Obesity Prevention: A Framework to Inform Decision Making. Washington, DC: The National Academies Press.
27. IOM (Institute of Medicine). (2012). Accelerating Progress in Obesity Prevention: Solving the Weight of the Nation. Washington, DC: The National Academies Press.
28. Lakdawalla, 474 D. and Philipson, T. (2009). The growth of obesity and technological change. Economics and Human Biology, 7:3, 283-293.
29. Larson, N.I., Story, M.T., Nelson, M.C. (2009). Neighborhood environments: disparities in access to healthy foods in the U.S. American Journal of Preventive Medicine, 36:1, 74-81.
30. Levy, D.T., Mabry, P.L., Wang, Y.C., Gortmaker, S., Huang, T. T.-K., Marsh, T., Moodie, M., Swinburn, B. (2011). Simulation models of obesity: a review of the literature and implications for research and policy. Obes Rev, 12:5, 378-394.
31. Lovic, V., Saunders, B.T., Yager, L.M., Robinson, T.E. (2011). Rats prone to attribute incentive salience to reward cues are also prone to impulsive action. Behavioural brain research, 223, 255-261.
32. Lowe, M. R. and Butryn, M. L. (2007). Hedonic hunger: a new dimension of appetite? Physiology & Behavior, 91:4: 432-439.
33. Mabry, P.L., Olster, D.H., Morgan, G.D., Abrams, D.B. (2008). Interdisciplinarity and systems science to improve population health: a view from the NIH Office of Behavioral and Social Sciences Research. Am J Prev Med, 35:S2, 11-24.
34. Mabry, P.L., Marcus, S.E., Clark, P.I., Leischow, S.J., Mendez, D. (2010). Systems science: A revolution in public health policy research. Am J Pub Health, 100, 1161-1163.
35. McMillen, C. and Robinson, J.S. (2005). Developmental origins of the metabolic syndrome: prediction, plasticity, and programming. Physiological Reviews, 85:2, 571-633.
36. Mela, D. J. and Saccheti, D. A. (1991). Sensory preferences for fats: relationships with diet and body composition. Am J Clin Nutr, 53, 908-915.
37. Montague, P. R., Dayan, P., and Sejnowski, T. J. (1996). A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J Neuroscience, 16:5, 1936-1947.
38. Morland, K., Diez Roux, A.V., Wing, S. (2006). Supermarkets, other food stores, and obesity: the atherosclerosis risk in communities study. American Journal of Preventive Medicine, 30:4, 333-339.
39. Murakamia, K., Sasakia, S., Takahashib, Y., Uenishi, K. (2010). No meaningful association of neighborhood food store availability with dietary intake, body mass index, or waist circumference in young Japanese women. Nutrition Research, 30:8, 565-573.
40. Nader, 515 P.R., Huang, T.T.-K., Gahagan, S., Kumanyika, S., Hammond, R.A., Christoffel, K.K. (2012). Next steps in obesity prevention: altering early life systems to support healthy parents, infants, and toddlers. Childhood Obesity, 8:3, 195-204.
41. Ng, A.Y., Coates, A., Diel, M., Ganapathi, V., Schulte, J., Tse, B., Berger, E., Liang, E. (2004). Inverted autonomous helicopter flight via reinforcement learning. International Symposium on Experimental Robotics.
42. O'Doherty, J. P, Dayan, P., Friston, K., Critchley, H., Dolan, R.J. (2003). Temporal difference models and reward-related learning in the human brain. Neuron, 28, 329-337.
43. Page, S.E. (1999). Computational models from A to Z. Complexity, 5:1, 35-40.
44. Petrovich, G. D. and Gallagher, M. (2007). Control of food consumption by learned cues: a forebrain-hypothalamic network. Physiol Behav, 91:4, 397-403.
45. Robinson, T. E. and Berridge, K. C. (1993). The neural basis of drug craving: an incentive532 sensitization theory of addiction. Brain Research Reviews, 18:3, 247-291.
46. Rothemund, Y., Preuschhof, C., Bohner, Georg, Bauknecht, H., Klingebiel, R., Flor, H., Klapp, B. F. (2007). Differential activation of the dorsal striatum by high-calorie visual food stimuli in obese individuals. NeuroImage, 37:2, 410-421.
47. Samson, R. D., Frank, M. J., and Fellous, J.M. (2010). Computational models of reinforcement learnining: the role of dopamine as a reward signal. Cognitive Neurodynamics, 4, 91-105.
48. Schultz, W., Dayan, P., and Montague, P. R. (1997). A neural substrate of prediction and reward. Science, 275, 1593-1599.
49. Schultz, W. (1998). Predictive reward signal of dopamine neurons. J Neurophysiology, 80:1, 1-27.
50. Sclafani, A., Touzani, K., Bodnar, R. J. (2011). Dopamine and learned food preferences. Physiology & Behavior, 104:1, 64-68.
51. Sutton, R. S., Barto, A. G. (1998). Reinforcement Learning. Cambridge, MA: MIT Press.
52. Tesauro, G. (1992). Practical issues in temporal difference learning. Machine Learning, 8, 257-277.
53. Tesfatsion, 555 L., Judd, K.J., Eds. (2006). Handbook of computational economics, vol 2. Agent556 based computational economics. Amsterdam (NL): North-Holland.
54. Whiteson, S., Taylor, M.E., Stone, P. (2010). Critical factors in the empirical performance of temporal difference and evolutionary methods for reinforcement learning. Autonomous Agents and Multi-agent Systems, 21:1, 1-35.
55. Yager, L.M., Robinson, T.E. (2010). Cue-induced reinstatement of food seeking in rats that differ in their propensity to attribute incentive salience to food cues. Behavioural brain research, 214, 30-34.