Optimal inference with suboptimal models: Addiction and active Bayesian inference

Medical Hypotheses

Volumn 84, Issue 2, 2015, Pages 109-117

  Schwartenbeck, Philipp ^a,b,c   FitzGerald, Thomas H B ^a   Mathys, Christoph ^a   Dolan, Ray ^a   Wurst, Friedrich ^c   Kronbichler, Martin ^b,c   Friston, Karl ^a  

^a UNIVERSITY COLLEGE LONDON   (United Kingdom)

^b UNIVERSITY OF SALZBURG   (Austria)

^c PARACELSUS MEDICAL UNIVERSITY   (Austria)

Author keywords

[No Author keywords available]

Indexed keywords

ADDICTION; ARTICLE; BAYES THEOREM; BAYESIAN INFERENCE; BEHAVIOR ASSESSMENT; DECISION MAKING; HYPOTHESIS; MENTAL DISEASE; SIMULATION; COGNITION; COMPUTER SIMULATION; HUMAN; PHYSIOLOGY; PSYCHOLOGICAL MODEL; PSYCHOLOGY;

BAYES THEOREM; BEHAVIOR, ADDICTIVE; CHOICE BEHAVIOR; COGNITION; COMPUTER SIMULATION; DECISION MAKING; HUMANS; MODELS, PSYCHOLOGICAL;

EID: 84921063723     PISSN: 03069877     EISSN: 15322777     Source Type: Journal    
DOI: 10.1016/j.mehy.2014.12.007     Document Type: Article

References (83)

1. Adams R., Perrinet L.U., Friston K. Smooth pursuit and visual occlusion: active inference and oculomotor control in schizophrenia. PLoS ONE 2012, 7(10):e47502. 10.1371/journal.pone.0047502.
2. Adams R., Stephan K.E., Brown H.R., Frith C.D., Friston K.J. The computational anatomy of psychosis. Front. Psychiatry 2013, 4:47. 10.3389/fpsyt.2013.00047.
3. Bechara A. Decision making, impulse control and loss of willpower to resist drugs: a neurocognitive perspective. Nat. Neurosci. 2005, 8(11):1458-1463. 10.1038/nn1584.
4. Beck J.M., Ma W.J., Kiani R., Hanks T., Churchland A.K., Roitman J., et al. Probabilistic population codes for Bayesian decision making. Neuron 2008, 60(6):1142-1152. 10.1016/j.neuron.2008.09.021.
5. Beck J.M., Ma W.J., Pitkow X., Latham P.E., Pouget A. Not noisy, just wrong: the role of suboptimal inference in behavioral variability. Neuron 2012, 74(1):30-39. 10.1016/j.neuron.2012.03.016.
6. Belin D., Mar A.C., Dalley J.W., Robbins T.W., Everitt B.J. High impulsivity predicts the switch to compulsive cocaine-taking. Science 2008, 320(5881):1352-1355. 10.1126/science.1158136.
7. Berke J.D., Hyman S.E. Addiction, dopamine, and the molecular mechanisms of memory. Neuron 2000, 25(3):515-532.
8. Berridge K.C. The debate over dopamine's role in reward: the case for incentive salience. Psychopharmacology (Berl) 2007, 191(3):391-431. 10.1007/s00213-006-0578-x.
9. Berridge K.C. From prediction error to incentive salience: mesolimbic computation of reward motivation. Eur J Neurosci 2012, 35(7):1124-1143. 10.1111/j.1460-9568.2012.07990.x.
10. Bickel W.K., Miller M.L., Yi R., Kowal B.P., Lindquist D.M., Pitcock J.A. Behavioral and neuroeconomics of drug addiction: competing neural systems and temporal discounting processes. Drug Alcohol Depend 2007, 90:585-591. 10.1016/j.drugalcdep.2006.09.016.
11. Bishop C. Pattern recognition and machine learning 2006, Springer.
12. Blum K., Braverman E.R., Holder J.M., Lubar J.F., Monastra V.J., Miller D., et al. Reward deficiency syndrome: a biogenetic model for the diagnosis and treatment of impulsive, addictive, and compulsive behaviors. J Psychoact Drugs 2000, 32:1-112.
13. Blum K., Sheridan P.J., Wood R.C., Braverman E.R., Chen T.J., Comings D.E. Dopamine D2 receptor gene variants: association and linkage studies in impulsive-addictive-compulsive behaviour. Pharmacogenetics 1995, 5(3):121-141.
14. Blum K., Wood R.C., Braverman E.R., Chen T.J., Sheridan P.J. The D2 dopamine receptor gene as a predictor of compulsive disease: Bayes' theorem. Funct Neurol 1995, 10(1):37-44.
15. Bowers J.S., Davis C.J. More varieties of Bayesian theories, but no enlightenment. Behav Brain Sci 2011, 34(04):193-194. 10.1017/S0140525X11000227.
16. Bowers J.S., Davis C.J. Bayesian just-so stories in psychology and neuroscience. Psychol Bull 2012, 138(3):389-414. 10.1037/a0026450.
17. Brighton H., Gigerenzer G. Bayesian brains and cognitive mechanisms: harmony or dissonance?. The probabilistic mind: prospects for a Bayesian cognitive science 2008, 189-208. Oxford University Press, New York. N. Chater, M. Oaksford (Eds.).
18. Cain M.S., Vul E., Clark K., Mitroff S.R. A bayesian optimal foraging model of human visual search. Psychol Sci 2012, 23(9):1047-1054. 10.1177/0956797612440460.
19. Charness G., Levin D. When optimal choices feel wrong: a laboratory study of and affect Bayesian updating, complexity, and affect. Am Econ Rev 2014, 95(4):1300-1309.
20. Clark L. Decision-making during gambling: an integration of cognitive and psychobiological approaches. Philos Trans R Soc London, Ser B 2010, 365(1538):319-330. 10.1098/rstb.2009.0147.
21. Clark L., Limbrick-Oldfield E.H. Disordered gambling: a behavioral addiction. Curr Opin Neurobiol 2013, 23(4):655-659. 10.1016/j.conb.2013.01.004.
22. Comings D.E., Blum K. Reward deficiency syndrome: genetic aspects of behavioral disorders. Prog Brain Res 2000, 126:325-341. 10.1016/S0079-6123(00)26022-6.
23. Conant R.C., Ashby R.W. Every good regulator of a system must be a model of that system. Int J Syst Sci 1970, 1(2):89-97. 10.1080/00207727008920220.
24. Daunizeau J., den Ouden H.E.M., Pessiglione M., Kiebel S.J., Stephan K.E., Friston K.J. Observing the observer (I): meta-bayesian models of learning and decision-making. PLoS ONE 2010, 5(12):e15554. 10.1371/journal.pone.0015554.
25. Daw N.D., Doherty J.P.O., Dayan P., Seymour B., Dolan R.J. Cortical substrates for exploratory decisions in humans. Nature 2009, 441(7095):876-879. 10.1038/nature04766.Cortical.
26. Dayan P. Rationalizable irrationalities of choice. Top Cognit Sci 2014, 1-25. 10.1111/tops.12082.
27. Dayan P., Hinton G.E., Neal R.M., Zemel R.S. The Helmholtz machine. Neur Comp 1995, 7(5):889-904.
28. Diana M. The dopamine hypothesis of drug addiction and its potential therapeutic value. Front Psychiatry 2011, 2:64. 10.3389/fpsyt.2011.00064.
29. Dolan R.J., Dayan P. Goals and habits in the brain. Neuron 2013, 80(2):312-325. 10.1016/j.neuron.2013.09.007.
30. Doyon W.M., Dong Y., Ostroumov A., Thomas A.M., Zhang T., Dani J. Nicotine decreases ethanol-induced dopamine signaling and increases self-administration via stress hormones. Neuron 2013, 79(3):530-540. 10.1016/j.neuron.2013.06.006.
31. Edwards M., Adams R., Brown H., Pareés I., Friston K. A Bayesian account of "hysteria". Brain 2012, 135(11):3495-3512. 10.1093/brain/aws129.
32. Everitt B.J., Belin D., Economidou D., Pelloux Y., Dalley J.W., Robbins T.W. Review. Neural mechanisms underlying the vulnerability to develop compulsive drug-seeking habits and addiction. Philos Trans R Soc London Ser B 2008, 363(1507):3125-3135. 10.1098/rstb.2008.0089.
33. Everitt B.J., Robbins T.W. Neural systems of reinforcement for drug addiction: from actions to habits to compulsion. Nat Neurosci 2005, 8(11):1481-1489. 10.1038/nn1579.
34. Eysenck M., Derakshan N., Santos R., Calvo M. Anxiety and cognitive performance: attentional control theory. Emotion 2007, 7(2):336-353. 10.1037/1528-3542.7.2.336.
35. FitzGerald T.H.B., Dolan R., Friston K. Model averaging, optimal inference and model averaging. Front Hum Neurosci 2014, 8:457. 10.3389/fnhum.2014.00457.
36. Friston K. A theory of cortical responses. Philos Trans R Soc London Ser B 2005, 360(1456):815-836. 10.1098/rstb.2005.1622.
37. Friston K. The free-energy principle: a rough guide to the brain?. Trends Cognit Sci 2009, 13(7):293-301. 10.1016/j.tics.2009.04.005.
38. Friston K. A free energy principle for biological systems. Entropy 2012, 14(11):2100-2121. 10.3390/e14112100.
39. Friston K.J., Dolan R.J. Computational and dynamic models in neuroimaging. NeuroImage 2010, 52(3):752-765. 10.1016/j.neuroimage.2009.12.068.
40. Friston K., Schwartenbeck P., FitzGerald T., Moutoussis M., Behrens T., Dolan R.J. The anatomy of choice: active inference and agency. Front Hum Neurosci 2013, 7:598. 10.3389/fnhum.2013.00598.
41. Friston K., Stephan K.E., Montague R., Dolan R.J. Computational psychiatry: the brain as a phantastic organ. Lancet Psychiatry 2014, 1(2):148-158. 10.1016/S2215-0366(14)70275-5.
42. Geisler W.S., Kersten D. Illusions, perception and Bayes. Nat Neurosci 2002, 5(6):508-510. 10.1038/nn0602-508.
43. Gigerenzer G., Gaissmaier W. Heuristic decision making. Annu Rev Psychol 2011, 62:451-482. 10.1146/annurev-psych-120709-145346.
44. Gillan C.M., Papmeyer M., Morein-Zamir S., Sahakian B.J., Fineberg N.A., Robbins T.W., et al. Disruption in the balance between goal-directed behavior and habit learning in obsessive-compulsive disorder. Am J Psychiatry 2011, 168(7):718-726. 10.1176/appi.ajp.2011.10071062.
45. Graybiel A.M. Habits, rituals, and the evaluative brain. Annu Rev Neurosci 2008, 31:359-387. 10.1146/annurev.neuro.29.051605.112851.
46. Grether D.M. Testing Bayes rule and the representativeness heuristic: some experimental evidence. J Econ Behav Organiz 1992, 17(1):31-57. 10.1016/0167-2681(92)90078-P.
47. Griffiths T.L., Tenenbaum J.B. Optimal predictions in everyday cognition. Psychol Sci 2006, 17(9):767-773. 10.1111/j.1467-9280.2006.01780.x.
48. Griffiths T.L., Tenenbaum J.B. Predicting the future as Bayesian inference: people combine prior knowledge with observations when estimating duration and extent. J Exp Psychol Gen 2011, 140(4):725-743. 10.1037/a0024899.
49. Henden E., Melberg H.O., Røgeberg O.J. Addiction: choice or compulsion?. Front Psychiatry 2013, 4:77. 10.3389/fpsyt.2013.00077.
50. Hommer D., Bjork J.M., Gilman J.M. Imaging brain response to reward in addictive disorders. Ann NY Acad Sci 2011, 1216:50-61. 10.1111/j.1749-6632.2010.05898.x.
51. Jones M., Love B.C. Bayesian fundamentalism or enlightenment? On the explanatory status and theoretical contributions of Bayesian models of cognition. Behav Brain Sci 2011, 34(4):169-188. 10.1017/S0140525X10003134.
52. Kahnemann D., Tversky A. Subjective probability: a judgement of representativeness. Cognit Psychol 1972, 1(3):430-454.
53. Kersten D., Mamassian P., Yuille A. Object perception as Bayesian inference. Annu Rev Psychol 2004, 55:271-304. 10.1146/annurev.psych.55.090902.142005.
54. Knill D.C., Pouget A. The Bayesian brain: the role of uncertainty in neural coding and computation. Trends Neurosci 2004, 27(12):712-719. 10.1016/j.tins.2004.10.007.
55. Lee T.S., Mumford D. Hierarchical Bayesian inference in the visual cortex. J Opt Soc Am 2003, 20(7):1434. 10.1364/JOSAA.20.001434.
56. MacKay D. Information theory 2003, Cambridge University Press, Inference and Learning Algorithms.
57. Mahmood O.M., Goldenberg D., Thayer R., Migliorini R., Simmons N., Tapert S.F. Adolescents' fMRI activation to a response inhibition task predicts future substance use. Addict Behav 2013, 38(1):1435-1441. 10.1016/j.addbeh.2012.07.012.
58. Maia T.V., Frank M.J. From reinforcement learning models to psychiatric and neurological disorders. Nat Neurosci 2011, 14(2):154-162. 10.1038/nn.2723.
59. Miedl S.F., Peters J., Büchel C. Altered neural reward representations in pathological gamblers revealed by delay and probability discounting. Arch Gen Psychiatry 2012, 69(2):177-186. 10.1001/archgenpsychiatry.2011.1552.
60. Montague P.R., Dolan R.J., Friston K.J., Dayan P. Computational psychiatry. Trends Cognit Sci 2012, 16(1):72-80. 10.1016/j.tics.2011.11.018.
61. Monterosso J., Piray P., Luo S. Neuroeconomics and the study of addiction. Biol Psychiatry 2012, 72(2):107-112. 10.1016/j.biopsych.2012.03.012.
62. Pellicano E., Burr D. When the world becomes "too real": a Bayesian explanation of autistic perception. Trends Cognit Sci 2012, 16(10):504-510. 10.1016/j.tics.2012.08.009.
63. Peters J., Büchel C. The neural mechanisms of inter-temporal decision-making: understanding variability. Trends Cognit Sci 2011, 15(5):227-239. 10.1016/j.tics.2011.03.002.
64. Peters J., Miedl S.F., Büchel C. Formal comparison of dual-parameter temporal discounting models in controls and pathological gamblers. PLoS ONE 2012, 7(11):e47225. 10.1371/journal.pone.0047225.
65. Pouget A., Beck J.M., Ma W.J., Latham P.E. Probabilistic brains: knowns and unknowns. Nat Neurosci 2013, 16(9):1170-1178. 10.1038/nn.3495.
66. Redish A.D., Jensen S., Johnson A. A unified framework for addiction: vulnerabilities in the decision process. Behav Brain Sci 2008, 31(4):415-437. 10.1017/S0140525X0800472X.
67. Redish A.D., Jensen S., Johnson A., Kurth-Nelson Z. Reconciling reinforcement learning models with behavioral extinction and renewal: implications for addiction, relapse, and problem gambling. Psychol Rev 2007, 114(3):784-805. 10.1037/0033-295X.114.3.784.
68. Robinson T.E., Berridge K.C. The neural basis of drug craving: an incentive-sensitization theory of addiction. Brain Res Rev 1993, 18(3):247-291.
69. Schwartenbeck P., FitzGerald T.H.B., Mathys C., Dolan R., Friston K. The dopaminergic midbrain encodes the expected certainty about desired outcomes. Cereb Cortex 2014, 10.1093/cercor/bhu159.
70. Simon H.A. A behavioral model of rational choice. Q J Econ 1955, 69(1):99. 10.2307/1884852.
71. Simon H.A. Bounded rationality and organizational learning. Organiz Sci 1991, 2(1):125-134. 10.1287/orsc.2.1.125.
72. Stephan K.E., Mathys C. Computational approaches to psychiatry. Curr Opin Neurobiol 2014, 25:85-92. 10.1016/j.conb.2013.12.007.
73. Story G.W., Vlaev I., Seymour B., Darzi A., Dolan R.J. Does temporal discounting explain unhealthy behavior? A systematic review and reinforcement learning perspective. Front Behav Neurosci 2014, 8:76. 10.3389/fnbeh.2014.00076.
74. Sutton R., Barto A. Reinforcement learning. An introduction 1998, MIT Press, Cambridge, MA.
75. Taber K.H., Black D.N., Porrino L.J., Hurley R.A. Neuroanatomy of dopamine: reward and addiction. J Neuropsychiatry Clin Neurosci 2012, 24(1):1-4. 10.1176/appi.neuropsych.24.1.1.
76. Tenenbaum JB., Griffiths TL. Generalization, similarity, and Bayesian inference. Behav Brain Sci 2001, 24(4):629-640. discussion 652-791. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/12048947.
77. Tversky A., Kahneman D. Availability: a heuristic for judging frequency and probability. Cognit Psychol 1973, 5:207-232.
78. Tversky A., Kahneman D. Judgment under uncertainty: Heuristics and Biases. Science (New York, N.Y.) 1974, 185(4157):1124-1131. 10.1126/science.185.4157.1124.
79. Weiss Y., Simoncelli E.P., Adelson E.H. Motion illusions as optimal percepts. Nat Neurosci 2002, 5(6):598-604. 10.1038/nn858.
80. Whalley K. Addiction: dopamine dips as drug use increases. Nat Rev Neurosci 2014, 15(6):349. 10.1038/nrn3752.
81. Willuhn I., Burgeno L.M., Groblewski P.A., Phillips P.E.M. Excessive cocaine use results from decreased phasic dopamine signaling in the striatum. Nat Neurosci 2014, 17(5):704-709. 10.1038/nn.3694.
82. Wolpert D.M. Probabilistic models in human sensorimotor control. Hum Mov Sci 2007, 26(4):511-524. 10.1016/j.humov.2007.05.005.
83. Yu A.J. Adaptive behavior: humans act as Bayesian learners. Curr Biol 2007, 17(22):977-980. 10.1016/j.cub.2007.09.007.