Exploration, novelty, surprise, and free energy minimization

Frontiers in Psychology

Volumn 4, Issue OCT, 2013, Pages

  Schwartenbeck, Philipp ^a   FitzGerald, Thomas ^a   Dolan, Raymond J ^a   Friston, Karl ^a  

^a UNIVERSITY COLLEGE LONDON   (United Kingdom)

Author keywords

Active inference; Exploitation; Exploration; Free energy; Novelty; Reinforcement learning

Indexed keywords

EID: 84885811267     PISSN: None     EISSN: 16641078     Source Type: Journal    
DOI: 10.3389/fpsyg.2013.00710     Document Type: Article

References (33)

1. Adams, R., Shipp, S., and Friston, K. J. (2013a). Predictions not commands: active inference in the motor system. Brain Struct. Funct. 218, 611-643. doi: 10.1007/s00429-012-0475-5
2. Adams, R., Stephan, K. E., Brown, H. R., Frith, C. D., and Friston, K. J. (2013b). The computational anatomy of psychosis. Front. Psychiatry 4:47. doi: 10.3389/fpsyt.2013.00047
3. Apps, M. A. J., and Tsakiris, M. (2013). The free-energy self: a predictive coding account of self-recognition. Neurosci. Biobehav. Rev. doi: 10.1016/j.neubiorev.2013.01.029. [Epub ahead of print].
4. Bastos, A. M., Usrey, W. M., Adams, R., Mangun, G. R., Fries, P., and Friston, K. (2012). Canonical microcircuits for predictive coding. Neuron 76, 695-711. doi: 10.1016/j.neuron.2012.10.038
5. Brown, H., Friston, K., and Bestmann, S. (2011). Active inference, attention, and motor preparation. Front. Psychol. 2:218. doi: 10.3389/fpsyg.2011.00218
6. Brown, H., and Friston, K. J. (2012). Free-energy and illusions: the cornsweet effect. Front. Psychol. 3:43. doi: 10.3389/fpsyg.2012.00043
7. Clark, A. (2013). Whatever next. Predictive brains, situated agents, and the future of cognitive science. Behav. Brain Sci. 36, 181-204. doi: 10.1017/S0140525X12000477
8. Cohen, J. D., McClure, S. M., and Yu, A. J. (2007). Should I stay or should I go. How the human brain manages the trade-offbetween exploitation and exploration. Philos. Trans. R. Soc. Lond. B Biol. Sci. 362, 933-942. doi: 10.1098/rstb.2007.2098
9. Edwards, M., Adams, R., Brown, H., Pareés, I., and Friston, K. (2012). A Bayesian account of hysteria. Brain 135, 3495-3512. doi: 10.1093/brain/aws129
10. Feldman, H., and Friston, K. J. (2010). Attention, uncertainty, and free-energy. Front. Hum. Neurosci. 4:215. doi: 10.3389/fnhum.2010.00215
11. Feynman, R. A. (1972). Statistical Mechanics. Reading, MA: Benjamin.
12. Friston, K. (2005). A theory of cortical responses. Philos. Trans. R. Soc. Lond. B Biol. Sci. 360, 815-836. doi: 10.1098/rstb.2005.1622
13. Friston, K. (2009). The free-energy principle: a rough guide to the brain. Trends Cogn. Sci. 13, 293-301. doi: 10.1016/j.tics.2009.04.005
14. Friston, K. (2010). The free-energy principle: a unified brain theory. Nat. Rev. Neurosci. 11, 127-138. doi: 10.1038/nrn2787
15. Friston, K. (2011). "Embodied Inference: or 'I think therefore I am, if I am what I think,"' in The Implications of Embodiment (Cognition and Communication), eds W. Tschacher and C. Bergomi (Exeter: Imprint Academic), 89-125.
16. Friston, K. (2012). A free energy principle for biological systems. Entropy 14, 2100-2121. doi: 10.3390/e14112100
17. Friston, K. (2013). Active inference and free energy. Behav. Brain Sci. 36, 212-213. doi: 10.1017/S0140525X12002142
18. Friston, K., Kilner, J., and Harrison, L. (2006). A free energy principle for the brain. J. Physiol. 100, 70-87. doi: 10.1016/j.jphysparis.2006.10.001
19. Friston, K., Thornton, C., and Clark, A. (2012). Free-energy minimization and the dark-room problem. Front. Psychol. 3:130. doi: 10.3389/fpsyg.2012.00130
20. Friston, K., Schwartenbeck, P., FitzGerald, T., Moutoussis, M., Behrens, T., and Dolan, R. J. (2013). The anatomy of choice: active inference and agency. Front. Hum. Neurosci. 7:598. doi: 10.3389/fnhum.2013.00598
21. Friston, K. J., Daunizeau, J., and Kiebel, S. J. (2009). Reinforcement learning or active inference. PLoS ONE 4:e6421. doi: 10.1371/journal.pone.0006421
22. Hinton, G. E., and van Camp, D. (1993). Keeping the neural networks simple by minimizing the description length of the weights, in Proceedings of the Sixth Annual Conference on Computational Learning Theory-COLT93 (New York, NY: ACM Press), 5-13. doi: 10.1145/168304.168306
23. Huang, Y., and Rao, R. P. N. (2013). Reward optimization in the primate brain: a probabilistic model of decision making under uncertainty. PLoS ONE 8:e53344. doi: 10.1371/journal.pone.0053344
24. Jaynes, E. (1957). Information theory and statistical mechanics. Phys. Rev. 106, 620-630. doi: 10.1103/PhysRev.106.620
25. Joffily, M., and Coricelli, G. (2013). Emotional valence and the free-energy principle. PLoS Comput. Biol. 9:e1003094. doi: 10.1371/journal.pcbi.1003094
26. Kakade, S., and Dayan, P. (2002). Dopamine: generalization and bonuses. Neural Netw. 15, 549-559. doi: 10.1016/S0893-6080(02)00048-5
27. Luciw, M., Kompella, V., Kazerounian, S., and Schmidhuber, J. (2013). An intrinsic value system for developing multiple invariant representations with incremental slowness learning. Front. Neurorobot. 7:9. doi: 10.3389/fnbot.2013.00009
28. Moran, R. J., Campo, P., Symmonds, M., Stephan, K. E., Dolan, R. J., and Friston, K. (2013). Free energy, precision and learning: the role of cholinergic neuromodulation. J. Neurosci. 33, 8227-8236. doi: 10.1523/JNEUROSCI.4255-12.2013
29. Schmidhuber, J. (1991). Curious model-building control systems, in Proceedings of IEEE International Joint Conference on Neural Networks (Singapore), 1458-1463.
30. Schmidhuber, J. (2009). Simple algorithmic theory of subjective beauty, novelty, surprise, interestingness, attention, curiosity, creativity, art, science, music, jokes. 48, 21-32.
31. Solway, A., and Botvinick, M. M. (2012). Goal-directed decision making as probabilistic inference: a computational framework and potential neural correlates. Psychol. Rev. 119, 120-154. doi: 10.1037/a0026435
32. Sutton, R., and Barto, A. (1998). Reinforcement Learning. An Introduction. Cambridge, MA: MIT Press.
33. Thrun, S. B. (1992). The Role of Exploration in Learning Control. Handbook of Intelligent Control: Neural, Fuzzy and Adaptive Approaches. Frorence, KY: Van Nostrand Reinhold.