Observing the observer (I): Meta-bayesian models of learning and decision-making

PLoS ONE

Volumn 5, Issue 12, 2010, Pages

  Daunizeau, Jean ^a,c   den Ouden, Hanneke E M ^e   Pessiglione, Matthias ^b   Kiebel, Stefan J ^d   Stephan, Klaas E ^a,c   Friston, Karl J ^a  

^a UNIVERSITY COLLEGE LONDON   (United Kingdom)

^b PITIÉ SALPÊTRIÈRE HOSPITAL   (France)

^c UNIVERSITY OF ZURICH   (Switzerland)

^d MAX PLANCK INSTITUTE FOR HUMAN COGNITIVE AND BRAIN SCIENCES   (Germany)

^e RADBOUD UNIVERSITY   (Netherlands)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; BAYES THEOREM; DECISION MAKING; FUNCTIONAL STATUS; GENETIC ANALYSIS; LEARNING; MATHEMATICAL MODEL; PERCEPTION; PREDICTION; PROCESS OPTIMIZATION; RESPONSE TIME; SIMULATION; THEORETICAL STUDY; ALGORITHM; DISCRIMINATION LEARNING; HUMAN; OBSERVER VARIATION; PHYSIOLOGY; STATISTICAL MODEL; THEORETICAL MODEL;

ALGORITHMS; BAYES THEOREM; DECISION MAKING; DISCRIMINATION LEARNING; HUMANS; LEARNING; MODELS, STATISTICAL; MODELS, THEORETICAL; OBSERVER VARIATION; PERCEPTION;

EID: 78650771350     PISSN: None     EISSN: 19326203     Source Type: Journal    
DOI: 10.1371/journal.pone.0015554     Document Type: Article

References (68)

1. Daw ND, O'Doherty JP, Dayan P, Seymour B, Dolan RJ (2006) Cortical substrates for exploratory decisions in humans. Nature. 2006 Jun 15; 441(7095): 876-9.
2. Fehr E, Schmidt KM (1999) A theory of fairness, competition, and cooperation. The Quarterly Journal of Economics 114: 817-868.
3. Kahneman D, Tversky A (1979) Prospect theory: An analysis of decisions under risk. Econometrica 47: 313-327.
4. Beck JM, Ma WJ, Kiani R, Hanks T, Churchland AK, et al. (2008) Probabilistic population codes for Bayesian decision making. Neuron 60(6): 1142-52.
5. Dayan P, Hinton GE, Neal RM, Zemel RS (1995) The Helmholtz machine. Neural Comput 7(5): 889-904.
6. Rao RP, Ballard DH (1999) Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive field effects. Nat Neurosci 2: 79-87.
7. Friston K, Kilner J, Harrison L (2006) A free-energy principle for the brain, J of physiol Paris 100: 70-87.
8. Helmholtz H (1925) Physiological optics, Vol. III: the perception of vision (J.P. Southall, Trans.), Optical Soc. Of Am., Rochester, NY USA.
9. Mach E (1980) Contributions to the analysis of the sensations (C. M. Williams, Trans.), Chicago, IL, USA: Open Court Publishing Co.
10. Kersten D, Knill DC, Mamassian P, Bülthoff I (1996) Illusory motion from shadows, Nature 379, 31.
11. Rao RP (1998) An optimal estimation approach to visual perception and learning, Vision Res 39: 1963-1989.
12. Lee TS (2003) Computations in the early visual cortex, J Physiol Paris 9: 121-139.
13. Lee TS, Mumford D (2003) Hierarchical Bayesian inference in the visual cortex, J Opt Soc Am A Opt Image Sci Vis 20: 1434-1448.
14. Mamassian P, Landy MS, Maloney MS (2002) Bayesian modelling of visual perception. In: Rao R, Olshausen B, Lewicki M, eds. Probabilistic Models of the Brain. Cambridge, MA: MIT Press.
15. Mamassian P, Jentsch I, Bacon BA, Schweinberger SR (2003) Neural correlates of shape from shading, NeuroReport 14: 971-975.
16. Weiss Y, Simoncelli EP, Adelson EH (2002) Motion illusions as optimal percepts, Nature Neuroscience 5: 598-604.
17. Knill DC (1998) Discrimination of planar surface slant from texture: human and ideal observers compared. Vision Res 38: 780-784.
18. Knill DC, Pouget A (2004) The Bayesian brain: the role of uncertainty in neural coding and computation, Trends in Neurosci 27: 712-719.
19. Ernst MO, Banks MS (2002) Humans integrate visual and haptic information in a statistically optimal fashion, Nature 415: 429-433.
20. Van Ee R, et al. (2003) Bayesian modelling of cue interaction: bistability in stereoscopic slant perception. J Opt Soc Opt Am A Opt Image Sci Vis 20: 1398-1406.
21. Kording KP, Beierholm U, Ma WJ, Quartz S, Tenenbaum JB, et al. (2007) Causal inference in multisensory perception. PLoS ONE. September 2007, Issue 9, e943.
22. Wolpert DM, et al. (1995) An internal model for sensorimotor integration. Science 269: 1880-1882.
23. Harris CM, Wolpert DM (1998) Signal-dependent noise determines motor planning. Nature 394: 780-784.
24. Van Beers RJ, et al. (2002) Role of uncertainty in sensorimotor control. Philos Trans R Soc Lond B Biol Sci 357: 1137-1145.
25. Trommershauser J, Maloney LT, Landy MS (2003) Statistical decision theory and trade-offs in the control of motor response, Spatial vision 16: 255-275.
26. Kording KP, Fukunaga I, Howard IS, Ingram JN, Wolpert DM (2004) A neuroeconomics approach to inferring utility functions in sensorimotor control. PloS Biol 2(10): e330.
27. Saunders JA, Knill DC (2004) Visual feedback control of hand movement. J Neurosci 24: 3223-3234.
28. Behrens TE, Woolrich MW, Walton ME, Rushworth MF. Learning the value of information in an uncertain world. Nat Neurosci. 2007 Sep; 10(9): 1214-21.
29. Den Ouden HEM, Daunizeau J, Roiser J, Friston KJ, Stephan KE (2010) Striatal prediction error modulates cortical coupling. J Neurosci 30: 3210-3219.
30. Dayan P, Kakade S, Montague PR (2000) Learning and selective attention, Nat Rev Neurosci 3: 1218-1223.
31. Dayan P, Yu AJ (2003) Uncertainty and learning, IETE J Research 49: 171-181.
32. Tenenbaum JB, Kemp C, Shafto P (2007) Theory-based Bayesian models of inductive reasoning. In Feeney A, Heit E, eds. Inductive reasoning Cambridge University Press.
33. Tenenbaum JB, Griffiths TL, Kemp C (2006) Theory-based Bayesian models of inductive learning and reasoning. Trends in Cognitive Sciences 10(7). pp 309-318.
34. Robert C (1992) L'analyse statistique Bayesienne. Economica.
35. Von Neumann J, Morgenstern O (1944) Theory of games and economic behavior, Princeton University Press, ISBN: 978-0-691-13061-3.
36. Hadamard J (1902) Sur les problèmes aux dérivées partielles et leur signification physique. Princeton University Bulletin. pp 49-52.
37. Swartz RJ, Cox DD, Scott SB, Davies K, Follen M (2006) inverse decision theory: characterizing losses for a decision rule with applications in cervical cancer screening. J am Stat Assoc 101: 1-8.
38. Wolpert D, Tumer K (2001) Reinforcement learning in distributed domains: an inverse game theoretic approach. 2001 AAAI Spring Symposium on Game theoretic and decision theoretic agents Parsons S, Gmytrasiewicz G, eds. Stanford C.A.
39. Ng AY, Russel S (2000) Algorithms for inverse reinforcement learning. Proc 17th Int Conf. Machine Learning.
40. Abbeel P, Ng AY (2004) Apprenticeship learning via inverse reinforcement learning. Proc 21st Int Conf. machine Learning, Bannf, Canada, 2004.
41. Ramachandran D, Amir E (2007) Bayeisan inverse reinforcement learning. Proc Conf. IJCAI-07.
42. Krishnamurthy D, Todorov E (2009) Efficient algorithms for inverse optimal control. Submitted.
43. Beal M (2003) Variational algorithms for approximate Bayesian inference, PhD thesis, ION, UCL, UK.
44. Brodersen KH, Penny WD, Harrison LM, Daunizeau J, Ruff CC, et al. (2008) Integrated Bayesian models of learning and decision making for saccadic eye movements. Neural Networks 21: 1247-1260.
45. Spiegelhalter DJ, Best NG, Carlin BP, Van Der Linde A (2002) Bayesian measures of complexity and fit, J R Statist Soc B 64: 583-639.
46. Friston K, Mattout J, Trujillo-Barreto N, Ashburner J, Penny W (2007) Variational free-energy and the Laplace approximation. NeuroImage 1: 220-234.
47. Gold JI, Shadlen MN (2001) Neural computations that underlie decisions about sensory stimuli. Trends Cogn Sci 5(1): 10-16.
48. Glimcher PW (2003) The neurobiology of visual-saccadic decision making. Annu Rev Neurosci 2003; 26: 133-79.
49. Carpenter RH, Williams ML (1995) Neural computation of log likelihood in control of saccadic eye movements. Nature 377: 59-62.
50. Frith U, Frith CD (2003) Development and neurophysiology of mentalizing. Philos Trans R Soc Lond B Biol Sci 358(1431): 459-73.
51. Yoshida W, Dolan RJ, Friston KJ (2008) Game Theory of Mind. PLoS Comput Biol 4(12): e1000254.
52. Kersten D, Mamassian P, Yuille A (2004) Object perception as Bayesian inference, Annual Review of Psychology 55: 271-304.
53. Kiebel SJ, Daunizeau J, Friston KJ (2009) Perception and hierarchical dynamics. Frontiers in neuroinformatics (2009) 3: 20.
54. Dayan P (2009) Dopamine, reinforcement learning, and addiction. Pharmacopsychiatry 42(S1): S56-S65.
55. Dayan P, Huys QJM (2009) Serotonin in Affective Control. Annual Review of Neuroscience 32: 95-126.
56. Yu AJ, Dayan P (2002) Acetylcholine in cortical inference. Neural Networks 15: 719-730.
57. Dayan P (2009) Goal-directed control and its antipodes. Neural Networks 22: 213-219.
58. Friston KJ, Daunizeau J, Kiebel SJ (2009) Reinforcement learning or active inference. Plos ONE 4(7): e6421.
59. Friston KJ, Daunizeau J, Kilner J, Kiebel SJ (2010) Action and behaviour: a free energy formulation. Bio Cybern 102: 227-260.
60. Nienborg H, Cumming BG (2009) Decision-related activity in sensory neurons reflects more than a neuron's causal effect. Nature 459: 89-92.
61. Kirk DE (2004) Optimal control theory: an introduction. Dover publications, ISBN: 9780486434841.
62. Shoham Y, Leyton-Brown K (2009) Multiagent Systems: Algorithmic, Game-Theoretic, and Logical Foundations, New York: Cambridge University Press, ISBN: 978-0-521-89943-7.
63. Sutton RS, Barto AG (1998) Reinforcement learning: an introduction, Cambridge, MA: MIT Press, A Bradford book.
64. Todorov E (2006) Linearly-solvable Markov decision problems. In Advances in Neural Information Processing Systems 19: 1369-1376, Scholkopf, et al (eds), MIT Press.
65. Watkins CJCH, Dayan P (1992) Q-learning. Machine Learning 8: 279-292.
66. McClelland JL, Botvinick MM, Noelle DC, Plaut DC, Rogers TT, et al. (2010) Letting Structure Emerge: Connectionist and Dynamical Systems Approaches to Understanding Cognition. Trends in Cognitive Sciences 14: 348-356.
67. Daunizeau J, Preuschoff K, Friston KJ, Stephan KE (in preparation). Optimizing experimental design for Bayesian model comparison.
68. Daunizeau J, Den Ouden HEM, Pessiglione M, Kiebel SJ, Friston KJ, et al. (2010) Observing the Observer (II): Deciding when to decide. PLoS ONE 5(12): e15555. doi:10.1371/journal.pone.0015555.