Trial-by-trial data analysis using computational models

Decision Making, Affect, and Learning: Attention and Performance XXIII

Volumn , Issue , 2011, Pages

  Daw, Nathaniel D ^a  

^a NEW YORK UNIVERSITY   (United States)

Author keywords

Computational models; Data analysis; Decision making; Neural data; Reward learning; Statistical methods

Indexed keywords

EID: 84921627984     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1093/acprof:oso/9780199600434.003.0001     Document Type: Chapter

References (60)

1. Akaike, H. (1974). A new look at the statistical model identification. IEEE transactions on automatic control, 19(6): 716-23.
2. Barraclough, D.J., Conroy, M.L., and Lee, D. (2004). Prefrontal cortex and decisionmaking in a mixed-strategy game. Nature Neuroscience, 7(4): 404-10.
3. Barto, A.G. (1995). Adaptive critics and the basal ganglia. In Houk, J.C., Davis, J.L., and Beiser, D.G. (eds), Models of information processing in the basal ganglia, pp. 215-32. MIT Press, Cambridge, MA.
4. Bayer, H.M. and Glimcher, P.W. (2005). Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron, 47: 129-41.
5. Behrens, T.E.J., Woolrich, M.W., Walton, M.E., and Rushworth, M.F.S. (2007). Learning the value of information in an uncertain world. Nature Neuroscience, 10(9): 1214-21.
6. Bertsekas, D.P. and Tsitsiklis, J.N. (1996). Neuro-dynamic programming. Athena Scientific, Belmont, MA.
7. Bhat, C. (2001). Quasi-random maximum simulated likelihood estimation of the mixed multinomial logit model. Transportation Research Part B, 35(7): 677-93.
8. Bishop, C. (2006). Pattern recognition and machine learning. Springer, New York.
9. Brandstätter, E., Gigerenzer, G., and Hertwig, R. (2006). The priority heuristic:making choices without trade-offs. Psychological Review, 113(2): 409-32.
10. Camerer, C. and Ho, T. (1998). Experience-weighted attraction learning in coordination games: probability rules, heterogeneity, and time-variation. Journal of Mathematical Psychology, 42(2-3): 305-26.
11. Camerer, C. and Ho, T. (1999). Experience-weighted attraction learning in games: a unifying approach. Econometrica, 67(4): 827-74.
12. Daw, N.D. and Doya, K. (2006). The computational neurobiology of learning and reward. Current Opinion in Neurobiology, 16: 199-204.
13. Daw, N.D., Niv, Y., and Dayan, P. (2005). Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nature Neuroscience, 8:1704-11.
14. Daw, N.D., O'Doherty, J.P., Dayan, P., Seymour, B., and Dolan, R.J. (2006). Cortical substrates for exploratory decisions in humans. Nature, 441(7095): 876-9.
15. Daw, N.D., Courville, A.C., and Dayan, P. (2008). Semi-rational models: The case of trial order. In Chater, N. and Oaksford, M. (eds), The probabilistic mind. Oxford University Press.
16. Dayan, P. and Abbott, L.F. (2001). Theoretical neuroscience: computational and mathematical modeling of neural systems. MIT Press, Cambridge, MA.
17. Dayan, P. and Niv, Y. (2008). Reinforcement learning: the good, the bad and the ugly. Current Opinion in Neurobiology, 18(2): 185-96.
18. Delgado, M.R., Nystrom, L.E., Fissell, C., Noll, D.C., and Fiez, J.A. (2000). Tracking the hemodynamic responses to reward and punishment in the striatum. Journal of Neurophysiology, 84: 3072-7.
19. Frank, M.J., Moustafa, A.A., Haughey, H.M., Curran, T., and Hutchison, K.E. (2007). Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning. Proceedings of the National Academy of Sciences of the United States of America, 104(41): 16311-6.
20. Frank, M.J., Seeberger, L.C., and O'Reilly, R.C. (2004). By carrot or by stick:cognitive reinforcement learning in Parkinsonism. Science, 306(5703): 1940-3.
21. Freedman, D. (2006). On the so-called "Huber sandwich estimator" and"robust standard errors." American Statistician, 60(4): 299-302.
22. Friston, K.J. and Penny, W. (2003). Posterior probability maps and spms. Neuroimage, 19(3): 1240-9.
23. Friston, K.J., Fletcher, P., Josephs, O., Holmes, A., Rugg, M.D., and Turner, R. (1998). Event-related fMRI: characterizing differential responses. Neuroimage, 7(1): 30-40.
24. Friston, K.J., Stephan, K.E., Lund, T.E., Morcom, A., and Kiebel, S. (2005). Mixedeffects and fmri studies. Neuroimage, 24(1): 244-52.
25. Gelman, A., Carlin, J., and Stern, H. (2004). Bayesian data analysis. CRC Press Boca Raton.
26. Gelman, A. and Hill, J. (2007). Data analysis using regression and multilevel/hierarchical models. Cambridge University Press, New York.
27. Hampton, A.N., Bossaerts, P., and O'Doherty, J.P. (2008). Neural correlates of mentalizing-related computations during strategic interactions in humans. Proceedings of the National Academy of Sciences of the United States of America, 105(18): 6741-6.
28. Hare, T.A., O'Doherty, J., Camerer, C.F., Schultz, W., and Rangel, A. (2008). Dissociating the role of the orbitofrontal cortex and the striatum in the computation of goal values and prediction errors. Journal of Neuroscience, 28(22): 5623-30.
29. Holmes, A. and Friston, K. (1998). Generalisability, random effects and population inference. Neuroimage, 7: s754.
30. Huber, P. (1967). The behavior of maximum likelihood estimates under nonstandard conditions. In Proceedings of the fifth Berkeley symposium in mathematical statistics, vol. 1, pp. 221-33.
31. Kable, J.W. and Glimcher, P.W. (2007). The neural correlates of subjective value during intertemporal choice. Nature Neuroscience, 10(12): 1625-33.
32. Kakade, S. and Dayan, P. (2002). Acquisition and extinction in autoshaping. Psychological Review, 109: 533-44.
33. Kass, R.E. and Raftery, A.E. (1995). Bayes factors. Journal of the American Statistical Association, 90: 7730-95.
34. Knutson, B., Westdorp, A., Kaiser, E., and Hommer, D. (2000). FMRI visualization of brain activity during a monetary incentive delay task. NeuroImage, 12: 20-27.
35. Lau, B. and Glimcher, P.W. (2005). Dynamic response-by-response models of matching behavior in rhesus monkeys. Journal of the Experimental Analysis of Behavior, 84: 555-79.
36. Li, J., McClure, S.M., King-Casas, B., and Montague, P.R. (2006). Policy adjustment in a dynamic economic game. PLoS One, 1: e103.
37. Lohrenz, T., McCabe, K., Camerer, C.F., and Montague, P.R. (2007). Neural signature of fictive learning signals in a sequential investment task. Proceedings of the National Academy of Sciences of the United States of America, 104(22): 9493-8.
38. MacKay, D. (2003). Information theory, inference, and learning algorithms. Cambridge University Press.
39. McClure, S.M., Berns, G.S., and Montague, P.R. (2003). Temporal prediction errors in a passive learning task activate human striatum. Neuron, 38(2): 339-46.
40. McFadden, D. (1974). Conditional logit analysis of qualitative choice behavior. In Zarembka, P., (ed.), Frontiers in econometrics, pp. 105-42. Academic Press, New York.
41. Ng, A. and Jordan, M. (2000). PEGASUS: A policy search method for large MDPs and POMDPs. In Proceedings of the sixteenth Conference on Uncertainty in Artificial Intelligence, pp. 406-415.
42. Norman, K.A., Polyn, S.M., Detre, G.J., and Haxby, J.V. (2006). Beyond mindreading:multi-voxel pattern analysis of fmri data. Trends in Cognitive Science, 10(9): 424-30.
43. O'Doherty, J.P., Dayan, P., Friston, K., Critchley, H., and Dolan, R.J. (2003). Temporal difference models and reward-related learning in the human brain. Neuron, 38(2): 329-37.
44. O'Doherty, J.P., Hampton, A., and Kim, H. (2007). Model-based fMRI and its application to reward learning and decision-making. Annuals of the New York Academy of Sciences, 1104: 35-53.
45. Penny, W. and Friston, K. (2004). Hierarchical models. Human brain function (2nd edn), pp. 851-63. Elsevier, London.
46. Pessiglione, M., Petrovic, P., Daunizeau, J., Palminteri, S., Dolan, R.J., and Frith, C.D. (2008). Subliminal instrumental conditioning demonstrated in the human brain. Neuron, 59(4): 561-7.
47. Plassmann, H., O'Doherty, J., Shiv, B., and Rangel, A. (2008). Marketing actions can modulate neural representations of experienced pleasantness. Proceedings of the National Academy of Sciences of the United States of America, 105(3): 1050-4.
48. Platt, M.L. and Glimcher, P.W. (1999). Neural correlates of decision variables in parietal cortex. Nature, 400(6741): 233-8.
49. Samejima, K., Doya, K., Ueda, Y., and Kimura, M. (2004). Estimating internal variables and parameters of a learning agent by a particle filter. Advances in Neural Information Processing Systems, 16, 1335-42.
50. Samejima, K., Ueda, Y., Doya, K., and Kimura, M. (2005). Representation of actionspecific reward values in the striatum. Science, 310(5752): 1337-40.
51. Schonberg, T., Daw, N.D., Joel, D., and O'Doherty, J.P. (2007). Reinforcement learning signals in the human striatum distinguish learners from nonlearners during reward-based decision-making. Journal of Neuroscience, 27(47): 12860-7.
52. Schultz, W., Dayan, P., and Montague, P.R. (1997). A neural substrate of prediction and reward. Science, 275: 1593-99.
53. Schwarz, G. (1978). Estimating the dimension of a model. Annals of Statistics, 6: 461-4.
54. Seymour, B., Daw, N., Dayan, P., Singer, T., and Dolan, R. (2007). Differential encoding of losses and gains in the human striatum. Journal of Neuroscience, 27(18): 4826-31.
55. Stephan, K.E., Penny, W.D., Daunizeau, J., Moran, R.J., and Friston, K.J. (2009). Bayesian model selection for group studies. Neuroimage, 46: 1004-17.
56. Sugrue, L.P., Corrado, G.S., and Newsome, W.T. (2004). Matching behavior and the representation of value in the parietal cortex. Science, 304: 1782-7.
57. Sutton, R.S. and Barto, A.G. (1998). Reinforcement learning: an introduction. MIT Press, Cambridge, MA.
58. Tom, S.M., Fox, C.R., Trepel, C., and Poldrack, R.A. (2007). The neural basis of loss aversion in decision-making under risk. Science, 315(5811): 515-8.
59. Watkins, C.J.C.H. (1989). Learning from delayed rewards. PhD thesis, Cambridge University, Cambridge, England.
60. Wittmann, B.C., Daw, N.D., Seymour, B., and Dolan, R.J. (2008). Striatal activity underlies novelty-based choice in humans. Neuron, 58(6): 967-73.