Why more is better: Simultaneous modeling of EEG, fMRI, and behavioral data

NeuroImage

Volumn 128, Issue , 2016, Pages 96-115

  Turner, Brandon M ^a   Rodriguez, Christian A ^b   Norcia, Tony M ^b   McClure, Samuel M ^c   Steyvers, Mark ^d  

^a OHIO STATE UNIVERSITY   (United States)

^b STANFORD UNIVERSITY   (United States)

^c ARIZONA STATE UNIVERSITY   (United States)

^d UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Bayesian modeling; EEG; FMRI; Joint modeling framework; Linear ballistic accumulator model

Indexed keywords

ADULT; ARTICLE; BIVARIATE JOINT MODEL; BOLD SIGNAL; COGNITION; ELECTROENCEPHALOGRAPHY; FEMALE; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HUMAN; HUMAN EXPERIMENT; INTERMETHOD COMPARISON; MALE; MEASUREMENT ACCURACY; NORMAL HUMAN; PREDICTION; PRIORITY JOURNAL; PROCESS MODEL; TASK PERFORMANCE; BAYES THEOREM; BIOLOGICAL MODEL; BRAIN; BRAIN MAPPING; NUCLEAR MAGNETIC RESONANCE IMAGING; PHYSIOLOGY; PROCEDURES;

BAYES THEOREM; BRAIN; BRAIN MAPPING; ELECTROENCEPHALOGRAPHY; HUMANS; MAGNETIC RESONANCE IMAGING; MODELS, NEUROLOGICAL;

EID: 84954288617     PISSN: 10538119     EISSN: 10959572     Source Type: Journal    
DOI: 10.1016/j.neuroimage.2015.12.030     Document Type: Article

References (80)

1. Amano K., Goda N., Nishida S., Ejima Y., Takeda T., Ohtani Y. Estimation of the timing of human visual perception from magnetoencephalography. J. Neurosci. 2006, 26:3981-3991.
2. Astolfi L., Cincotti F., Mattia D., Salinari S., Babiloni C., Basilisco A., Rossini P.M., Ding L., Ni Y., He B., Marciani M.G., Babiloni F. Estimation of the effective and functional human cortical connectivity with structural equation modeling and directed transfer function applied to high-resolution EEG. Magn. Reson. Imaging 2004, 22:1457-1470.
3. Bai X., Towle V.L., He E.J., He B. Evaluation of cortical current density imaging methods using intracranial electrocorticograms and functional MRI. NeuroImage 2007, 35:598-608.
4. Beckmann C.F., Smith S.M. Tensorial extensions of independent component analysis for multi-subject fMRI analysis. NeuroImage 2005, 25:294-311.
5. Boehm U., Van Maanen L., Forstmann B., Van Rijn H. Trial-by-trial fluctuations in CNV amplitude reflect anticipatory adjustment of response caution. NeuroImage 2014, 96:95-105.
6. Brown S., Heathcote A. The simplest complete model of choice reaction time: linear ballistic accumulation. Cogn. Psychol. 2008, 57:153-178.
7. Busemeyer J., Townsend J. Decision Field Theory: a dynamic-cognitive approach to decision making in an uncertain environment. Psychol. Rev. 1993, 100:432-459.
8. Calhoun V.D., Adali T. Feature-based fusion of medical imaging data. IEEE Trans. Inf. Technol. Biomed. 2009, 13:711-720.
9. Calhoun V.D., Adali T., Liu J. A feature-based approach to combine functional MRI, structural MRI and EEG brain imaging data. Proceedings of the 28th IEEE EMBS Annual International Conference 2006.
10. Calhoun V.D., Liu J., Adali T. A review of group ICA for fMRI data and ICA for joint inference of imaging, genetic, and ERP data. NeuroImage 2009, 45:163-172.
11. Calhoun V., Wu L., Kiehl K., Eichele T., Pearlson G. Aberrant processing of deviant stimuli in schizophrenia revealed by fusion of fMRI and EEG data. Acta Neuropsychiatry 2011, 22:127-138.
12. Cassey P., Gaut G., Steyvers M., Brown S. A Generative Joint Model for Spike Trains and Saccades during Perceptual Decision Making 2015, (Under review).
13. Christensen R., Johnson W., Branscum A., Hanson T.E. Bayesian Ideas and Data Analysis: An Introduction for Scientists and Statisticians 2011, CRC Press, Taylor and Francis Group, Boca Ranton, FL.
14. Correa N.M., Li Y., Adali T., Calhoun V.D. Canonical correlation analysis for feature based fusion of biomedical imaging modalities and its application to detection of associative networks in schizophrenia. J. Sel. Top. Sign. Proces. 2008, 2:998-1007.
15. Correa N.M., Adali T., Li Y., Calhoun V.D. Canonical correlation analysis for data fusion and group inferences: examining applications of medical imaging data. Signal Process. Mag. 2010, 27:39-50.
16. Correa N.M., Eichele T., Adali T., Li Y., Calhoun V.D. Multi-set canonical correlation analysis for the fusion of concurrent single trial ERP and functional MRI. NeuroImage 2010, 50:1438-1445.
17. Cottereau B.R., Ales J.M., Norcia A.M. How to use fMRI functional localizers to improve EEG/MEG source estimation. J. Neurosci. Methods 2015, 250:64-73.
18. Dähne S., Biemann F., Samek W., Haufe S., Goltz D., Gundlach C., Villringer A., Fazli S., Müller K.R. Multivariate machine learning methods for fusing multimodal functional neuroimaging data. Proc. IEEE 2015, 103:1507-1530.
19. Daw N.D. Trial-by-Trial Data Analysis using Computational Models. Decision Making, Affect, and Learning: Attention and Performance XXIII 2011, 23:1.
20. De Martino F., Valente G., de Borst A.W., Esposito F., Roebroeck A., Goebel R., Formisano E. Multimodal imaging: an evaluation of univariate and multivariate methods for simultaneous EEG/fMRI. Magn. Reson. Imaging 2010, 28:1104-1112.
21. Delorme A., Makeig S. EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J. Neurosci. Methods 2004, 134:9-21.
22. Ditterich J. A comparison between mechanisms of multi-alternative perceptual decision making: ability to explain human behavior, predictions for neurophysiology, and relationship with decision theory. Front. Neurosci. 2010, 4:184.
23. Eichele T., Specht K., Moosmann M., Jongsma M.L., Quiroga R.Q., Nordby H.E.A. Assessing the spatiotemporal evolution of neuronal activation with single-trial event-related potentials and functional. MRI 2005, 102:17798-17803.
24. Eichele T., Calhoun V.D., Debener S. Mining EEG-fMRI using independent component analysis. Int. J. Psychophysiol. 2009, 75:53-61.
25. Forstmann B.U., Wagenmakers E.J. An Introduction to Model-based Cognitive Neuroscience 2015, Springer, New York, New York.
26. Forstmann B.U., Dutilh G., Brown S., Neumann J., von Cramon D.Y., Ridderinkhof K.R., Wagenmakers E.J. Striatum and pre-SMA facilitate decision-making under time pressure. Proc. Natl. Acad. Sci. 2008, 105:17538-17542.
27. Forstmann B.U., Anwander A., Schäfer A., Neumann J., Brown S., Wagenmakers E.J., Bogacz R., Turner R. Cortico-striatal connections predict control over speed and accuracy in perceptual decision making. Proc. Natl. Acad. Sci. 2010, 107:15916-15920.
28. Forstmann B.U., Tittgemeyer M., Wagenmakers E.J., Derrfuss J., Imperati D., Brown S. The speed-accuracy tradeoff in the elderly brain: a structural model-based approach. J. Neurosci. 2011, 31:17242-17249.
29. Forstmann B.U., Wagenmakers E.J., Eichele T., Brown S., Serences J.T. Reciprocal relations between cognitive neuroscience an formal cognitive models: opposites attract?. Trends Cogn. Sci. 2011, 15:272-279.
30. Franco A.R., Ling J., Caprihan A., Calhoun V.D., Jung R.E., Heileman G.L.E.A. Multimodal and multi-tissue measures of connectivity revealed by joint independent component analysis. J. Sel. Top. Sign. Process. 2008, 2:986-997.
31. Friston K., Harisson L., Penny W. Dynamic causal modeling. NeuroImage 2003, 19:1273-1302.
32. Gelman A., Carlin J.B., Stern H.S., Rubin D.B. Bayesian Data Analysis 2004, Chapman and Hall, New York, NY.
33. Gershman S.J., Blei D.M., Pereira F., Norman K.A. A topographic latent source model for fMRI data. NeuroImage 2011, 57:89-100.
34. Green D.M., Swets J.A. Signal Detection Theory and Psychophysics 1966, Wiley Press, New York.
35. Hamandi K., Powell H.W., Laufs H., Symms M.R., Barker G.J., Parker G.J., Lemieux L., Duncan J.S. Combined EEG-fMRI and tractography to visualise propagation of epileptic activity. J. Neurol. Neurosurg. Psychiatry 2008, 79:594-597.
36. Heathcote A., Brown S.D., Wagemakers E.J. An Introduction to Good Practices in Cognitive Modeling. An Introduction to Model-based Cognitive Neuroscience 2015, 25-48. Springer, New York. B.U. Forstmann, E.J. Wagenmakers (Eds.).
37. Ho T., Brown S., van Maanen L., Forstmann B.U., Wagenmakers E.J., Serences J.T. The optimality of sensory processing during the speed-accuracy tradeoff. J. Neurosci. 2012, 32:7992-8003.
38. Huster R.J., Debener S., Eichele T., Herrmann C.S. Methods for simultaneous EEG-fMRI: an introductory review. J. Neurosci. 2013, 32:6053-6060.
39. Kayser J., Tenke C.E. Principal components analysis of Laplacian waveforms as a generic method for identifying ERP generator patterns: II. Adequacy of low-density estimates. Clin. Neurophysiol. 2006, 117:369-380.
40. Kirby K.N. One-year temporal stability of delay-discount rates. Psychon. Bull. Rev. 2009, 16:457-462.
41. Krishnan A., Williams L.J., McIntosh A., Abdi H. Partial least squares (PLS) methods for neuroimaging: a tutorial and review. NeuroImage 2015, 56:455-475.
42. Lee M.D., Wagenmakers E.J. Bayesian Modeling for Cognitive Science: A Practical Course 2013, Cambridge University Press.
43. Lin F.H., McIntosh A.R., Agnew J.A., Eden G.F., Zeffiro T.A., Belliveau J.W. Multivariate analysis of neuronal interactions in the generalized partial least squares framework: simulations and empirical studies. NeuroImage 2003, 20:625-642.
44. Liu J., Calhoun V.D. Parallel independent component analysis for multimodal analysis: application to fMRI and EEG data. Proceedings of IEEE ISBI conference 2007.
45. Love B.C. The algorithmic level is the bridge between computation and brain. Top. Cogn. Sci. 2015, 7.
46. Mack M.L., Preston A.R., Love B.C. Decoding the brain's algorithm for categorization from its neural implementation. Curr. Biol. 2013, 23:2023-2027.
47. Manning J.R., Ranganath R., Norman K.A., Blei D.A. Topographic factor analysis: a Bayesian model for inferring brain networks from neural data. PLoS One 2014, 9.
48. Marr D. Vision: A Computational Investigation into the Human Representation and Processing of Visual Information 1982, Freeman, New York.
49. Martinez-Montes E., Valdes-Sosa P.A., Miwakeichi F., Goldman R.I., Cohen M.S. Concurrent EEG/fMRI analysis by multiway partial least squares. NeuroImage 2004, 22:1023-1034.
50. Mulder M.J., Wagenmakers E.J., Ratcliff R., Boekel W., Forstmann B.U. Bias in the brain: a diffusion model analysis of prior probability and potential payoff. J. Neurosci. 2012, 32:2335-2343.
51. Mulert C., Lemieux L. EEG-fMRI: Physiological Basis, Technique and Applications 2009, Springer, Berlin. first ed.
52. Norman K., Polyn S., Detre G., Haxby J. Beyond mind-reading: multi-voxel pattern analysis of fMRI data. Trends Cogn. Sci. 2006.
53. Nunez M.D., Srinivasan R., Vandekerckhove J. Individual differences in attention influence perceptual decision making. Front. Psychol. 2015, 8:1-13.
54. Palmeri T., Schall J., Logan G. Neurocognitive Modelling of Perceptual Decisions. Oxford Handbook of Computational and Mathematical Psychology 2015, Oxford University Press. J.R. Busemeyer, J. Townsend, Z.J. Wang, A. Eidels (Eds.).
55. Plummer M., Best N., Cowles K., Vines K. CODA: convergence diagnosis and output analysis for MCMC. R News 2006, 6:7-11. (URL: http://CRAN.R-project.org/doc/Rnews/).
56. Purcell B., Heitz R., Cohen J., Schall J., Logan G., Palmeri T. Neurally-constrained modeling of perceptual decision making. Psychol. Rev. 2010, 117:1113-1143.
57. Ratcliff R. A theory of memory retrieval. Psychol. Rev. 1978, 85:59-108.
58. Robert C.P., Casella G. Monte Carlo Statistical Methods 2004, Springer, New York, NY.
59. Rodriguez C.A., Turner B.M., McClure S.M. Intertemporal choice as discounted value accumulation. PLoS One 2014, 9.
60. Rodriguez C.A., Norcia A.M., Cohen M.X., McClure S.M. Frontoparietal cortex dynamics of value integration in intertemporal choice. Manuscript 2015, (in preparation).
61. Rodriguez C.A., Turner B.M., Van Zandt T., McClure S.M. The neural basis of value accumulation in intertemporal choice. Eur. J. Neurosci. 2015.
62. Rouder J.N., Lu J. An introduction to Bayesian hierarchical models with an application in the theory of signal detection. Psychon. Bull. Rev. 2005, 12:573-604.
63. Rouder J.N., Lu J., Speckman P., Sun D., Jiang Y. A hierarchical model for estimating response time distributions. Psychon. Bull. Rev. 2005, 12:195-223.
64. Shadlen M.N., Newsome W.T. Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey. J. Neurophysiol. 2001, 86:1916-1936.
65. Shiffrin R.M., Lee M.D., Kim W., Wagenmakers E.J. A survey of model evaluation approaches with a tutorial on hierarchical Bayesian methods. Cogn. Sci. 2008, 32:1248-1284.
66. Sui J., Adali T., Yu Q., Chen J., Calhoun V.D. A review of multivariate methods for multimodal fusion of brain imaging data. J. Neurosci. Methods 2012, 204:68-81.
67. Summerfield C., Koechlin E. Economic value biases uncertain perceptual choices in the parietal and prefrontal cortices. Front. Hum. Neurosci. 2010, 4:208. 10.3389/fnhum.2010.00208.
68. Teipel S.J., Bokde A.L., Meindl T., Amaro E.J., Soldner J., Reiser M.F., Herpertz S.C., Moller H.J., Hampel H. White matter microstructure underlying default mode network connectivity in the human brain. NeuroImage 2010, 49:2021-2032.
69. Turner B.M. Constraining cognitive abstractions through Bayesian modeling. An Introduction to Model-based Cognitive Neuroscience 2015, 199-220. Springer, New York. B.U. Forstmann, E.J. Wagenmakers (Eds.).
70. Turner B.M., Forstmann B.U., Wagenmakers E.J., Brown S.D., Sederberg P.B., Steyvers M. A Bayesian framework for simultaneously modeling neural and behavioral data. NeuroImage 2013, 72:193-206.
71. Turner B.M., Sederberg P.B., Brown S., Steyvers M. A method for efficiently sampling from distributions with correlated dimensions. Psychol. Methods 2013, 18:368-384.
72. Turner B.M., Forstmann B.U., Love B., Palmeri T., Van Maanen L. Approaches of analysis in model-based cognitive neuroscience. Manuscript 2015, (accepted) Journal of Mathematical Psychology.
73. Turner B.M., Van Maanen L., Forstmann B.U. Combining cognitive abstractions with neurophysiology: the neural drift diffusion model. Psychol. Rev. 2015, 122:312-336.
74. Ullsperger M., Debener S. Simultaneous EEG and fMRI: recording, analysis, and application 2010, Oxford University Press, USA.
75. Usher M., McClelland J.L. On the time course of perceptual choice: the leaky competing accumulator model. Psychol. Rev. 2001, 108:550-592.
76. van Maanen L., Brown S.D., Eichele T., Wagenmakers E.J., Ho T., Serences J. Neural correlates of trial-to-trial fluctuations in response caution. J. Neurosci. 2011, 31:17488-17495.
77. Van Maanen L., Forstmann B., Keuken M., Wagenmakers E.J., Heathcote A. The Impact of MRI Scanner Environment on Perceptual Decision Making 2015, (In Press).
78. White C.N., Congdon E., Mumford J.A., Karlsgodt K.H., Sabb F.W., Freimer N.B., London E.D., Cannon T.D., Bilder R.M., Poldrack R.A. Decomposing decision components in the stop-signal task: a model-based approach to individual differences in inhibitory control. J. Cogn. Neurosci. 2014, 26:1601-1614.
79. Wilson T.D., Houston C.E., Etling K.M., Brekke N. Is model fitting necessary for model-based fMRI?. PLoS Comput. Biol. 2015, 11:e1004237.
80. Xu L., Pearlson G., Calhoun V.D. Joint source based morphometry identifies linked gray and white matter group differences. NeuroImage 2009, 44:777-789.