A review of multivariate methods for multimodal fusion of brain imaging data

Journal of Neuroscience Methods

Volumn 204, Issue 1, 2012, Pages 68-81

  Sui, Jing ^a   Adali, Tülay ^b   Yu, Qingbao ^a   Chen, Jiayu ^a,c   Calhoun, Vince D ^a,c,d  

^a MIND RESEARCH NETWORK   (United States)

^b UNIVERSITY OF MARYLAND BALTIMORE COUNTY   (United States)

^c UNIVERSITY OF NEW MEXICO   (United States)

^d YALE UNIVERSITY SCHOOL OF MEDICINE   (United States)

Author keywords

CCA; ICA; MRI; Multimodal fusion; Multivariate methods; PLS

Indexed keywords

ANALYTIC METHOD; BRAIN FUNCTION; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HUMAN; INDEPENDENT COMPONENT ANALYSIS; MULTIVARIATE ANALYSIS; NEUROIMAGING; PARTIAL LEAST SQUARES REGRESSION; PRIORITY JOURNAL; REVIEW; SIMULATION; STATISTICAL ANALYSIS;

ANIMALS; BRAIN; DATA INTERPRETATION, STATISTICAL; FUNCTIONAL NEUROIMAGING; HUMANS; IMAGE INTERPRETATION, COMPUTER-ASSISTED; MULTIVARIATE ANALYSIS; PATTERN RECOGNITION, AUTOMATED; REPRODUCIBILITY OF RESULTS; SENSITIVITY AND SPECIFICITY; SUBTRACTION TECHNIQUE;

EID: 84855185464     PISSN: 01650270     EISSN: 1872678X     Source Type: Journal    
DOI: 10.1016/j.jneumeth.2011.10.031     Document Type: Review

References (77)

1. Astolfi L., Cincotti F., Mattia D., Salinari S., Babiloni C., Basilisco A., et al. Estimation of the effective and functional human cortical connectivity with structural equation modeling and directed transfer function applied to high-resolution EEG. Magn Reson Imaging 2004, 22:1457-1470.
2. Babiloni F., Mattia D., Babiloni C., Astolfi L., Salinari S., Basilisco A., et al. Multimodal integration of EEG, MEG and fMRI data for the solution of the neuroimage puzzle. Magn Reson Imaging 2004, 22:1471-1476.
3. Beckmann C.F., Smith S.M. Tensorial extensions of independent component analysis for multisubject FMRI analysis. NeuroImage 2005, 25:294-311.
4. Bell A.J., Sejnowski T.J. An information-maximization approach to blind separation and blind deconvolution. Neural Comput 1995, 7:1129-1159.
5. Bishop C.M. Variational principal components. Articial Neural Networks 1999, 7:509.
6. Blum A.L., Langley P. Selection of relevant features and examples in machine learning. Artif Intell 1997, 97:245-271.
7. Calhoun V., Wu L., Kiehl K., Eichele T., Pearlson G. Aberrant processing of deviant stimuli in Schizophrenia revealed by fusion of FMRI and EEG data. Acta Neuropsychiatr 2011, 22:127-138.
8. Calhoun V.D., Adali T. Feature-based fusion of medical imaging data. IEEE Trans Inf Technol Biomed 2009, 13:711-720.
9. Calhoun V.D., Adali T., Giuliani N.R., Pekar J.J., Kiehl K.A., Pearlson G.D. Method for multimodal analysis of independent source differences in schizophrenia: combining gray matter structural and auditory oddball functional data. Hum Brain Mapp 2006, 27:47-62.
10. Calhoun V.D., Adali T., Kiehl K.A., Astur R., Pekar J.J., Pearlson G.D. A method for multitask fMRI data fusion applied to schizophrenia. Hum Brain Mapp 2006, 27:598-610.
11. Calhoun V.D., Adali T., Liu J. A feature-based approach to combine functional MRI, structural MRI and EEG brain imaging data. Proceedings of the 28th IEEE EMBS Annual International Conference 2006, 3672-3675.
12. Calhoun V.D., Liu J., Adali T. A review of group ICA for fMRI data and ICA for joint inference of imaging, genetic, and ERP data. NeuroImage 2009, 45:S163-S172.
13. Camara E., Rodriguez-Fornells A., Munte T.F. Microstructural brain differences predict functional hemodynamic responses in a reward processing task. J Neurosci 2010, 30:11398-11402.
14. Caprihan A., Pearlson G.D., Calhoun V.D. Application of principal component analysis to distinguish patients with schizophrenia from healthy controls based on fractional anisotropy measurements. NeuroImage 2008, 42:675-682.
15. Chen K., Reiman E.M., Huan Z., Caselli R.J., Bandy D., Ayutyanont N., et al. Linking functional and structural brain images with multivariate network analyses: a novel application of the partial least square method. NeuroImage 2009, 47:602-610.
16. Choi K., Yang Z., Hu X., Mayberg H. A combined functional-structural connectivity analysis of major depression using joint independent components analysis 2008, ISMRM, Toronto, Cananda, p. 3555.
17. Correa N., Adali T., Calhoun V.D. Performance of blind source separation algorithms for fMRI analysis using a group ICA method. Magn Reson Imaging 2007, 25:684-694.
18. Correa N.M., Adali T., Li Y.O., Calhoun V.D. Canonical correlation analysis for data fusion and group inferences: examining applications of medical imaging data. IEEE Signal Process Mag 2010, 27:39-50.
19. Correa N.M., Eichele T., Adali T., Li Y.O., Calhoun V.D. Multi-set canonical correlation analysis for the fusion of concurrent single trial ERP and functional MRI. NeuroImage 2010.
20. Correa N.M., Li Y.O., Adali T., Calhoun V.D. Canonical correlation analysis for feature-based fusion of biomedical imaging modalities and its application to detection of associative networks in Schizophrenia. IEEE J Sel Top Signal Process 2008, 2:998-1007.
21. Dale A.M., Halgren E. Spatiotemporal mapping of brain activity by integration of multiple imaging modalities. Curr Opin Neurobiol 2001, 11:202-208.
22. De Martino F., Valente G., de Borst A.W., Esposito F., Roebroeck A., Goebel R., et al. Multimodal imaging: an evaluation of univariate and multivariate methods for simultaneous EEG/fMRI. Magn Reson Imaging 2010, 28:1104-1112.
23. Eichele T., Calhoun V.D., Debener S. Mining EEG-fMRI using independent component analysis. Int J Psychophysiol 2009, 73:53-61.
24. Eichele T., Specht K., Moosmann M., Jongsma M.L., Quiroga R.Q., Nordby H., et al. Assessing the spatiotemporal evolution of neuronal activation with single-trial event-related potentials and functional MRI. Proc Natl Acad Sci U S A 2005, 102:17798-17803.
25. Franco A.R., Ling J., Caprihan A., Calhoun V.D., Jung R.E., Heileman G.L., et al. Multimodal and multi-tissue measures of connectivity revealed by joint independent component analysis. IEEE J Sel Top Signal Process 2008, 2:986-997.
26. Friston K.J. Modalities, modes, and models in functional neuroimaging. Science 2009, 326:399-403.
27. Garrity A.G., Pearlson G.D., McKiernan K., Lloyd D., Kiehl K.A., Calhoun V.D., et al. default mode functional connectivity in schizophrenia. Am J Psychiatry 2007, 164:450-457.
28. Goldberg-Zimring D., Mewes A.U., Maddah M., Warfield S.K. Diffusion tensor magnetic resonance imaging in multiple sclerosis. J Neuroimaging 2005, 15:68S-81S.
29. Groves A.R., Beckmann C.F., Smith S.M., Woolrich M.W. Linked independent component analysis for multimodal data fusion. NeuroImage 2011, 54:2198-2217.
30. Hamandi K., Powell H.W., Laufs H., Symms M.R., Barker G.J., Parker G.J., et al. Combined EEG-fMRI and tractography to visualise propagation of epileptic activity. J Neurol Neurosurg Psychiatry 2008, 79:594-597.
31. Henson R.N., Flandin G., Friston K.J., Mattout J. A parametric empirical bayesian framework for fMRI-constrained MEG/EEG source reconstruction. Hum Brain Mapp 2010, 31:1512-1531.
32. Horwitz B., Poeppel D. How can EEG/MEG and fMRI/PET data be combined?. Hum Brain Mapp 2002, 17:1-3.
33. Hyvarinen A., Oja E. Independent component analysis: algorithms and applications. Neural Netw 2000, 13:411-430.
34. Jagannathan K., Calhoun V.D., Gelernter J., Stevens M.C., Liu J., Bolognani F., et al. Genetic associations of brain structural networks in schizophrenia: a preliminary study. Biol Psychiatry 2010, 68:657-666.
35. Jamadar S., Powers N.R., Meda S.A., Gelernter J., Gruen J.R., Pearlson G.D. Genetic influences of cortical gray matter in language-related regions in healthy controls and schizophrenia. Schizophr Res 2010.
36. Kiebel S.J., Garrido M.I., Moran R.J., Friston K.J. Dynamic causal modelling for EEG and MEG. Cogn Neurodyn 2008, 2:121-136.
37. Kiehl K.A., Liddle P.F. An event-related functional magnetic resonance imaging study of an auditory oddball task in schizophrenia. Schizophr Res 2001, 48:159-171.
38. Kiehl K.A., Stevens M.C., Celone K., Kurtz M., Krystal J.H. Abnormal hemodynamics in schizophrenia during an auditory oddball task. Biol Psychiatry 2005, 57:1029-1040.
39. Kim D.I., Sui J., Rachakonda S., White T., Manoach D.S., Clark V.P., et al. Identification of imaging biomarkers in schizophrenia: a coefficient-constrained independent component analysis of the mind multi-site schizophrenia study. Neuroinformatics 2010.
40. Krishnan A., Williams L.J., McIntosh A.R., Abdi H. Partial Least Squares (PLS) methods for neuroimaging: a tutorial and review. NeuroImage 2010, 56:455-475.
41. Lemieux L. Electroencephalography-correlated functional MR imaging studies of epileptic activity. Neuroimaging Clin N Am 2004, 14:487-506.
42. Li Y.O., Adali T., Wang W., Calhoun V.D. Joint blind source separation by multi-set canonical correlation analysis. IEEE Trans Signal Process 2009, 57:3918-3929.
43. Lin F.H., McIntosh A.R., Agnew J.A., Eden G.F., Zeffiro T.A., Belliveau J.W. Multivariate analysis of neuronal interactions in the generalized partial least squares framework: simulations and empirical studies. NeuroImage 2003, 20:625-642.
44. Liu J., Bixler J.N. V.D.C. A multimodality ICA study-integrating genomic single nucleotid polymorphisms with functional neuroimaging data. Proceedings of the bioinformatics and biomedicine (BIBM) 2008, 151-157.
45. Liu J., Calhoun V.D. Parallel independent component analysis for multimodal analysis: application to fMRI and EEG data. Proceedings of IEEE ISBI conference 2007, 1028-1231.
46. Liu J., Pearlson G., Windemuth A., Ruano G., Perrone-Bizzozero N.I., Calhoun V. Combining fMRI and SNP data to investigate connections between brain function and genetics using parallel ICA. Hum Brain Mapp 2009, 30:241-255.
47. Liu J., Xu L., Caprihan A., Calhoun V.D. Extracting principle components for discriminant analysis of fmri images. Proceedings of IEEE int conf acoust speech signal process 2008, 449-452.
48. Martinez-Montes E., Valdes-Sosa P.A., Miwakeichi F., Goldman R.I., Cohen M.S. Concurrent EEG/fMRI analysis by multiway partial least squares. NeuroImage 2004, 22:1023-1034.
49. McIntosh A.M., Maniega S.M., Lymer G.K., McKirdy J., Hall J., Sussmann J.E., et al. White matter tractography in bipolar disorder and schizophrenia. Biol Psychiatry 2008, 64:1088-1092.
50. McIntosh A.M., Moorhead T.W., Job D., Lymer G.K., Munoz Maniega S., McKirdy J., et al. The effects of a neuregulin 1 variant on white matter density and integrity. Mol Psychiatry 2008, 13:1054-1059.
51. McIntosh A.R., Lobaugh N.J. Partial least squares analysis of neuroimaging data: applications and advances. NeuroImage 2004, 23(Suppl. 1):S250-S263.
52. Meda S.A., Jagannathan K., Gelernter J., Calhoun V.D., Liu J., Stevens M.C., et al. A pilot multivariate parallel ICA study to investigate differential linkage between neural networks and genetic profiles in schizophrenia. NeuroImage 2010, 53:1007-1015.
53. Michael A.M., Baum S.A., White T., Demirci O., Andreasen N.C., Segall J.M., et al. Does function follow form?: methods to fuse structural and functional brain images show decreased linkage in schizophrenia. NeuroImage 2010, 49:2626-2637.
54. Oakes T.R., Fox A.S., Johnstone T., Chung M.K., Kalin N., Davidson R.J. Integrating VBM into the general linear model with voxelwise anatomical covariates. Neuroimage 2007, 34:500-508.
55. Plis S.M., Calhoun V.D., Weisend M.P., Eichele T., Lane T. MEG and fMRI fusion for non-linear estimation of neural and BOLD signal changes. Front Neuroinformatics 2010, 4:114.
56. Plis S.M., Weisend M.P., Damaraju E., Eichele T., Mayer A., Clark V.P., et al. Effective connectivity analysis of fMRI and MEG data collected under identical paradigms. Comput Biol Med 2011.
57. Ramnani N., Lee L., Mechelli A., Phillips C., Roebroeck A., Formisano E. Exploring brain connectivity: a new frontier in systems neuroscience. functional brain connectivity, 4-6 April 2002, Dusseldorf, Germany. Trends Neurosci. 2002, 25:496-497.
58. Rykhlevskaia E., Gratton G., Fabiani M. Combining structural and functional neuroimaging data for studying brain connectivity: a review. Psychophysiology 2008, 45:173-187.
59. Schlosser R., Gesierich T., Kaufmann B., Vucurevic G., Hunsche S., Gawehn J., et al. Altered effective connectivity during working memory performance in schizophrenia: a study with fMRI and structural equation modeling. Neuroimage 2003, 19:751-763.
60. Shenton M.E., Dickey C.C., Frumin M., McCarley R.W. A review of MRI findings in schizophrenia. Schizophr Res 2001, 49:1-52.
61. Skudlarski P., Jagannathan K., Anderson K., Stevens M.C., Calhoun V.D., Skudlarska B.A., et al. Brain connectivity is not only lower but different in schizophrenia: a combined anatomical and functional approach. Biol Psychiatry 2010, 68:61-69.
62. Smith S.M., Fox P.T., Miller K.L., Glahn D.C., Fox P.M., Mackay C.E., et al. Correspondence of the brain's functional architecture during activation and rest. Proc Natl Acad Sci U S A 2009, 106:13040-13045.
63. Specht K., Zahn R., Willmes K., Krause B.J., Herzog H., Huber W. Tracing the recovery of aphasia with a joint ICA of functional and structural data 2008, OHBM, Melbourne, Australia.
64. Sui J., Adali T., Li Y.O., Yang H., Calhoun D.C. A review of multivariate methods in brain imaging data fusion. SPIE medical imaging conference 2010.
65. Sui J., Adali T., Pearlson G.D., Calhoun V.D. An ICA-based method for the identification of optimal FMRI features and components using combined group-discriminative techniques. NeuroImage 2009, 46:73-86.
66. Sui J., Adali T., Pearlson G.D., Clark V.P., Calhoun V.D. A method for accurate group difference detection by constraining the mixing coefficients in an ICA framework. Hum Brain Mapp 2009, 30:2953-2970.
67. Sui J., Pearlson G., Caprihan A., Adali T., Kiehl K.A., Liu J., et al. Discriminating schizophrenia and bipolar disorder by fusing fMRI and DTI in a multimodal CCA+ joint ICA model. NeuroImage 2011, 57:839-855.
68. Sussmann J.E., Lymer G.K., McKirdy J., Moorhead T.W., Maniega S.M., Job D., et al. White matter abnormalities in bipolar disorder and schizophrenia detected using diffusion tensor magnetic resonance imaging. Bipolar Disord 2009, 11:11-18.
69. Teipel S.J., Bokde A.L., Meindl T., Amaro E., Soldner J., Reiser M.F., et al. White matter microstructure underlying default mode network connectivity in the human brain. NeuroImage 2010, 49:2021-2032.
70. Thomos N., Boulgouris N.V., Strintzis M.G. Optimized transmission of JPEG2000 streams over wireless channels. IEEE Trans Image Process 2006, 15.
71. Tregellas J.R., Tanabe J.L., Miller D.E., Ross R.G., Olincy A., Freedman R. Neurobiology of smooth pursuit eye movement deficits in schizophrenia: an fMRI study. Am J Psychiatry 2004, 161:315-321.
72. Valdes-Sosa P.A. Spatio-temporal autoregressive models defined over brain manifolds. Neuroinformatics 2004, 2:239-250.
73. Valdes-Sosa P.A., Sanchez-Bornot J.M., Sotero R.C., Iturria-Medina Y., Aleman-Gomez Y., Bosch-Bayard J., et al. Model driven EEG/fMRI fusion of brain oscillations. Hum Brain Mapp 2009, 30:2701-2721.
74. Worsley K.J., Cao J., Paus T., Petrides M., Evans A.C. Applications of random field theory to functional connectivity. Hum Brain Mapp 1998, 6:364-367.
75. Wu L., Eichele T., Calhoun V.D. Reactivity of hemodynamic responses and functional connectivity to different states of alpha synchrony: a concurrent EEG-fMRI study. NeuroImage 2010, 52:1252-1260.
76. Xu L., Pearlson G., Calhoun V.D. Joint source based morphometry identifies linked gray and white matter group differences. NeuroImage 2009, 44:777-789.
77. Yang H., Liu J., Sui J., Pearlson G., Calhoun V.D. A hybrid machine learning method for fusing fMRI and genetic data: combining both improves classification of schizophrenia. Front Hum Neurosci 2010, 4:192.