Multimodal Data Fusion Using Source Separation: Application to Medical Imaging

Proceedings of the IEEE

Volumn 103, Issue 9, 2015, Pages 1494-1506

  Adali, Tulay ^a   Levin Schwartz, Yuri ^a   Calhoun, Vince D ^b,c  

^a Baltimore County   (United States)

^b UNIVERSITY OF NEW MEXICO   (United States)

^c MIND RESEARCH NETWORK   (United States)

Author keywords

Data fusion; electroencephalography (EEG); functional magnetic resonance imaging (fMRI); independent

Indexed keywords

BLIND SOURCE SEPARATION; DATA FUSION; ELECTROENCEPHALOGRAPHY; ELECTROPHYSIOLOGY; INDEPENDENT COMPONENT ANALYSIS; MAGNETIC RESONANCE IMAGING;

DENSITY MATCHING; EFFECTIVE SOLUTION; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HEALTHY CONTROLS; INDEPENDENT; INDEPENDENT VECTOR ANALYSIS; MULTIMODAL DATA FUSION; MULTIPLE MODALITIES;

MEDICAL IMAGING;

EID: 84940384892     PISSN: 00189219     EISSN: 15582256     Source Type: Journal    
DOI: 10.1109/JPROC.2015.2461601     Document Type: Article

References (57)

1. T. AdalN, Y. Levin-Schwartz, and V. D. Calhoun, "Multi-modal data fusion using source separation: Two effective models based on ICA and IVA and their properties," Proc. IEEE, vol. 103, no. 9, Sep. 2015, DOI: 10. 1109/JPROC.2015.2461624.
2. A. P. James and B. V. Dasarthy, "Medical image fusion: A survey of the state of the art," Inf. Fusion, vol. 19, pp. 4-19, 2014.
3. V. D. Calhoun and T. AdalN, "Feature-based fusion of medical imaging data," IEEE Trans. Inf. Technol. Biomed., vol. 13, no. 5, pp. 711-720, Sep. 2009.
4. E. Acar, M. A. Rasmussen, F. Savorani, T. Naes, and R. Bro, "Understanding data fusion within the framework of coupled matrix and tensor factorizations," Chemometrics Intell. Lab. Syst., vol. 129, pp. 53-63, 2013.
5. I. V. Mechelen and A. K. Smilde, "A generic linked-mode decomposition model for data fusion," Chemometrics Intell. Lab. Syst., vol. 104, pp. 83-94, 2010.
6. K. Kreutz-Delgado et al., "Dictionary learning algorithms for sparse representation," Neural Comput., vol. 15, no. 2, pp. 349-396, 2003.
7. D. D. Lee and H. S. Seung, "Algorithms for non-negative matrix factorization," in Advances in Neural Information Processing Systems, vol. 13. Cambridge, MA, USA: MIT Press, 2000.
8. M. Remazani, K. Marble, H. Trang, I. S. Johnsrude, and P. Abolmaesumi, "Joint sparse representation of brain activity patterns in multi-task fMRI data," IEEE Trans. Med. Imaging, vol. 34, no. 1, pp. 2-12, Jan. 2015.
9. A. Anderson et al., "Non-negative matrix factorization of multimodal MRI, fMRI and phenotypic data reveals differential changes in default mode subnetworks in ADHD," NeuroImage, vol. 102, no. 1, pp. 207-219, 2014.
10. J. Sui, T. AdalN, Q. Yu, J. Chen, and V. Calhoun, "A review of multivariate methods for multimodal fusion of brain imaging data," J. Neurosci. Methods, vol. 204, pp. 68-81, 2012.
11. V. D. Calhoun, T. AdalN, G. D. Pearlson, and K. A. Kiehl, "Neuronal chronometry of target detection: Fusion of hemodynamic and event-related potential data," NeuroImage, vol. 30, no. 2, pp. 544-553, Apr. 2006.
12. Fusion ICA Toolbox. [Online]. Available: http://mialab.mrn.org/software/fit/
13. T. Eichele et al., "Unmixing concurrent EEG-fMRI with parallel independent component analysis," Int. J. Psychophysiol., vol. 67, no. 3, pp. 222-234, 2008.
14. B. Mijovic et al., "The 'why' and 'how' of JointICA: Results from a visual detection task," NeuroImage, vol. 60, pp. 1171-1185, 2012.
15. B. Mijovic et al., "The dynamics of contour integration: A simultaneous EEG-fMRI study," NeuroImage, vol. 88, pp. 10-21, 2014.
16. J. Mangalathu-Arumana, S. Beardsley, and E. Liebenthal, "Within-subject joint independent component analysis of simultaneous fMRI/ERP in an auditory oddball paradigm," NeuroImage, vol. 60, pp. 2247-2257, 2012.
17. W. Swinnen, B. Hunyadi, E. Acar, S. V. Huffel, and M. D. Vos, "Incorporating higher dimensionality in joint decomposition of EEG and fMRI," in Proc. Eur. Signal Process. Conf., Lisbon, Portugal, Sep. 2014, pp. 121-125.
18. V. D. Calhoun, J. Liu, and T. AdalN, "A review of group ICA for fMRI data and ICA for joint inference of imaging, genetic, ERP data," NeuroImage, vol. 45, no. 1, pp. S163-S172, Mar. 2009.
19. V. D. Calhoun et al., "Method for multimodal analysis of independent source differences in schizophrenia: Combining gray matter structural and auditory oddball functional data," Human Brain Mapping, vol. 27, pp. 47-62, 2006.
20. K. Magtibay et al., "Fusion of structural and functional cardiac magnetic resonance imaging data for studying ventricular fibrillation," in Proc. IEEE Int. Conf. Eng. Med. Biol. Soc., 2014, pp. 5579-5582.
21. Z. Yuan and J.-C. Ye, "Fusion of fNIRS and fMRI data: Identifying when and where hemodynamic signals are changing in human brains," Front. Human Neurosci., vol. 7, Oct. 2013, DOI: 10.3389/fnhum.2013.00676.
22. J. Sui et al., "Discriminating schizophrenia and bipolar disorder by fusing fMRI and DTI in a multimodal CCA+ joint ICA model," NeuroImage, vol. 57, no. 3, pp. 839-855, 2011.
23. X. Chen, Z. J. Wang, and M. J. McKeown, "A three-step multimodal analysis framework for modeling corticomuscular activity with application to Parkinson's disease," IEEE J. Biomed. Health Inf., vol. 18, no. 4, pp. 1232-1241, Jul. 2014.
24. K. Specht et al., "Joint independent component analysis of structural and functional images reveals complex patterns of functional reorganisation in stroke aphasia," NeuroImage, vol. 47, pp. 2057-2063, 2009.
25. H. Hotelling, "Relations between two sets of variates," Biometrika, vol. 28, no. 3/4, pp. 321-377, Dec. 1936.
26. P. Geladi and B. R. Kowalski, "Partial least-squares regression: A tutorial," Analytica Chimica Acta, vol. 185, pp. 1-17, 1986.
27. N. M. Correa, Y.-O. Li, T. AdalN, and V. D. Calhoun, "Canonical correlation analysis for feature-based fusion of biomedical imaging modalities and its application to detection of associative networks in schizophrenia," IEEE J. Sel. Top. Signal Process., vol. 2, no. 6, pp. 998-1007, Jun. 2009.
28. N. Correa, T. AdalN, and V. D. Calhoun, "Canonical correlation analysis for data fusion and group inferences: Examining applications of medical imaging data," IEEE Signal Process. Mag., vol. 27, no. 4, pp. 39-50, Jul. 2010.
29. A. Krishnan, L. J. Williamd, A. R. McIntosh, and H. Abdi, "Partial least squares (PLS) methods for neuroimaging: A tutorial and review," NeuroImage, vol. 56, pp. 455-475, 2011.
30. E. Martinez-Montes, P. A. Valdés-Sosa, A. Miwakeichi, R. I. Goldman, and M. S. Cohen, "Concurrent EEG/fMRI analysis by multiway partial least squares," NeuroImage, vol. 22, pp. 1023-1034, 2004.
31. J. R. Kettenring, "Canonical analysis of several sets of variables," Biometrika, vol. 58, no. 3, pp. 433-451, Dec. 1971.
32. N. M. Correa, Y.-O. Li, T. AdalN, and V. D. Calhoun, "Fusion of fMRI, sMRI, EEG data using canonical correlation analysis," in Proc. IEEE Int. Conf. Acoust. Speech Signal Process., 2009, pp. 385-388.
33. N. M. Correa, T. Eichele, T. AdalN, Y.-O. Li, and V. D. Calhoun, "Multi-set canonical correlation analysis for the fusion of concurrent single trial ERP and functional MRI," NeuroImage, vol. 50, no. 4, pp. 1438-1445, May 2010.
34. J. Sui et al., "Three-way (N-way) fusion of brain imaging data based on mCCA+jICA and its application to discriminating schizophrenia," NeuroImage, vol. 66, pp. 119-132, Feb. 2013.
35. V. M. Vergara et al., "A three-way parallel ICA approach to analyze links among genetics, brain structure and brain function," NeuroImage, vol. 98, pp. 386-394, Sep. 2014.
36. J. Sui et al., "A CCA + ICA based model for multi-task brain imaging data fusion and its application to schizophrenia," NeuroImage, vol. 51, no. 1, pp. 123-134, May 2010.
37. X.-L. Li and T. AdalN, "Independent component analysis by entropy bound minimization," IEEE Trans. Signal Process., vol. 58, no. 10, pp. 5151-5164, Oct. 2010.
38. A. Bell and T. Sejnowski, "An information maximization approach to blind separation and blind deconvolution," Neural Comput., vol. 7, no. 6, pp. 1129-1159, Nov. 1995.
39. K. A. Kiehl et al., "An adaptive reflexive processing model of neurocognitive function: Supporting evidence from a large scale n (14) 100 fMRI study of an auditory oddball task," NeuroImage, vol. 25, pp. 899-915, 2005.
40. K. Friston, SPM8, Apr. 2009. [Online]. Available: http://www.fil.ion.ucl.ac.uk/spm/software/spm8/.
41. M. S. Cohen, R. M. DuBois, and M. M. Zeineh, "Rapid and effective correction of RF inhomogeneity for high field magnetic resonance imaging," Human Brain Mapping, vol. 10, pp. 204-211, 2000.
42. G. Gerig et al., "Unsupervised tissue type segmentation of 3D dual-echo MR head data," Image Vis. Comput., vol. 10, no. 6, pp. 349-360, 1992.
43. Y. Zhang, M. Brady, and S. Smith, "Segmentation of brain MR images through a hidden Markov random field model and the expectation-maximization algorithm," IEEE Trans. Med. Imaging, vol. 20, no. 1, pp. 45-57, Jan. 2001.
44. I. W. M. Wells, W. E. L. Grimson, R. Kikinis, and F. A. Jolesz, "Adaptive segmentation of MRI data," IEEE Trans. Med. Imaging, vol. 15, no. 4, pp. 429-442, Aug. 1996.
45. J. C. Bezdek, L. O. Hall, and L. P. Clarke, "Review of MR image segmentation techniques using pattern recognition," Med. Phys., vol. 20, pp. 1033-1048, 1993.
46. M. Anderson, X.-L. Li, and T. AdalN, "Joint blind source separation with multivariate Gaussian model: Algorithms and performance analysis," IEEE Trans. Signal Process., vol. 60, no. 4, pp. 2049-2055, Apr. 2012.
47. Himberg, A. Hyvärinen, and A. Esposito, "Validating the independent components of neuroimaging time-series via clustering and visualization," NeuroImage, vol. 22, pp. 1214-1222, 2004.
48. S. Ma et al., "Automatic identification of functional clusters in fMRI data using spatial dependence," IEEE Trans. Biomed. Eng., vol. 58, no. 12, pp. 3406-3417, Dec. 2011.
49. W. Du, S. Ma, G.-S. Fu, V. D. Calhoun, and T. AdalN, "A novel approach for assessing reliability of ICA for fMRI analysis," in Proc. IEEE Int. Conf. Acoust. Speech Signal Process., Florence, Italy, May 2014, pp. 2084-2088.
50. Group ICA of fMRI Toolbox. [Online]. Available: http://mialab.mrn.org/software/gift/
51. Y.-O. Li, T. AdalN, and V. D. Calhoun, "Estimating the number of independent components for fMRI data," Human Brain Mapping, vol. 28, no. 11, pp. 1251-1266, Nov. 2007.
52. M. Ramezani, P. Abolmaesumi, K. Marble, H. Trang, and I. Johnsrude, "Fusion analysis of functional MRI data for classification of individuals based on patterns of activation," Brain Imaging Behav., vol. 9, pp. 149-161, 2014.
53. Y. Song, P. Schreier, and N. J. Roseveare, "Determining the number of correlated signals between two data sets using PCA-CCA when sample support is extremely small," in Proc. IEEE Int. Conf. Acoust. Speech Signal Process., Brisbane, Australia, Apr. 2015, pp. 5684-5687.
54. V. D. Calhoun and T. AdalN, "Multisubject independent component analysis of fMRI: A decade of intrinsic networks, default mode, neurodiagnostic discovery," IEEE Rev. Biomed. Eng., vol. 5, pp. 60-73, Aug. 2012.
55. A. R. Groves, C. F. Beckmann, S. M. Smith, and M. W. Woolrich, "Linked independent component analysis for multimodal data fusion," NeuroImage, vol. 54, no. 3, pp. 2198-2217, 2011. [Online]. Available: http://dx.doi.org/10.1016/j.neuroimage.2010. 09.073.
56. J. Chen et al., "Guided exploration of genomic risk for gray matter abnormalities in schizophrenia using parallel independent component analysis with reference," NeuroImage, vol. 83, pp. 384-396, Dec. 2013.
57. D. Lahat, T. AdalN, and C. Jutten, "Multimodal data fusion: An overview of methods, challenges, and prospects," Proc. IEEE, vol. 103, no. 9, Sep. 2015, DOI: 10.1109/JPROC.2015.2460697.