Hierarchical feature representation and multimodal fusion with deep learning for AD/MCI diagnosis

NeuroImage

Volumn 101, Issue , 2014, Pages 569-582

  Suk, Heung Il ^a   Lee, Seong Whan ^b   Shen, Dinggang ^a,b  

^a UNIVERSITY OF NORTH CAROLINA SCHOOL OF MEDICINE   (United States)

^b Korea University   (South Korea)

Author keywords

Alzheimer's Disease; Deep boltzmann machine; Mild cognitive impairment; Multimodal data fusion; Shared feature representation

Indexed keywords

FLUORODEOXYGLUCOSE F 18;

ADULT; AGED; ALZHEIMER DISEASE; ARTICLE; CLASSIFIER; COMPUTER ASSISTED DIAGNOSIS; CONTROLLED STUDY; DIAGNOSTIC ACCURACY; DIAGNOSTIC TEST ACCURACY STUDY; FALSE NEGATIVE RESULT; FALSE POSITIVE RESULT; FEMALE; HUMAN; IMAGE PROCESSING; INTERMETHOD COMPARISON; MACHINE LEARNING; MAJOR CLINICAL STUDY; MALE; MATHEMATICAL MODEL; MILD COGNITIVE IMPAIRMENT; MULTIMODAL DEEP BOLTZMANN MACHINE; NUCLEAR MAGNETIC RESONANCE IMAGING; NUCLEAR MAGNETIC RESONANCE SCANNER; POSITRON EMISSION TOMOGRAPHY; PREDICTIVE VALUE; RECEIVER OPERATING CHARACTERISTIC; SENSITIVITY AND SPECIFICITY; VALIDATION STUDY; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; MIDDLE AGED; MULTIMODAL IMAGING; PROCEDURES; VERY ELDERLY;

AGED; AGED, 80 AND OVER; ALZHEIMER DISEASE; ARTIFICIAL INTELLIGENCE; FEMALE; HUMANS; IMAGE PROCESSING, COMPUTER-ASSISTED; MAGNETIC RESONANCE IMAGING; MALE; MIDDLE AGED; MILD COGNITIVE IMPAIRMENT; MULTIMODAL IMAGING; NEURAL NETWORKS (COMPUTER); POSITRON-EMISSION TOMOGRAPHY;

EID: 84907019192     PISSN: 10538119     EISSN: 10959572     Source Type: Journal    
DOI: 10.1016/j.neuroimage.2014.06.077     Document Type: Article

References (78)

1. Alzheimer's Association 2012 Alzheimer's disease facts and figures. Alzheimers Dement. 2012, 8:131-168.
2. Baron J., Chtelat G., Desgranges B., Perchey G., Landeau B., de la Sayette V., Eustache F. In vivo mapping of gray matter loss with voxel-based morphometry in mild Alzheimer's disease. Neuroimage 2001, 14:298-309.
3. Belleville S., ClZˇment F., Mellah S., Gilbert B., Fontaine F., Gauthier S. Training-related brain plasticity in subjects at risk of developing Alzheimer's disease. Brain 2011, 134:1623-1634.
4. Bengio Y. Learning deep architectures for AI. Foundations and Trends in Machine Learning 2009, 2:1-127.
5. Bengio Y., Lamblin P., Popovici D., Larochelle H. Greedy layer-wise training of deep networks. Advances in Neural Information Processing Systems 2007, 19:153-160. MIT Press, Cambridge, MA. B. Scho¨lkopf, J. Platt, T. Hoffman (Eds.).
6. Bokde A.L.W., Lopez-Bayo P., Meindl T., Pechler S., Born C., Faltraco F., Teipel S.J., Mo¨ller H.J., Hampel H. Functional connectivity of the fusiform gyrus during a face-matching task in subjects with mild cognitive impairment. Brain 2006, 129:1113-1124.
7. Braak H., Braak E. Neuropathological stageing of Alzheimer-related changes. Acta Neuropathol. 1991, 82:239-259.
8. Burton E.J., Barber R., Mukaetova-Ladinska E.B., Robson J., Perry R.H., Jaros E., Kalaria R.N., OBrien J.T. Medial temporal lobe atrophy on MRI differentiates Alzheimer's disease from dementia with Lewy bodies and vascular cognitive impairment: a prospective study with pathological verification of diagnosis. Brain 2009, 132:195-203.
9. Catana C., Drzezga A., Heiss W.D., Rosen B.R. PET/MRI for neurologic applications. J. Nucl. Med. 2012, 53:1916-1925.
10. Ciresan D.C., Giusti A., Gambardella L.M., Schmidhuber J. Mitosis detection in breast cancer histology images with deep neural networks. Medical Image Computing and Computer-Assisted Intervention MICCAI 2013 2013, 411-418.
11. Cui Y., Liu B., Luo S., Zhen X., Fan M., Liu T., Zhu W., Park M., Jiang T., Jin J.S., the Alzheimer's Disease Neuroimaging Initiative Identification of conversion from mild cognitive impairment to Alzheimer's disease using multivariate predictors. PLoS One 2011, 6:e21896.
12. Cuingnet R., Gerardin E., Tessieras J., Auzias G., Lehe´ricy S., Habert M.O., Chupin M., Benali H., Colliot O., The Alzheimer's Disease Neuroimaging Initiative Automatic classification of patients with Alzheimer's disease from structural MRI: a comparison of ten methods using ADNI database. Neuroimage 2011, 56:766-781.
13. Dai W., Lopez O., Carmichael O., Becker J., Kuller L., Gach H. Mild cognitive impairment and Alzheimer disease: patterns of altered cerebral blood flow at MR imaging. Radiology 2009, 250:856-866.
14. Davatzikos C., Genc A., Xu D., Resnick S.M. Voxel-based morphometry using the RAVENS maps: methods and validation using simulated longitudinal atrophy. Neuroimage 2001, 14:1361-1369.
15. Davatzikos C., Bhatt P., Shaw L.M., Batmanghelich K.N., Trojanowski J.Q. Prediction of MCI to AD conversion, via MRI, CSF biomarkers, and pattern classification. Neurobiol. Aging 2011, 32. (2322.e19-2322.e27).
16. de Jong L.W., van der Hiele K., Veer I.M., Houwing J.J., Westendorp R.G.J., Bollen E.L.E.M., de Bruin P.W., Middelkoop H.A.M., van Buchem M.A., van der Grond J. Strongly reduced volumes of putamen and thalamus in Alzheimer's disease: an MRI study. Brain 2008, 131:3277-3285.
17. Desikan R., Cabral H., Hess C., Dillon W., Salat D., Buckner R., Fischl B., Initiative A.D.N. Automated MRI measures identify individuals with mild cognitive impairment and Alzheimer's disease. Brain 2009, 132:2048-2057.
18. Devanand D.P., Pradhaban G., Liu X., Khandji A., De Santi S., Segal S., Rusinek H., Pelton G.H., Hoing L.S., Mayeux R., Stern Y., Tabert M.H., de Leon J.J. Hippocampal and entorhinal atrophy in mild cognitive impairment. Neurology 2007, 68:828-836.
19. Dinov I., Boscardin J., Mega M., Sowell E., Toga A. A wavelet-based statistical analysis of fMRI data. Neuroinformatics 2005, 3:319-342.
20. Ewers M., Walsh C., Trojanowski J.Q., Shaw L.M., Petersen R.C., C.R.J., Feldman H.H., Bokde A.L., Alexander G.E., Scheltens P., Vellas B., Dubois B., Weiner M., Hampel H. Prediction of conversion from mild cognitive impairment to Alzheimer's disease dementia based upon biomarkers and neuropsychological test performance. Neurobiol. Aging 2012, 33:1203-1214. (e2).
21. Fan Y., Rao H., Hurt H., Giannetta J., Korczykowski M., Shera D., Avants B.B., Gee J.C., Wang J., Shen D. Multivariate examination of brain abnormality using both structural and functional MRI. Neuroimage 2007, 36:1189-1199.
22. Fan Y., Shen D., Gur R., Gur R., Davatzikos C. COMPARE: classification of morphological patterns using adaptive regional elements. IEEE Trans. Med. Imaging 2007, 26:93-105.
23. Friston K.J. Functional and effective connectivity in neuroimaging: a synthesis. Hum. Brain Mapp. 1995, 2:56-78.
24. Greicius M.D., Srivastava G., Reiss A.L., Menon V. Default-mode network activity distinguishes Alzheimer's disease from healthy aging: evidence from functional MRI. Proc. Natl. Acad. Sci. U. S. A. 2004, 101:4637-4642.
25. Hackmack K., Paul F., Weygandt M., Allefeld C., Haynes J.D. Multi-scale classification of disease using structural MRI and wavelet transform. Neuroimage 2012, 62:48-58.
26. Hinrichs C., Singh V., Xu G., Johnson S.C. Predictive markers for AD in a multi-modality framework: an analysis of MCI progression in the ADNI population. Neuroimage 2011, 55:574-589.
27. Hinton G.E., Salakhutdinov R.R. Reducing the dimensionality of data with neural networks. Science 2006, 313:504-507.
28. Hinton G.E., Osindero S., Teh Y.W. A fast learning algorithm for deep belief nets. Neural Comput. 2006, 18:1527-1554.
29. Hjelm R.D., Calhoun V.D., Salakhutdinov R., Allen E.A., Adali T., Plis S.M. Restricted boltzmann machines for neuroimaging: an application in identifying intrinsic networks. Neuroimage 2014, 96:245-260.
30. Ishii K., Kawachi T., Sasaki H., Kono A.K., Fukuda T., Kojima Y., Mori E. Voxel-based morphometric comparison between early- and late-onset mild Alzheimer's disease and assessment of diagnostic performance of z score images. Am. J. Neuroradiol. 2005, 26:333-340.
31. Jia H., Wu G., Wang Q., Shen D. ABSORB: atlas building by self-organized registration and bundling. Neuroimage 2010, 51:1057-1070.
32. Johnson N.A., Jahng G.H., Weiner M.W., Miller B.L., Chui H.C., Jagust W.J., Gorno-Tempini M.L., Schuff N. Pattern of cerebral hypoperfusion in Alzheimer disease and mild cognitive impairment measured with arterial spin-labeling MR imaging: initial experience. Radiology 2005, 234:851-859.
33. Kabani N., MacDonald D., Holmes C., Evans A. A 3D atlas of the human brain. Neuroimage 1998, 7:S717.
34. Kohannim O., Hua X., Hibar D.P., Lee S., Chou Y.Y., Toga A.W., C.R.J., Weiner M.W., Thompson P.M. Boosting power for clinical trials using classifiers based on multiple biomarkers. Neurobiol. Aging 2010, 31:1429-1442.
35. Larochelle H., Bengio Y. Classification using discriminative restricted Boltzmann machines. Proceedings of the 25th International Conference on Machine Learning 2008, 536-543.
36. LeCun Y., Bottou L., Bengio Y., Haffner P. Gradient-based learning applied to document recognition. Proc. IEEE 1998, 86:2278-2324.
37. Lee A.C.H., Buckley M.J., Gaffan D., Emery T., Hodges J.R., Graham K.S. Differentiating the roles of the hippocampus and perirhinal cortex in processes beyond long-term declarative memory: a double dissociation in dementia. J. Neurosci. 2006, 26:5198-5203.
38. Lee H., Grosse R., Ranganath R., Ng A.Y. Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. Proceedings of the 26th International Conference on Machine Learning 2009, 609-616.
39. Li Y., Wang Y., Wu G., Shi F., Zhou L., Lin W., Shen D. Discriminant analysis of longitudinal cortical thickness changes in Alzheimer's disease using dynamic and network features. Neurobiol. Aging 2012, 33. (427.e15-427.e30).
40. Liao S., Gao Y., Oto A., Shen D. Representation learning: a unified deep learning framework for automatic prostate MR segmentation. Medical Image Computing and Computer-Assisted Intervention MICCAI 2013. Lecture Notes in Computer Science 2013, vol. 8150:254-261.
41. Liu M., Zhang D., Shen D. Ensemble sparse classification of Alzheimer's disease. Neuroimage 2012, 60:1106-1116.
42. Liu M., Zhang D., Shen D., the Alzheimer's Disease Neuroimaging Initiative Hierarchical fusion of features and classifier decisions for Alzheimer's disease diagnosis. Hum. Brain Mapp. 2013, 35:1305-1319.
43. Loewenstein D.A., Greig M.T., Schinka J.A., Barker W., Shen Q., Potter E., Raj A., Brooks L., Varon D., Schoenberg M., Banko J., Potter H., Duara R. An investigation of PreMCI: subtypes and longitudinal outcomes. Alzheimers Dement. 2012, 8:172-179.
44. Mark R.E., Sitskoorn M.M. Are subjective cognitive complaints relevant in preclinical Alzheimer's disease? A review and guidelines for healthcare professionals. Rev. Clin. Gerontol. 2013, 23:61-74.
45. Mohamed A., Dahl G.E., Hinton G.E. Acoustic modeling using deep belief networks. IEEE Trans. Audio Speech Lang. Process. 2012, 20:14-22.
46. Montavon G., Braun M.L., Mo¨ller K.R. Deep Boltzmann machines as feed-forward hierarchies. Journal of Machine Learning Research-Proceedings Track 2012, 22:798-804.
47. Mosconi L. Brain glucose metabolism in the early and specific diagnosis of Alzheimer's disease. Eur. J. Nucl. Med. Mol. Imaging 2005, 32:486-510.
48. Nair V., Hinton G.E. Implicit mixtures of restricted Boltzmann machines. Advances in Neural Information Processing Systems 2008, 1145-1152.
49. Ngiam J., Khosla A., Kim M., Nam J., Lee H., Ng A.Y. Multimodal deep learning. Proceedings of the 28th International Conference on Machine Learning 2011, 689-696.
50. Nordberg A., Rinne J.O., Kadir A., Langstrom B. The use of PET in Alzheimer disease. Nat. Rev. Neurol. 2010, 6:78-87.
51. Perrin R.J., Fagan A.M., Holtzman D.M. Multimodal techniques for diagnosis and prognosis of Alzheimer's disease. Nature 2009, 461:916-922.
52. Pichler B.J., Kolb A., Na¨gele T., Schlemmer H.P. PET/MRI: paving the way for the next generation of clinical multimodality imaging applications. J. Nucl. Med. 2010, 51:333-336.
53. Salakhutdinov R., Hinton G.E. Deep Boltzmann machines. Proceedings of the International Conference on Artificial Intelligence and, Statistics 2009, 448-455.
54. Salakhutdinov R., Hinton G. An efficient learning procedure for deep Boltzmann machines. Neural Comput. 2012, 24:1967-2006.
55. Shen D., Davatzikos C. HAMMER: hierarchical attribute matching mechanism for elastic registration. IEEE Trans. Med. Imaging 2002, 21:1421-1439.
56. Shin H.C., Orton M.R., Collins D.J., Doran S.J., Leach M.O. Stacked autoencoders for unsupervised feature learning and multiple organ detection in a pilot study using 4D patient data. IEEE Trans. Pattern Anal. Mach. Intell. 2013, 35:1930-1943.
57. Singh V., Chertkow H., Lerch J.P., Evans A.C., Dorr A.E., Kabani N.J. Spatial patterns of cortical thinning in mild cognitive impairment and Alzheimer's disease. Brain 2006, 129:2885-2893.
58. Sled J.G., Zijdenbos A.P., Evans A.C. A nonparametric method for automatic correction of intensity nonuniformity in MRI data. IEEE Trans. Med. Imaging 1998, 17:87-97.
59. Srivastava N., Salakhutdinov R. Multimodal learning with deep Boltzmann machines. Advances in Neural Information Processing Systems 2012, 25:2231-2239.
60. Suk H.I., Shen D. Deep learning-based feature representation for AD/MCI classification. Medical Image Computing and Computer-Assisted Intervention-MICCAI 2013. Lecture Notes in Computer Science 2013, vol. 8150:583-590.
61. Suk H.I., Wee C.Y., Shen D. Discriminative group sparse representation for mild cognitive impairment classification. Machine Learning in Medical Imaging. Lecture Notes in Computer Science 2013, vol. 8184:131-138.
62. Tanaka T. A theory of mean field approximation. Advances in Neural Information Processing Systems (NIPS) 1998, 351-360. The MIT Press.
63. Tang S., Fan Y., Wu G., Kim M., Shen D. RABBIT: rapid alignment of brains by building intermediate templates. Neuroimage 2009, 47:1277-1287.
64. Visser P.J., Verhey F.R.J., Hofman P.A.M., Scheltens P., Jolles J. Medial temporal lobe atrophy predicts Alzheimer's disease in patients with minor cognitive impairment. J. Neurol. Neurosurg. Psychiatry 2002, 72:491-497.
65. Walhovd K., Fjell A., Brewer J., McEvoy L., Fennema-Notestine C., D.H., Jennings R., Karow D., Dale A., the Alzheimer's Disease Neuroimaging Initiative Combining MR imaging, positron-emission tomography, and CSF biomarkers in the diagnosis and prognosis of Alzheimer disease. Am. J. Neuroradiol. 2010, 31:347-354.
66. Wang Y., Nie J., Yap P.T., Li G., Shi F., Geng X., Guo L., Shen D. Knowledge-guided robust MRI brain extraction for diverse large-scale neuroimaging studies on humans and non-human primates. PLoS One 2014, 9:e77810.
67. Ward A., Arrighi H.M., Michels S., Cedarbaum J.M. Mild cognitive impairment: disparity of incidence and prevalence estimates. Alzheimers Dement. 2012, 8:14-21.
68. Wee C.Y., Yap P.T., Li W., Denny K., Browndyke J.N., Potter G.G., Welsh-Bohmer K.A., Wang L., Shen D. Enriched white matter connectivity networks for accurate identification of MCI patients. Neuroimage 2011, 54:1812-1822.
69. Wee C.Y., Yap P.T., Zhang D., Denny K., Browndyke J.N., Potter G.G., Welsh-Bohmer K.A., Wang L., Shen D. Identification of MCI individuals using structural and functional connectivity networks. Neuroimage 2012, 59:2045-2056.
70. Westman E., Muehlboeck J.S., Simmons A. Combining MRI and CSF measures for classification of Alzheimer's disease and prediction of mild cognitive impairment conversion. Neuroimage 2012, 62:229-238.
71. Xue Z., Shen D., Davatzikos C. Statistical representation of high-dimensional deformation fields with application to statistically constrained 3D warping. Med. Image Anal. 2006, 10:740-751.
72. Yang J., Shen D., Davatzikos C., Verma R. Diffusion tensor image registration using tensor geometry and orientation features. Medical Image Computing and Computer-assisted Intervention - MICCAI 2008 2008, 905-913. Springer, Berlin Heidelberg. D. Metaxas, L. Axel, G. Fichtinger, G. SzZˇkely (Eds.).
73. Yuan L., Wang Y., Thompson P.M., Narayan V.A., Ye J. Multi-source feature learning for joint analysis of incomplete multiple heterogeneous neuroimaging data. Neuroimage 2012, 61:622-632.
74. Zhang D., Shen D. Multi-modal multi-task learning for joint prediction of multiple regression and classification variables in Alzheimer's disease. Neuroimage 2012, 59:895-907.
75. Zhang Y., Brady M., Smith S. Segmentation of brain MR images through a hidden Markov random field model and the expectation-maximization algorithm. IEEE Trans. Med. Imaging 2001, 20:45-57.
76. Zhang D., Wang Y., Zhou L., Yuan H., Shen D. Multimodal classification of Alzheimer's disease and mild cognitive impairment. Neuroimage 2011, 55:856-867.
77. Zhang D., Shen D., Alzheimer's Disease Neuroimaging I. Predicting future clinical changes of MCI patients using longitudinal and multimodal biomarkers. PLoS One 2012, 7:e33182.
78. Zhou L., Wang Y., Li Y., Yap P.T., Shen D., the Alzheimer's Disease Neuroimaging I. Hierarchical anatomical brain networks for MCI prediction: revisiting volumetric measures. PLoS One 2011, 6:e21935.