Using deep learning to investigate the neuroimaging correlates of psychiatric and neurological disorders: Methods and applications

Neuroscience and Biobehavioral Reviews

Volumn 74, Issue , 2017, Pages 58-75

  Vieira, Sandra ^a   Pinaya, Walter H L ^b   Mechelli, Andrea ^a  

^a KING'S COLLEGE LONDON   (United Kingdom)

^b UNIVERSIDADE FEDERAL DO ABC   (Brazil)

Author keywords

Autoencoders; Convolutional neural networks; Deep belief networks; Deep learning; Machine learning; Multilayer perceptron; Neuroimaging; Neurologic disorders; Pattern recognition; Psychiatric disorders

Indexed keywords

ALZHEIMER DISEASE; ARTIFICIAL NEURAL NETWORK; ATTENTION DEFICIT DISORDER; CEREBELLAR ATAXIA; COMPUTER ASSISTED TOMOGRAPHY; CONTROLLED STUDY; CONVOLUTIONAL NEURAL NETWORK; DEEP LEARNING; DEEP NEURAL NETWORK; DIFFUSION TENSOR IMAGING; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HUMAN; MACHINE LEARNING; MEASUREMENT ACCURACY; MEASUREMENT PRECISION; MILD COGNITIVE IMPAIRMENT; NUCLEAR MAGNETIC RESONANCE IMAGING; POSITRON EMISSION TOMOGRAPHY; PSYCHOSIS; REVIEW; SENSITIVITY AND SPECIFICITY; SUPPORT VECTOR MACHINE; TEMPORAL LOBE EPILEPSY; MENTAL DISEASE; NEUROIMAGING; NEUROLOGIC DISEASE; SPEECH;

HUMANS; MACHINE LEARNING; MENTAL DISORDERS; NERVOUS SYSTEM DISEASES; NEURAL NETWORKS (COMPUTER); NEUROIMAGING; SPEECH;

EID: 85009921542     PISSN: 01497634     EISSN: 18737528     Source Type: Journal    
DOI: 10.1016/j.neubiorev.2017.01.002     Document Type: Review

References (91)

1. Alain, G., Bengio, Y., 2016. Understanding intermediate layers using linear classifier probes. arXiv preprint arXiv:1610.01644.
2. Alberg, A.J., Park, J.W., Hager, B.W., Brock, M.V., Diener‐West, M., The use of overall accuracy to evaluate the validity of screening or diagnostic tests. J. Gen. Intern. Med. 19 (2004), 460–465.
3. Arbabshirani, M.R., Plis, S., Sui, J., Calhoun, V.D., Single subject prediction of brain disorders in neuroimaging: promises and pitfalls. Neuroimage, 2016, 137–165.
4. ® Mach. Learn. 2 (2009), 1–127.
5. Bergstra, J.S., Bardenet, R., Bengio, Y., Kégl, B., Algorithms for hyper-parameter optimization. Adv. Neural Inf. Process. Syst., 2011, 2546–2554.
6. Biswal, B.B., Mennes, M., Zuo, X.N., Gohel, S., Kelly, C., Smith, S.M., Beckmann, C.F., Adelstein, J.S., Buckner, R.L., Colcombe, S., Dogonowski, A.M., Ernst, M., Fair, D., Hampson, M., Hoptman, M.J., Hyde, J.S., Kiviniemi, V.J., Kotter, R., Li, S.J., Lin, C.P., Lowe, M.J., Mackay, C., Madden, D.J., Madsen, K.H., Margulies, D.S., Mayberg, H.S., McMahon, K., Monk, C.S., Mostofsky, S.H., Nagel, B.J., Pekar, J.J., Peltier, S.J., Petersen, S.E., Riedl, V., Rombouts, S.A., Rypma, B., Schlaggar, B.L., Schmidt, S., Seidler, R.D., Siegle, G.J., Sorg, C., Teng, G.J., Veijola, J., Villringer, A., Walter, M., Wang, L., Weng, X.C., Whitfield-Gabrieli, S., Williamson, P., Windischberger, C., Zang, Y.F., Zhang, H.Y., Castellanos, F.X., Milham, M.P., Toward discovery science of human brain function. Proc. Natl. Acad. Sci. 107 (2010), 4734–4739.
7. Boulesteix, A.L., Lauer, S., Eugster, M.J., A plea for neutral comparison studies in computational sciences. PLoS One, 8, 2013, e61562.
8. Brodersen, K.H., Ong, C.S., Stephan, K.E., Buhmann, J.M., The balanced accuracy and its posterior distribution. Proceedings of the IEEE 20th International Conference on Pattern Recognition, 2010, 3121–3124.
9. Brosch T., Tam R., Alzheimer's Disease Neuroimaging Initiative, 2013. Manifold learning of brain MRIs by deep learning. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, 633–640. Springer Berlin Heidelberg.
10. Cabral, C., Kambeitz-Ilankovic, L., Kambeitz, J., Calhoun, V.D., Dwyer, D.B., von Saldern, S., Urquijo, M.F., Falkai, P., Koutsouleris, N., Classifying schizophrenia using multimodal multivariate pattern recognition analysis: evaluating the impact of individual clinical profiles on the neurodiagnostic performance. Schizophr. Bull. 42 (2016), S110–S117.
11. Calhoun, V.D., Sui, J., Multimodal fusion of brain imaging data: a key to finding the missing link(s) in complex mental illness. Biol. Psychiatry: Cogn. Neurosci.Neuroimag. 1 (2016), 230–244.
12. Chen, Y., Shi, B., Smith, C.D., Liu, J., 2015. Nonlinear Feature Transformation and Deep Fusion for Alzheimer's Disease Staging Analysis. In: International Workshop on Machine Learning in Medical Imaging, 304–312. Springer International Publishing.
13. Deshpande, G., Wang, P., Rangaprakash, D., Wilamowski, B., Fully connected cascade artificial neural network architecture for attention deficit hyperactivity disorder classification from functional magnetic resonance imaging data. IEEE Trans. Cybernet. 45 (2015), 2668–2679.
14. Donahue, J., Anne Hendricks, L., Guadarrama, S., Rohrbach, M., Venugopalan, S., Saenko, K., Darrell, T., Long-term recurrent convolutional networks for visual recognition and description. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, 2625–2634.
15. Fei, B., Liu, J., Binary tree of SVM: a new fast multiclass training and classification algorithm. IEEE Trans. Neural Netw. 17 (2006), 696–704.
16. Fox, M.D., Snyder, A.Z., Vincent, J.L., Corbetta, M., Van Essen, D.C., Raichle, M.E., The human brain is intrinsically organized into dynamic, anticorrelated functional networks. Proc. Natl. Acad. Sci. U. S. A. 102 (2005), 9673–9678.
17. Gao, X.W., Hui, R., A deep learning based approach to classification of CT brain images. Science and Information Conference, London, UK, 2016.
18. Gelbart, M.A., Snoek, J., Adams, R.P., 2014. Bayesian optimization with unknown constraints. arXiv preprint arXiv:1403.5607.
19. Gong, Q., Li, L., Du, M., Pettersson-Yeo, W., Crossley, N., Yang, X., Li, J., Huang, X., Mechelli, A., Quantitative prediction of individual psychopathology in trauma survivors using resting-state FMRI. Neuropsychopharmacology 39 (2014), 681–687.
20. Grün, F., Rupprecht, C., Navab, N., Tombari, F., 2016. A Taxonomy and Library for Visualizing Learned Features in Convolutional Neural Networks. arXiv preprint arXiv:1606.07757.
21. Gupta, A., Ayhan, M., Maida, A., Natural image bases to represent neuroimaging data. International Conference on Machine Learning, 2013, 987–994.
22. Han X., Zhong Y., He L., Philip S.Y., Zhang L., 2015. The unsupervised hierarchical convolutional sparse auto-encoder for neuroimaging data classification. In: International Conference on Brain Informatics and Health, 156–166. Springer International Publishing.
23. Hao, A.J., He, B.L., Yin, C.H., Discrimination of ADHD children based on deep bayesian network. 2015 International Conference on Biomedical Image and Signal Processing, 2015, 1–6.
24. Hastie, T., Tibshirani, R., Friedman, J., The Elements of Statistical Learning: Data Mining, Inference and Prediction. 2001, Springer-Verlag, New York, NY.
25. He, K., Zhang, X., Ren, S., Sun, J., 2015. Deep residual learning for image recognition. arXiv preprint arXiv:1512.03385.
26. Hinton, G.E., Osindero, S., Teh, Y.W., A fast learning algorithm for deep belief nets. Neural Comput. 18 (2006), 1527–1554.
27. Hosseini-Asl, E., Gimel'farb, G., El-Baz, A., 2016. Alzheimer's Disease Diagnostics by a Deeply Supervised Adaptable 3D Convolutional Network. arXiv preprint arXiv:1607.00556.
28. Hsu, C.W., Lin, C.J., A comparison of methods for multiclass support vector machines. IEEE Trans. Neural Netw. 13 (2002), 415–425.
29. Hu, C., Ju, R., Shen, Y., Zhou, P., Li, Q., Clinical decision support for Alzheimer's disease based on deep learning and brain network. Proceedings of the IEEE International Conference on Communications, 2016, 1–6.
30. Hutchison, R.M., Womelsdorf, T., Allen, E.A., Bandettini, P.A., Calhoun, V.D., Corbetta, M., Della Penna, S., Duyn, J.H., Glover, G.H., Gonzalez-Castillo, J., Handwerker, D.A., Keilholz, S., Kiviniemi, V., Leopold, D.A., de Pasquale, F., Sporns, O., Walter, M., Chang, C., Dynamic functional connectivity: promise, issues, and interpretations. Neuroimage 80 (2013), 360–378.
31. Kennedy, D.P., Courchesne, E., The intrinsic functional organization of the brain is altered in autism. Neuroimage 39 (2008), 1877–1885.
32. Kim, J., Calhoun, V.D., Shim, E., Lee, J.H., Deep neural network with weight sparsity control and pre-training extracts hierarchical features and enhances classification performance: evidence from whole-brain resting-state functional connectivity patterns of schizophrenia. Neuroimage 124 (2016), 127–146.
33. Krizhevsky, A., Sutskever, I., Hinton, G.E., Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems, 2012, 1097–1105.
34. Kuang, D., He, L., Classification on ADHD with deep learning. Proceedings of the International Conference on Cloud Computing and Big Data, 2014, 27–32.
35. Kuang, D., Guo, X., An, X., Zhao, Y., He, L., Discrimination of ADHD based on fMRI data with deep belief network. International Conference on Intelligent Computing, 2014, 225–232.
36. Kumar, M.A., Gopal, M., Reduced one-against-all method for multiclass SVM classification. Expert Syst. Appl. 38 (2011), 14238–14248.
37. Larochelle, H., Erhan, D., Courville, A., Bergstra, J., Bengio, Y., An empirical evaluation of deep architectures on problems with many factors of variation. Proceedings of the 24th International Conference on Machine Learning, 2007, 473–480.
38. Le, Q., Ranzato, M., Monga, R., Devin, M., Chen, K., Corrado, G., Dean, J., Ng, A., Building high-level features using large scale unsupervised learning. International Conference on Machine Learning, 103, 2012.
39. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P., Gradient-based learning applied to document recognition. Proceedings of the IEEE 86 (1998), 2278–2324.
40. LeCun, Y., Bengio, Y., Hinton, G., Deep learning. Nature 521 (2015), 436–444.
41. Li, F., Tran, L., Thung, K.H., Ji, S., Shen, D., Li, J., Robust deep learning for improved classification of AD/MCI patients. International Workshop on Machine Learning in Medical Imaging, 2014, 240–247.
42. Liu, S., Liu, S., Cai, W., Pujol, S., Kikinis, R., Feng, D., Early diagnosis of Alzheimer's Disease with deep learning. IEEE 11th International Symposium on Biomedical Imaging, 2014, 1015–1018.
43. Liu, S., Liu, S., Cai, W., Che, H., Pujol, S., Kikinis, R., Feng, D., Fulham, M.J., Multimodal neuroimaging feature learning for multiclass diagnosis of Alzheimer's disease. IEEE Trans. Biomed. Eng. 62 (2015), 1132–1140.
44. Liu, S., Liu, S., Cai, W., Pujol, S., Kikinis, R., Feng, D.D., Multi-phase feature representation learning for neurodegenerative disease diagnosis. Australasian Conference on Artificial Life and Computational Intelligence, 2015, 350–359.
45. McCulloch, W., Pitts, W., A logical calculus of the ideas immanent in nervous activity. Bull. Math. Biophys. 7 (1943), 115–133.
46. Mechelli, A., Prata, D., Kefford, C., Kapur, S., Predicting clinical response in people at ultra-high risk of psychosis: a systematic and quantitative review. Drug Discovery Today 20 (2015), 924–927.
47. Milham, M.P., Fair, D., Mennes, M., Mostofsky, S.H., The ADHD-200 consortium: a model to advance the translational potential of neuroimaging in clinical neuroscience. Front. Syst. Neurosci., 6, 2012, 62.
48. Moody, J., Hanson, S., Krogh, A., Hertz, J.A., A simple weight decay can improve generalization. Adv. Neural Inf. Process. Syst. 4 (1995), 950–957.
49. Moradi, E., Pepe, A., Gaser, C., Huttunen, H., Tohka, J., Alzheimer's disease neuroimaging initiative. Machine learning framework for early MRI-based Alzheimer's conversion prediction in MCI subjects. Neuroimage 104 (2015), 398–412.
50. Mueller, S.G., Weiner, M.W., Thal, L.J., Petersen, R.C., Jack, C.R., Jagust, W., Trojanowski, J.Q., Toga, A.W., Beckett, L., Ways toward an early diagnosis in Alzheimer's disease: the Alzheimer's disease neuroimaging initiative (ADNI). Alzheimer's Dementia 1 (2005), 55–66.
51. Mueller, S.G., Weiner, M.W., Thal, L.J., Petersen, R.C., Jack, C.R., Jagust, W., Trojanowski, J.Q., Toga, A.W., Beckett, L., The Alzheimer's disease neuroimaging initiative. Neuroimaging Clin. N. Am. 15 (2005), 869–877.
52. Mulders, P.C., van Eijndhoven, P.F., Schene, A.H., Beckmann, C.F., Tendolkar, I., Resting-state functional connectivity in major depressive disorder: a review. Neurosci. Biobehav. Rev. 56 (2015), 330–344.
53. Munsell, B.C., Wee, C.Y., Keller, S.S., Weber, B., Elger, C., da Silva, L.A.T., Nesland, T., Styner, M., Shen, D., Bonilha, L., Evaluation of machine learning algorithms for treatment outcome prediction in patients with epilepsy based on structural connectome data. Neuroimage 118 (2015), 219–230.
54. Nieuwenhuis, M., van Haren, N.E., Pol, H.E.H., Cahn, W., Kahn, R.S., Schnack, H.G., Classification of schizophrenia patients and healthy controls from structural MRI scans in two large independent samples. Neuroimage 61 (2012), 606–612.
55. Nowlan, S.J., Hinton, G.E., Simplifying neural networks by soft weight-sharing. Neural Comput. 4 (1992), 473–493.
56. Orrù, G., Pettersson-Yeo, W., Marquand, A.F., Sartori, G., Mechelli, A., Using Support Vector Machine to identify imaging biomarkers of neurological and psychiatric disease: a critical review. Neurosci. Biobehav. Rev. 36 (2012), 1140–1152.
57. Page, A., Turner, J.T., Mohsenin, T., Oates, T., Comparing raw data and feature extraction for seizure detection with deep learning methods. International Florida Artificial Intelligence Research Society Conference, 2014.
58. Payan, A., Montana, G., 2015. Predicting Alzheimer's disease: a neuroimaging study with 3D convolutional neural networks. arXiv preprint arXiv: 1502.02506.
59. Pereira, F., Mitchell, T., Botvinick, M., Machine learning classifiers and fMRI: a tutorial overview. Machine learning classifiers and fMRI: a tutorial overview. Neuroimage 45 (2009), S199–S209.
60. Pettersson-Yeo, W., Benetti, S., Marquand, A.F., Dell‘Acqua, F., Williams, S.C.R., Allen, P., Prata, D., McGuire, P., Mechelli, A., Using genetic: cognitive and multi-modal neuroimaging data to identify ultra-high-risk and first-episode psychosis at the individual level. Psychol. Med. 43 (2013), 2547–2562.
61. Plis, S.M., Hjelm, D.R., Salakhutdinov, R., Allen, E.A., Bockholt, H.J., Long, J.D., Johnson, H.J., Paulsen, J.S., Turner, J., Calhoun, V.D., Deep learning for neuroimaging: a validation study. Front. Neurosci. 8 (2014), 1–11.
62. Radua, J., Borgwardt, S., Crescini, A., Mataix-Cols, D., Meyer-Lindenberg, A., McGuire, P.K., Fusar-Poli, P., Multimodal meta-analysis of structural and functional brain changes in first episode psychosis and the effects of antipsychotic medication. Neurosci. Biobehav. Rev. 36 (2012), 2325–2333.
63. Samek, W., Binder, A., Montavon, G., Bach, S., Müller, K.R., 2015. Evaluating the visualization of what a deep neural network has learned. arXiv preprint arXiv:1509.06321.
64. Sarraf, S., Tofighi, G., 2016. Classification of Alzheimer's Disease using fMRI Data and Deep Learning Convolutional Neural Networks. arXiv preprint arXiv:1603.08631.
65. Schmidhuber, J., Deep learning in neural networks: an overview. Neural Netw. 61 (2015), 85–117.
66. Schultz, C.C., Fusar-Poli, P., Wagner, G., Koch, K., Schachtzabel, C., Gruber, O., Sauer, H., Schlösser, R.G., Multimodal functional and structural imaging investigations in psychosis research. Eur. Arch. Psychiatry Clin. Neurosci. 262 (2012), 97–106.
67. Sheffield, J.M., Barch, D.M., Cognition and resting-state functional connectivity in schizophrenia. Neurosci. Biobehav. Rev. 61 (2016), 108–120.
68. Simonyan, K., Zisserman, A. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.
69. Simonyan, K., Vedaldi, A., Zisserman, A., 2013. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034.
70. Springenberg, J.T., Dosovitskiy, A., Brox, T., and Riedmiller, M., 2014. Striving for simplicity: the all convolutional net. arXiv preprint arXiv:1412.6806.
71. Srivastava, N., Hinton, G.E., Krizhevsky, A., Sutskever, I., Salakhutdinov, R., Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15 (2014), 1929–1958.
72. Stonnington, C.M., Chu, C., Klöppel, S., Jack, C.R., Ashburner, J., Frackowiak, R.S., Alzheimer Disease Neuroimaging Initiative. Predicting clinical scores from magnetic resonance scans in Alzheimer's disease. Neuroimage 51 (2010), 1405–1413.
73. Suk, H.I., Shen, D., Deep learning-based feature representation for AD/MCI classification. International Conference on Medical Image Computing and Computer-Assisted Intervention, 2013, 583–590.
74. Suk, H.I., Lee, S.W., Shen, D., Alzheimer's Disease Neuroimaging Initiative. Hierarchical feature representation and multimodal fusion with deep learning for AD/MCI diagnosis. Neuroimage 101 (2014), 569–582.
75. Suk, H.I., Lee, S.W., Shen, D., Alzheimer's disease neuroimaging initiative. Latent feature representation with stacked auto-encoder for AD/MCI diagnosis. Brain Struct. Funct. 220 (2015), 841–859.
76. Suk, H.I., Lee, S.W., Shen, D., Alzheimer's Disease Neuroimaging Initiative. Deep sparse multi-task learning for feature selection in Alzheimer's disease diagnosis. Brain Struct. Funct., 2015, 1–19.
77. Suk, H.I., Wee, C.Y., Lee, S.W., Shen, D., State-space model with deep learning for functional dynamics estimation in resting-state fMRI. Neuroimage 129 (2016), 292–307.
78. Szegedy, C., Ioffe, S., Vanhoucke, V., 2016. Inception-v4, inception-resnet and the impact of residual connections on learning. arXiv preprint arXiv:1602.07261.
79. Tognin, S., Pettersson-Yeo, W., Valli, I., Hutton, C., Woolley, J., Allen, P., McGuire, P., Mechelli, A., Using structural neuroimaging to make quantitative predictions of symptom progression in individuals at ultra-high risk for psychosis. Front. Psychiatry, 4, 2014, 187.
80. Valli, I., Marquand, A.F., Mechelli, A., Raffin, M., Allen, P., Seal, M.L., McGuire, P., Identifying individuals at high risk of psychosis: predictive utility of Support Vector Machine using structural and functional MRI data. Front. Psychiatry, 7, 2016.
81. van der Meer, L., Costafreda, S., Aleman, A., David, A.S., Self-reflection and the brain: a theoretical review and meta-analysis of neuroimaging studies with implications for schizophrenia. Neurosci. Biobehav. Rev. 34:6 (2010), 935–946.
82. Vapnik, V.N., The Nature of Statistical Learning Theory. 1995, Springer.
83. Vincent, P., Larochelle, H., Lajoie, I., Bengio, Y., Manzagol, P.A., Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion. J. Mach. Learn. Res. 11 (2010), 3371–3408.
84. Willette, A.A., Calhoun, V.D., Egan, J.M., Kapogiannis, D., Alzheimer׳ s Disease Neuroimaging Initiative. Prognostic classification of mild cognitive impairment and Alzheimer's disease: MRI independent component analysis. Psychiatry Res.: Neuroimag. 224 (2014), 81–88.
85. Wolfers, T., Buitelaar, J.K., Beckmann, C.F., Franke, B., Marquand, A.F., From estimating activation locality to predicting disorder: a review of pattern recognition for neuroimaging-based psychiatric diagnostics. Neurosci. Biobehav. Rev. 57 (2015), 328–349.
86. Yang, Z., Zhong, S., Carass, A., Ying, S.H., Prince, J.L., Deep learning for cerebellar ataxia classification and functional score regression. International Workshop on Machine Learning in Medical Imaging, 2014, 68–76.
87. Yosinski, J., Clune, J., Nguyen, A., Fuchs, T., Lipson, H., 2015. Understanding neural networks through deep visualization. arXiv preprint arXiv:1506.06579.
88. Yung, A.R., Yuen, H.P., McGorry, P.D., Phillips, L.J., Kelly, D., Dell'Olio, M., Francey, S.M., Cosgrave, E.M., Killackey, E., Stanford, C., Godfrey, K., Buckby, J., Mapping the onset of psychosis: the comprehensive assessment of at-risk mental states. Aust. N. Z. J. Psychiatry 39 (2005), 964–971.
89. Zarogianni, E., Moorhead, T.W., Lawrie, S.M., Towards the identification of imaging biomarkers in schizophrenia: using multivariate pattern classification at a single-subject level. NeuroImage: Clin. 3 (2013), 279–289.
90. Zeiler, M.D., Fergus, R., 2014. Visualizing and understanding convolutional networks. In European Conference on Computer Vision, 818–833. Springer International Publishing.
91. Zhang, D., Shen, D., Alzheimer's Disease Neuroimaging Initiative. Predicting future clinical changes of MCI patients using longitudinal and multimodal biomarkers. PLoS One, 7, 2012, e33182.