Deep multimodal learning: A survey on recent advances and trends

IEEE Signal Processing Magazine

Volumn 34, Issue 6, 2017, Pages 96-108

  Ramachandram, Dhanesh ^a   Taylor, Graham W ^a  

^a UNIVERSITY OF GUELPH   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

DEEP LEARNING;

COMPLEX MACHINES; LEARNING ARCHITECTURES; MULTI-MODAL; MULTI-MODAL FUSION; MULTI-MODAL LEARNING; MULTIPLE DATA; RESEARCH FIELDS; STATE OF THE ART;

LEARNING SYSTEMS;

EID: 85040306665     PISSN: 10535888     EISSN: None     Source Type: Journal    
DOI: 10.1109/MSP.2017.2738401     Document Type: Article

References (103)

1. Y. LeCun, Y. Bengio, and G. Hinton, "Deep learning, " Nature, vol. 521, no. 7553, pp. 436-444, 2015.
2. D. Lahat, T. Adali, and C. Jutten, "Multimodal data fusion: An overview of methods, challenges, and prospects, " Proc. IEEE, vol. 103, no. 9, pp. 1449-1477, 2015.
3. P. K. Atrey, M. A. Hossain, A. El Saddik, and M. S. Kankanhalli, "Multimodal fusion for multimedia analysis: A survey, " Multimedia Systems, vol. 16, no. 6, pp. 345-379, 2010.
4. B. Khaleghi, A. Khamis, F. O. Karray, and S. N. Razavi, "Multisensor data fusion: A review of the state-of-the-art, " Inform. Fusion, vol. 14, no. 1, pp. 28-44, 2013.
5. L. I. Kuncheva, Combining Pattern Classifiers: Methods and Algorithms. Hoboken, NJ: Wiley, 2004.
6. J. Ngiam, A. Khosla, M. Kim, J. Nam, H. Lee, and A. Y. Ng, "Multimodal deep learning, " in Proc. 28th Int. Conf. Machine Learning (ICML-11), 2011, pp. 689-696.
7. N. Srivastava and R. R. Salakhutdinov, "Multimodal learning with deep Boltzmann machines, " in Proc. Advances in Neural Inform. Processing Syst., 2012, pp. 2222-2230.
8. N. Neverova, C. Wolf, G. Taylor, and F. Nebout, "ModDrop: Adaptive multi-modal gesture recognition, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 38, no. 8, pp. 1692-1706, 2016.
9. S. S. Mukherjee and N. M. Robertson, "Deep head pose: Gaze-direction estimation in multimodal video, " IEEE Trans. Multimedia, vol. 17, no. 11, pp. 2094-2107, 2015.
10. C. Ding and D. Tao, "Robust face recognition via multimodal deep face representation, " IEEE Trans. Multimedia, vol. 17, no. 11, pp. 2049-2058, 2015.
11. S. E. Kahou, X. Bouthillier, P. Lamblin, C. Gulcehre, V. Michalski, K. Konda, S. Jean, P. Froumenty, et al., "EmoNets: Multimodal deep learning approaches for emotion recognition in video, " J. Multimedia User Interfaces, vol. 10, no. 2, pp. 99-111, 2015.
12. Z. Sitová, J. Šedenka, Q. Yang, G. Peng, G. Zhou, P. Gasti, and K. S. Balagani, "HMOG: New behavioral biometric features for continuous authentication of smartphone users, " IEEE Trans. Inf. Forensics Security, vol. 11, no. 5, pp. 877-892, 2016.
13. V. Radu, N. D. Lane, S. Bhattacharya, C. Mascolo, M. K. Marina, and F. Kawsar, "Toward multimodal deep learning for activity recognition on mobile devices, " in Proc. ACM Int. Joint Conf. Pervasive and Ubiquitous Computing: Adjunct, 2016, pp. 185-188.
14. A. Jain, J. Tompson, Y. LeCun, and C. Bregler, "Modeep: A deep learning framework using motion features for human pose estimation, " in Proc. Asian Conf. Computer Vision, 2014, pp. 302-315.
15. A. Valada, G. L. Oliveira, T. Brox, and W. Burgard, "Deep multispectral semantic scene understanding of forested environments using multimodal fusion, " in Proc. Int. Symp. Experimental Robotics (ISER 2016), 2016, pp. 465-477.
16. D. Shen, G. Wu, and H.-I. Suk, "Deep learning in medical image analysis, " Annu. Review Biomedical Eng., vol. 19, pp. 221-248, 2017.
17. R. Kiros, K. Popuri, D. Cobzas, and M. Jagersand, "Stacked multiscale feature learning for domain independent medical image segmentation, " in Proc. Int. Workshop on Mach. Learning in Medical Imaging, 2014, pp. 25-32.
18. P. Wu, S. C. Hoi, H. Xia, P. Zhao, D. Wang, and C. Miao, "Online multimodal deep similarity learning with application to image retrieval, " in Proc. 21st ACM Int. Conf. Multimedia. 2013, pp. 153-162.
19. M. Simonovsky, B. Gutiérrez-Becker, D. Mateus, N. Navab, and N. Komodakis, "A deep metric for multimodal registration, " in Proc. Int. Conf. Medical Image Computer and Computer-Assisted Intervention, 2016, pp. 10-18.
20. S. Liu, S. Liu, W. Cai, H. Che, S. Pujol, R. Kikinis, D. Feng, M. J. Fulham, et al., "Multimodal neuroimaging neature learning for multiclass diagnosis of Alzheimer's disease, " IEEE Trans. Biomed. Eng., vol. 62, no. 4, pp. 1132-1140, 2015.
21. P. Mamoshina, A. Vieira, E. Putin, and A. Zhavoronkov, "Applications of deep learning in biomedicine, " Molecular Pharmaceutics, vol. 13, no. 5, pp. 1445-1454, 2016.
22. Y. Guo, G. Wu, L. A. Commander, S. Szary, V. Jewells, W. Lin, and D. Shen, "Segmenting hippocampus from infant brains by sparse patch matching with deeplearned features, " in Proc. Int. Conf. Medical Image Computer and Computer-Assisted Intervention, 2014, p. 308.
23. N. Tajbakhsh, J. Y. Shin, S. R. Gurudu, R. T. Hurst, C. B. Kendall, M. B. Gotway, and J. Liang, "Convolutional neural networks for medical image analysis: Full training or fine tuning" IEEE Trans. Med. Imag., vol. 35, no. 5, pp. 1299-1312, 2016.
24. I. Sturm, S. Lapuschkin, W. Samek, and K.-R. Müller, "Interpretable deep neural networks for single-trial eeg classification, " J. Neuroscience Methods, vol. 274, pp. 141-145, Dec. 2016.
25. S. G. Kim, N. Theera-Ampornpunt, C.-H. Fang, M. Harwani, A. Grama, and S. Chaterji, "Opening up the blackbox: An interpretable deep neural network-based classifier for cell-type specific enhancer predictions, " BMC Syst. Biology, vol. 10, no. 2, p. 54, 2016.
26. C. Chen, A. Seff, A. Kornhauser, and J. Xiao, "Deepdriving: Learning affordance for direct perception in autonomous driving, " in Proc. IEEE Int. Conf. Computer Vision, 2015, pp. 2722-2730.
27. J. Long, E. Shelhamer, and T. Darrell, "Fully convolutional networks for semantic segmentation, " in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2015, pp. 3431-3440.
28. I. Lenz, H. Lee, and A. Saxena, "Deep learning for detecting robotic grasps, " Int. J. Robotics Res., vol. 34, no. 4-5, pp. 705-724, 2015.
29. S. Gu, E. Holly, T. Lillicrap, and S. Levine. (2016). Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates. ArXiv. [Online]. Available: https://arxiv. org/abs/1610. 00633
30. N. Srivastava and R. Salakhutdinov, "Learning representations for multimodal data with deep belief nets, " presented at Proc. 29th Int. Conf. Machine Learning (Workshop), 2012.
31. Y. Cao, S. Steffey, J. He, D. Xiao, C. Tao, P. Chen, and H. Müller, "Medical image retrieval: A multimodal approach, " Cancer Informatics, vol. 13, no. Suppl 3, p. 125, 2014.
32. M. Liang, Z. Li, T. Chen, and J. Zeng, "Integrative data analysis of multi-platform cancer data with a multimodal deep learning approach, " IEEE/ACM Trans. Comput. Biol. Bioinf., vol. 12, no. 4, pp. 928-937, 2015.
33. K. Simonyan and A. Zisserman, "Two-stream convolutional networks for action recognition in videos, " in Proc. Advances in Neural Information Processing Systems, 2014, pp. 568-576.
34. S. Poria, E. Cambria, and A. Gelbukh, "Deep convolutional neural network textual features and multiple kernel learning for utterance-level multimodal sentiment analysis, " in Proc. Conf. Empirical Methods on Natural Language Processing, 2015, pp. 2539-2544.
35. A. Jain, A. Singh, H. S. Koppula, S. Soh, and A. Saxena, "Recurrent neural networks for driver activity anticipation via sensory-fusion architecture, " in Proc. 2016 IEEE Int. Conf. Robotics and Automation (ICRA), 2016, pp. 3118-3125.
36. S. Antol, A. Agrawal, J. Lu, M. Mitchell, D. Batra, C. L. Zitnick, and D. Parikh, "VQA: Visual question answering, " in Proc. Int. Conf. Computer Vision (ICCV), 2015, pp. 2425-2433.
37. J. Donahue, L. Anne Hendricks, S. Guadarrama, M. Rohrbach, S. Venugopalan, K. Saenko, and T. Darrell, "Long-term recurrent convolutional networks for visual recognition and description, " in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2015, pp. 2625-2634.
38. A. Karpathy, A. Joulin, and F. F. F. Li, "Deep fragment embeddings for bidirectional image sentence mapping, " in Proc. Advances in Neural Information Processing Systems, 2014, pp. 1889-1897.
39. O. Vinyals, A. Toshev, S. Bengio, and D. Erhan, "Show and tell: A neural image caption generator, " in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2015, pp. 3156-3164.
40. M. Ren, R. Kiros, and R. Zemel, "Exploring models and data for image question answering, " in Proc. Advances in Neural Information Processing Systems, 2015, pp. 2953-2961.
41. J.-H. Kim, S.-W. Lee, D.-H. Kwak, M.-O. Heo, J. Kim, J.-W. Ha, and B.-T. Zhang. (2016). Multimodal residual learning for visual QA. ArXiv. [Online]. Available: https:// arxiv. org/abs/1606. 01455
42. F. J. Ordónez and D. Roggen, "Deep convolutional and LSTM recurrent neural networks for multimodal wearable activity recognition, " Sensors, vol. 16, no. 1, p. 115, 2016.
43. D. Wu, L. Pigou, P.-J. Kindermans, N. D.-H. Le, L. Shao, J. Dambre, and J.-M. Odobez, "Deep dynamic neural networks for multimodal gesture segmentation and recognition, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 38, no. 8, pp. 1583-1597, 2016.
44. H. Larochelle and Y. Bengio, "Classification using discriminative restricted Boltzmann machines, " in Proc. 25th Int. Conf. Machine Learning, 2008, pp. 536-543.
45. Y. Huang, W. Wang, and L. Wang, "Unconstrained multimodal multi-label learning, " IEEE Trans. Multimedia, vol. 17, no. 11, pp. 1923-1935, 2015.
46. I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, "Generative adversarial nets, " in Proc. Advances in Neural Information Processing Systems, 2014, pp. 2672-2680.
47. D. P. Kingma and M. Welling, "Auto-encoding variational bayes, " in Proc. Int. Conf. Learning Representations (ICLR), 2014.
48. S. Reed, Z. Akata, X. Yan, L. Logeswaran, B. Schiele, and H. Lee, "Generative adversarial text to image synthesis, " in Proc. 33rd Int. Conf. Machine Learning (ICML), 2016, pp. 1060-1069.
49. M. Suzuki, K. Nakayama, and Y. Matsuo. (2016). Joint multimodal learning with deep generative models. ArXiv. [Online]. Available: https://arxiv. org/ abs/1611. 01891
50. G. Pandey and A. Dukkipati. (2016). Variational methods for conditional multimodal deep learning. ArXiv. [Online]. Available: https://arxiv. org/abs/1603. 01801
51. L. Deng, "A tutorial survey of architectures, algorithms, and applications for deep learning, " APSIPA Trans. Signal and Inform. Processing, vol. 3, pp. 1-29, 2014.
52. M. R. Amer, T. Shields, B. Siddiquie, A. Tamrakar, A. Divakaran, and S. Chai, "Deep multimodal fusion: A hybrid approach, " Int. J. Comput. Vision, pp. 1-17, 2017. DOI: 10. 1007/s11263-017-0997-7.
53. M. R. Amer, B. Siddiquie, S. Khan, A. Divakaran, and H. Sawhney, "Multimodal fusion using dynamic hybrid models, " in Proc. IEEE 2014 Applications of Computer Vision Winter Conf., 2014, pp. 556-563.
54. D. S. Sachan, U. Tekwani, and A. Sethi, "Sports video classification from multimodal information using deep neural networks, " in Proc. 2013 Association for the Advancement of Artificial Intelligence Fall Symp., 2013, pp. 102-107.
55. Y. Liu, X. Feng, and Z. Zhou, "Multimodal video classification with stacked contractive autoencoders, " Signal Processing, vol. 120, pp. 761-766, Mar. 2016.
56. N. Sebe, Machine Learning in Computer Vision, vol. 29. Dordrecht, The Netherlands: Springer, 2005.
57. A. Owens, J. Wu, J. H. McDermott, W. T. Freeman, and A. Torralba, Ambient Sound Provides Supervision for Visual Learning. Cham, Switzerland: Springer International Publishing, 2016, pp. 801-816.
58. G. E. Hinton and R. R. Salakhutdinov, "Reducing the dimensionality of data with neural networks, " Science, vol. 313, no. 5786, pp. 504-507, 2006.
59. D. Wang, P. Cui, M. Ou, and W. Zhu, "Learning compact hash codes for multimodal representations using orthogonal deep structure, " IEEE Trans. Multimedia, vol. 17, no. 9, pp. 1404-1416, 2015.
60. J. Masci, M. M. Bronstein, A. M. Bronstein, and J. Schmidhuber, "Multimodal similarity-preserving hashing, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 36, no. 4, pp. 824-830, 2014.
61. H. P. Martínez and G. N. Yannakakis, "Deep multimodal fusion, " in Proc. 16th Int. Conf. Multimodal Interaction, 2014, pp. 34-41.
62. S. E. Kahou, C. Pal, X. Bouthillier, P. Froumenty, Ç. Gülçehre, R. Memisevic, P. Vincent, A. Courville, et al., "Combining modality specific deep neural networks for emotion recognition in video, " in Proc. 15th ACM Int. Conf. Multimodal Interaction, 2013, pp. 543-550.
63. D. Yi, Z. Lei, and S. Z. Li, "Shared representation learning for heterogenous face recognition, " in Proc. Automatic Face and Gesture Recognition 11th IEEE Int. Conf. Workshops, 2015, pp. 1-7.
64. A. Karpathy, G. Toderici, S. Shetty, T. Leung, R. Sukthankar, and L. Fei-Fei, "Large-scale video classification with convolutional neural networks, " in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2014, pp. 1725-1732.
65. M. J. Wainwright, "Structured regularizers for high-dimensional problems: Statistical and computational issues, " Annu. Rev. Statistics Application, vol. 1, pp. 233-253, Apr. 2014.
66. K. Sohn, W. Shang, and H. Lee, "Improved multimodal deep learning with variation of information, " in Proc. Advances in Neural Information Processing Systems., 2014, pp. 2141-2149.
67. J. J.-Y. Wang, Y. Wang, S. Zhao, and X. Gao, "Maximum mutual information regularized classification, " Eng. Applicat. Artificial Intell., vol. 37, pp. 1-8, Jan. 2015.
68. S. Zhu, X. Li, and S. Shen, "Multimodal deep network learning-based image annotation, " IET Electron. Lett., vol. 51, no. 12, pp. 905-906, 2015.
69. H. Fei and J. Huan, "Structured feature selection and task relationship inference for multi-task learning, " Knowledge and Inform. Syst., vol. 35, no. 2, pp. 345-364, 2013.
70. Z. Wu, Y.-G. Jiang, J. Wang, J. Pu, and X. Xue, "Exploring inter-feature and interclass relationships with deep neural networks for video classification, " in Proc. ACM Int. Conf. Multimedia, 2014, pp. 167-176.
71. A. Wang, J. Lu, J. Cai, T. J. Cham, and G. Wang, "Large-margin multi-modal deep learning for RGB-D object recognition, " IEEE Trans. Multimedia, vol. 17, no. 11, pp. 1887-1898, Nov. 2015.
72. A. Wang, J. Cai, J. Lu, and T.-J. Cham, "MMSS: Multi-modal sharable and specific feature learning for RGB-D object recognition, " in Proc. IEEE Int. Conf. Computer Vision, 2015, pp. 1125-1133.
73. J. Elman, "Learning and development in neural networks: The importance of starting small, " Cognition, vol. 48, no. 1, pp. 71-99, 1993.
74. T. Chen, I. Goodfellow, and J. Shlens. (2015). Net2Net: Accelerating learning via knowledge transfer. ArXiv. [Online]. Available: https://arxiv. org/abs/1511. 05641
75. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, "Going deeper with convolutions, " in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2015, pp. 1-9.
76. R. Reed, "Pruning algorithms-A survey, " IEEE Trans. Neural Netw., vol. 4, no. 5, pp. 740-747, 1993.
77. J. Feng and T. Darrel, "Learning the structure of deep convolutional networks, " in Proc. Int. Conf. Computer Vision, 2015, pp. 2749-2757.
78. J. Yang, J. Ma, M. Berryman, and P. Perez, "A structure optimization algorithm of neural networks for large-scale data sets, " in Proc. 2014 IEEE Int. Conf. Fuzzy Syst. (FUZZ-IEEE), 2014, pp. 956-961.
79. D. Whitley, T. Starkweather, and C. Bogart, "Genetic algorithms and neural networks: Optimizing connections and connectivity, " Parallel Comput., vol. 14, no. 3, pp. 347-361, 1990.
80. K. O. Stanley and R. Miikkulainen, "Efficient evolution of neural network topologies, " in Proc. Congr. Evolutionary Computation (CEC02), 2002, pp. 1757-1762.
81. T. Shinozaki and S. Watanabe, "Structure discovery of deep neural network based on evolutionary algorithms, " in Proc. 2015 IEEE Int. Conf. Acoustics, Speech and Signal Processing (ICASSP), 2015, pp. 4979-4983.
82. B. Shahriari, K. Swersky, Z. Wang, R. P. Adams, and N. de Freitas, "Taking the human out of the loop: A review of Bayesian optimization, " Proc. IEEE, vol. 104, no. 1, pp. 148-175, 2016.
83. D. Ramachandram, M. Lisicki, T. Shields, M. Amer, and G. Taylor, "Structure optimization for deep multimodal fusion networks using graph-induced kernels, " in Proc. 25th European Symp. Artificial Neural Networks, Computational Intelligence, and Machine Learning (ESANN), Bruges, Belgium, 2017, pp. 11-16.
84. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, vol. 1. Cambridge, MA: MIT Press, 1998.
85. B. Zoph and Q. V. Le. (2016). Neural architecture search with reinforcement learning. ArXiv. [Online]. Available: https://arxiv. org/abs/1611. 01578
86. P. Kulkarni, J. Zepeda, F. Jurie, P. Pérez, and L. Chevallier, "Learning the structure of deep architectures using L1 regularization, " in Proc. British Machine Vision Conf., 2015, pp. 23. 1-23. 11.
87. N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, "Dropout: A simple way to prevent neural networks from overfitting, " J. Mach. Learning Res., vol. 15, no. 1, pp. 1929-1958, 1 Jan. 2014.
88. C. Murdock, Z. Li, H. Zhou, and T. Duerig, "Blockout: Dynamic model selection for hierarchical deep networks, " in Proc. Int. Conf. Computer Vision and Pattern Recognition, 2016, pp. 2583-2591.
89. F. Li, N. Neverova, C. Wolf, and G. Taylor, "Modout: Learning multi-modal architectures by stochastic regularization, " in Proc. 2017 IEEE Conf. Automatic Face and Gesture Recognition, 2017, pp. 422-429.
90. M. Firman, "RGBD data sets: Past, present and future, " in Proc. CVPR Workshop on Large Scale 3D Data: Acquisition, Modelling, and Analysis, 2016.
91. A. Geiger, P. Lenz, C. Stiller, and R. Urtasun, "Vision meets robotics: The Kitti data set, " Int. J. Robotics Res., vol. 32, no. 11, pp. 1231-1237, 2013.
92. W. Maddern, G. Pascoe, C. Linegar, and P. Newman, "1 Year, 1000km: The Oxford RobotCar data set, " Int. J. Robotics Res., vol. 36, no. 1, pp. 3-15, 2017.
93. C. Chen, R. Jafari, and N. Kehtarnavaz, "UTD-MHAD: A multimodal data set for human action recognition utilizing a depth camera and a wearable inertial sensor, " in Proc. 2015 IEEE Int. Conf Image Processing (ICIP), 2015, pp. 168-172.
94. S. Escalera, X. Baró, J. Gonzalez, M. A. Bautista, M. Madadi, M. Reyes, V. Ponce-López, H. J. Escalante, et al., "Chalearn looking at people challenge 2014: Data set and results, " in Proc. Workshop at the European Conf. Computer Vision, 2014, pp. 459-473.
95. F. Ofli, R. Chaudhry, G. Kurillo, R. Vidal, and R. Bajcsy, "Berkeley MHAD: A comprehensive multimodal human action database, " in Proc. 2013 IEEE Workshop on Applications of Computer Vision, 2013, pp. 53-60.
96. A. Pablo, Y. Mollard, F. Golemo, A. C. Murillo, M. Lopes, and J. Civera, "A multimodal human-robot interaction data set, " in Proc. Neural Information Processing Systems, 2016, pp. 1-5.
97. F. Ringeval, A. Sonderegger, J. Sauer, and D. Lalanne, "Introducing the Recola multimodal corpus of remote collaborative and affective interactions, " in Proc. Automatic Face and Gesture Recognition 10th IEEE Int. Conf. Workshops, 2013, pp. 1-8.
98. O. Banos, C. Villalonga, R. Garcia, A. Saez, M. Damas, J. A. Holgado-Terriza, S. Lee, H. Pomares, and I. Rojas, "Design, implementation and validation of a novel open framework for agile development of mobile health applications, " Biomedical Eng. Online, vol. 14, no. 2, p. S6, 2015. [Online]. Available: https://doi. org/10. 1186/1475-925X-14-S2-S6
99. J. Mao, J. Xu, K. Jing, and A. L. Yuille, "Training and evaluating multimodal word embeddings with large-scale web annotated images, " in Proc. Advances in Neural Information Processing Systems, 2016, pp. 442-450.
100. J. Arevalo, T. Solorio, M. Montes-y Gómez, and F. A. González. (2017). Gated multimodal units for information fusion. ArXiv. [Online]. Available: https://arxiv. org/ abs/1702. 01992
101. Y.-G. Jiang, Z. Wu, J. Wang, X. Xue, and S.-F. Chang, "Exploiting feature and class relationships in video categorization with regularized deep neural networks, " IEEE Trans. Pattern Anal. Mach. Intell., 2017. doi: https://doi. org/10. 1109/ TPAMI. 2017. 2670560
102. R. Min, N. Kose, and J.-L. Dugelay, "KinectFaceDB: A kinect database for face recognition, " IEEE Trans. Syst., Man, Cybern., Syst., vol. 44, no. 11, pp. 1534-1548, Nov. 2014.
103. B. H. Menze, A. Jakab, S. Bauer, J. Kalpathy-Cramer, K. Farahani, J. Kirby, Y. Burren, N. Porz, et al., "The multimodal brain tumor image segmentation benchmark (BRATS), " IEEE Trans. Med. Imag., vol. 34, no. 10, pp. 1993-2024, 2015.