Deep Bayesian active learning with image data

34th International Conference on Machine Learning, ICML 2017

Volumn 3, Issue , 2017, Pages 1923-1932

  Gal, Yarin ^a,b   Islam, Riashat ^a   Ghahramani, Zoubin ^a,c  

^a UNIVERSITY OF CAMBRIDGE   (United Kingdom)

^b ALAN TURING INSTITUTE   (United Kingdom)

^c Uber AI Labs   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; BAYESIAN NETWORKS; CLUSTERING ALGORITHMS; IMAGE ENHANCEMENT; UNCERTAINTY ANALYSIS;

ACTIVE LEARNING; HIGH DIMENSIONAL DATA; LARGE AMOUNTS OF DATA; LEARNING METHODS; LEARNING TOOL; LESION IMAGE; MNIST DATASET; MODEL UNCERTAINTIES;

DEEP LEARNING;

EID: 85048415204     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (40)

1. Agarwal, Mohit, Damaraju, Nandita, and Chaieb, Sahbi. D18803. Https://github.com/NanditaDamaraJU/DL8803, 2016.
2. Cohn, David A, Ghahramani, Zoubin, and Jordan, Michael I. Active learning with statistical models. Journal of artificial intelligence research, 1996.
3. Cortes, Corinna and Vapnik, Vladimir. Support-vector networks. Machine learning, 20(3):273-297, 1995.
4. Deng, Jia, Dong, Wei, Socher, Richard, Li, Li-Jia, Li, Kai, and Fei-Fei, Li. Imagenet: A large-scale hierarchical image database. In Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, pp. 248-255. IEEE, 2009.
5. fchollet. Kcras. Https://github.com/fchollet/keras,2015.
6. Freeman, Linton G. Elementary applied statistics, 1965.
7. Gal, Yarin. Uncertainty in Deep Learning. PhD thesis, University of Cambridge, 2016.
8. Gal, Yarin and Ghahramani, Zoubin. Bayesian convolutional neural networks with Bernoulli approximate variational inference. ICLR workshop track, 2016a.
9. Gal, Yarin and Ghahramani, Zoubin. Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. ICML, 2016b.
10. Gutman, David, Codella, Noel CF, Celebi, Emre, Helba, Brian, Marchetti, Michael, Mishra, Nabin, and Halpern, Allan. Skin lesion analysis toward melanoma detection: A challenge at the international symposium on biomedical imaging (ISBI) 2016, hosted by the international skin imaging collaboration (ISIC). ArXiv preprint arXiv:1605.01397,20\6.
11. He, Kaiming, Zhang, Xiangyu, Ren, Shaoqing, and Sun, Jian. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE International Conference on Computer Vision,pp. 1026-1034,2015.
12. Hernandez-Lobato, Jose Miguel and Adams, Ryan. Probabilistic backpropagation for scalable learning of Bayesian neural networks. In Proceedings of The 32nd International Conference on Machine Learning, pp. 1861-1869, 2015.
13. Hinton, Geoffrey E, Srivastava, Nitish, Krizhevsky, Alex, Sutskever, Ilya, and Salakhutdinov, Ruslan R. Improving neural networks by preventing co-adaptation of feature detectors. ArXiv preprint arXiv: 1207.0580, 2012.
14. Holub, Alex, Perona, Pietro, and Burl, Michael C. Entropy-based active learning for object recognition. In Computer Vision and Pattern Recognition Workshops, 2008. CVPRW'08. IEEE Computer Society Conference on, pp. 1-8. IEEE, 2008.
15. Houlsby, Neil, Huszar, Ferenc, Ghahramani, Zoubin, and Lengyel, Mâté. Bayesian active learning for classification and preference learning. ArXiv preprint arXiv:! 112.5745, 2011.
16. Joshi, Ajay J, Porikli, Fatih, and Papanikolopoulos, Niko-laos. Multi-class active learning for image classification. In Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, pp. 2372-2379. IEEE, 2009.
17. Jozefowicz, Rafal, Vinyals, Oriol, Schuster, Mike, Shazccr, Noam, and Wu, Yonghui. Exploring the limits of language modeling. ArXiv preprint arXiv: 1602.02410, 2016.
18. Kalchbrenner, Nal and Blunsom, Phil. Recurrent continuous translation models. In EMNLP, 2013.
19. Kampffmeyer, Michael, Salberg, Arnt-Borre, and Jenssen, Robert. Semantic segmentation of small objects and modeling of uncertainty in urban remote sensing images using deep convolutional neural networks. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, June 2016.
20. Kendall, Alex, Badrinarayanan, Vijay, and Cipolla, Roberto. Bayesian segnet: Model uncertainty in deep convolutional encoder-decoder architectures for scene understanding. ArXiv preprint arXiv:1511.02680, 2015.
21. Kingma, Diederik P, Mohamed, Shakir, Rezende, Danilo Jimenez, and Welling, Max. Semi-supervised learning with deep generative models. In Advances in Neural Information Processing Systems, pp. 3581-3589, 2014.
22. Krizhevsky, Alex, Sutskever, Ilya, and Hinton, Geoffrey E. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, pp. 1097-1105, 2012.
23. LeCun, Yann and Cortes, Corinna. The MNIST database of handwritten digits, 1998.
24. LeCun, Yann, Böser, Bernhard, Denker, John S, Henderson, Donnie, Howard, Richard E, Hubbard, Wayne, and Jackel, Lawrence D. Backpropagation applied to handwritten zip code recognition. Neural Computation, 1(4):541-551, 1989.
25. Lee, Dong-Hyun. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In Workshop on Challenges in Representation Learning, 2013.
26. Li, Xin and Guo, Yuhong. Adaptive active learning for image classification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 859-866, 2013.
27. Marcus, Daniel S, Fotenos, Anthony F, Csernansky, John G, Morris, John C, and Buckner, Randy L. Open access series of imaging studies: longitudinal mri data in nonde-mented and demented older adults. Journal of cognitive neuroscience, 22(12):2677-2684, 2010.
28. Miyato, Takeru, Maeda, Shin-ichi, Koyama, Masanori, Nakae, Ken, and Ishii, Shin. Distributional smoothing by virtual adversarial examples. ArXiv preprint arXiv:1507.00677, 2015.
29. Pitelis, Nikolaos, Russell, Chris, and Agapito, Lourdes. Semi-supervised learning using an unsupervised atlas. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 565-580. Springer, 2014.
30. Rasmus, Antti, Berglund, Mathias, Honkala, Mikko, Valpola, Harri, and Raiko, Tapani. Scmi-supcrviscd learning with ladder networks. In Advances in Neural Information Processing Systems, pp. 3546-3554, 2015.
31. Rifai, Salah, Dauphin, Yann N, Vincent, Pascal, Bengio, Yoshua, and Muller, Xavier. The manifold tangent classifier. In Advances in Neural Information Processing Systems, pp. 2294-2302, 2011.
32. Rumelhart, David E, Hinton, Geoffrey E, and Williams, Ronald J. Learning internal representations by error propagation. Technical report, DTIC Document, 1985.
33. Shannon, Claude Elwood. A mathematical theory of communication. Bell System Technical Journal, 27(3):379-423, 1948.
34. Simonyan, K. And Zisserman, A. Very deep convolutional networks for large-scale image recognition. In International Conference on Learning Representations, 2015.
35. Srivastava, Nitish, Hinton, Geoffrey, Krizhevsky, Alex, Sutskever, Ilya, and Salakhutdinov, Ruslan. Dropout: A simple way to prevent neural networks from overfitting. JMLR, 2014.
36. Sundermeyer, Martin, Schlüter, Ralf, and Ney, Hermann. LSTM neural networks for language modeling. In INTERSPEECH, 2012.
37. Sutskever, Ilya, Vinyals, Oriol, and Le, Quoc VV. Sequence to sequence learning with neural networks. In NIPS, 2014.
38. Tong, Simon. Active Learning: Theory and Applications. PhD thesis, 2001. AAI3028187.
39. Weston, Jason, Ratle, Frédéric, Mobahi, Hossein, and Col-lobert, Ronan. Deep learning via semi-supervised embedding. In Neural Networks: Tricks of the Trade, pp. 639-655. Springer, 2012.
40. Zhu, X, Lafferty, J, and Ghahramani, Z. Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions. In Proceedings of the ICML-2003 Workshop on The Continuum from Labeled to Unlabeled Data, pp. 58-65. ICML, 2003.