Automatically identifying, counting, and describing wild animals in camera-trap images with deep learning

Proceedings of the National Academy of Sciences of the United States of America

Volumn 115, Issue 25, 2018, Pages E5716-E5725

  Norouzzadeh, Mohammad Sadegh ^a   Nguyen, Anh ^b   Kosmala, Margaret ^c   Swanson, Alexandra ^d   Palmer, Meredith S ^e   Packer, Craig ^e   Clune, Jeff ^a,f  

^a UNIVERSITY OF WYOMING   (United States)

^b AUBURN UNIVERSITY   (United States)

^c HARVARD UNIVERSITY   (United States)

^d UNIVERSITY OF OXFORD   (United Kingdom)

^e UNIVERSITY OF MINNESOTA   (United States)

^f UBER TECHNOLOGIES INC   (United States)

Author keywords

Artificial intelligence; Camera trap images; Deep learning; Deep neural networks; Wildlife ecology

Indexed keywords

ANIMAL BEHAVIOR; ANIMAL TRAPPING; ARTICLE; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; AUTOMATION; COMPUTER ANALYSIS; CONSERVATION BIOLOGY; DEEP LEARNING; DEEP NEURAL NETWORK; INFORMATION PROCESSING; MACHINE LEARNING; NONHUMAN; POPULATION DISPERSAL; PRIORITY JOURNAL; SPECIES CONSERVATION; SPECIES IDENTIFICATION; WILD ANIMAL; WILDLIFE; ALGORITHM; ANIMAL; ECOLOGY; ECOSYSTEM; HUMAN; PHYSIOLOGY; PROCEDURES;

ALGORITHMS; ANIMALS; ANIMALS, WILD; ARTIFICIAL INTELLIGENCE; BEHAVIOR, ANIMAL; ECOLOGY; ECOSYSTEM; HUMANS; MACHINE LEARNING; NEURAL NETWORKS (COMPUTER);

EID: 85048965218     PISSN: 00278424     EISSN: 10916490     Source Type: Journal    
DOI: 10.1073/pnas.1719367115     Document Type: Article

References (63)

1. Swanson A, et al. (2015) Snapshot Serengeti, high-frequency annotated camera trap images of 40 mammalian species in an African savanna. Sci Data 2:150026.
2. Harris G, Thompson R, Childs JL, Sanderson JG (2010) Automatic storage and analysis of camera trap data. Bull Ecol Soc Am 91:352-360.
3. O'Connell AF, Nichols JD, Karanth KU (2010) Camera Traps in Animal Ecology: Methods and Analyses (Springer, Tokyo).
4. Silveira L, Jacomo AT, Diniz-Filho JAF (2003) Camera trap, line transect census and track surveys: A comparative evaluation. Biol Conserv 114:351-355.
5. Bowkett AE, Rovero F, Marshall AR (2008) The use of camera-trap data to model habitat use by antelope species in the Udzungwa mountain forests, Tanzania. Afr J Ecol 46:479-487.
6. Fegraus EH, et al. (2011) Data acquisition and management software for camera trap data: A case study from the team network. Ecol Inform 6:345-353.
7. Krishnappa YS, Turner WC (2014) Software for minimalistic data management in large camera trap studies. Ecol Inform 24:11-16.
8. Swinnen KRR, Reijniers J, Breno M, Leirs H (2014) A novel method to reduce time investment when processing videos from camera trap studies. PLoS One 9:e98881.
9. Swanson A, Arnold T, Kosmala M, Forester J, Packer C (2016) In the absence of a “landscape of fear”: How lions, hyenas, and cheetahs coexist. Ecol Evol 6:8534-8545.
10. Palmer MS, Fieberg J, Swanson A, Kosmala M, Packer C (2017) A 'dynamic' landscape of fear: Prey responses to spatiotemporal variations in predation risk across the lunar cycle. Ecol Lett 20:1364-1373.
11. Anderson TM, et al. (2016) The spatial distribution of African savannah herbivores: Species associations and habitat occupancy in a landscape context. Phil Trans R Soc B 371:20150314.
12. Palmer MS, Packer C (2018) Giraffe bed and breakfast: Camera traps reveal Tanzanian yellow-billed oxpeckers roosting on their large mammalian hosts. Afr J Ecol.
13. Goodfellow I, Bengio Y, Courville A (2016) Deep Learning (MIT Press, Cambridge, MA).
14. Swanson A, Kosmala M, Lintott C, Packer C (2016) A generalized approach for producing, quantifying, and validating citizen science data from wildlife images. Conserv Biol 30:520-531.
15. Samuel AL (1959) Some studies in machine learning using the game of checkers. IBM J Res Dev 3:210-229.
16. Mohri M, Rostamizadeh A, Talwalkar A (2012) Foundations of Machine Learning (MIT Press, Cambridge, MA).
17. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521:436-444.
18. Hu W, Huang Y, Wei L, Zhang F, Li H (2015) Deep convolutional neural networks for hyperspectral image classification. J Sensors 2015:1-10.
19. Bridle JS (1990) Probabilistic interpretation of feedforward classification network outputs, with relationships to statistical pattern recognition. Neurocomputing (Springer, New York), pp 227-236.
20. Hinton G, et al. (2012) Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. IEEE Signal Proc Mag 29:82-97.
21. Deng L, Hinton G, Kingsbury B (2013) New types of deep neural network learning for speech recognition and related applications: An overview. 2013 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (IEEE, New York).
22. Bahdanau D, et al. (2016) End-to-end attention-based large vocabulary speech recognition. 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (IEEE, New York).
23. Sutskever I, Vinyals O, Le QV (2014) Sequence to sequence learning with neural networks. 2014 Advances in Neural Information Processing Systems (NIPS) (Neural Information Processing Systems Foundation, La Jolla, CA).
24. Cho K, et al. (2014) Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv:1406.1078.
25. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (IEEE, New York).
26. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556.
27. Mnih V, et al. (2015) Human-level control through deep reinforcement learning. Nature 518:529-533.
28. Figueroa K, Camarena-Ibarrola A, García J, Villela HT (2014) Fast automatic detection of wildlife in images from trap cameras. Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications: 19th Iberoamerican Congress, eds Bayro-Corrochano E, Hancock E (Springer International Publishing, Cham, Switzerland), pp 940-947.
29. Yu X, et al. (2013) Automated identification of animal species in camera trap images. EURASIP J Image Vide 2013:52.
30. Chen G, Han TX, He Z, Kays R, Forrester T (2014) Deep convolutional neural network based species recognition for wild animal monitoring. 2014 IEEE International Conference on Image Processing (ICIP) (IEEE, New York).
31. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. 2012 Advances in Neural Information Processing Systems (NIPS) (Neural Information Processing Systems Foundation, La Jolla, CA).
32. Yosinski J, Clune J, Bengio Y, Lipson H (2014) How transferable are features in deep neural networks? 2014 Advances in Neural Information Processing Systems (NIPS) (Neural Information Processing Systems Foundation, La Jolla, CA).
33. Bengio Y, Courville A, Vincent P (2013) Representation learning: A review and new perspectives. IEEE T Pattern Anal 35:1798-1828.
34. Graves A, Mohamed Ar, Hinton G (2013) Speech recognition with deep recurrent neural networks. 2013 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (IEEE, New York).
35. Bahdanau D, Cho K, Bengio Y (2014) Neural machine translation by jointly learning to align and translate. arXiv:1409.0473.
36. Wang H, et al. (2014) Mitosis detection in breast cancer pathology images by combining handcrafted and convolutional neural network features. J Med Imaging 1:034003.
37. Rampasek L, Goldenberg A (2018) Learning from everyday images enables expert-like diagnosis of retinal diseases. Cell 172:893-895.
38. Kashif MN, Raza SEA, Sirinukunwattana K, Arif M, Rajpoot N (2016) Handcrafted features with convolutional neural networks for detection of tumor cells in histology images. 2016 IEEE 13th International Symposium on Biomedical Imaging (IEEE, New York), pp 1029-1032.
39. Chherawala Y, Roy PP, Cheriet M (2013) Feature design for offline Arabic handwriting recognition: Handcrafted vs. automated? 2013 International Conference on Document Analysis and Recognition (ICDAR) (IEEE, New York).
40. Park SR, et al. (2018) De-multiplexing vortex modes in optical communications using transport-based pattern recognition. Opt Express 26:4004-4022.
41. Yang J, Yu K, Gong Y, Huang T (2009) Linear spatial pyramid matching using sparse coding for image classification. 2009 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (IEEE, New York).
42. Blei DM, Ng AY, Jordan MI (2003) Latent Dirichlet allocation. J Mach Learn Res 3:993-1022.
43. Fei-Fei L, Perona P (2005) A Bayesian hierarchical model for learning natural scene categories. 2005 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (IEEE, New York), 2:524-531.
44. Gomez A, Diez G, Salazar A, Diaz A (2016) Animal identification in low quality camera-trap images using very deep convolutional neural networks and confidence thresholds. 2016 International Symposium on Visual Computing (Springer, Cham, Switzerland), pp 747-756.
45. Gomez A, Salazar A, Vargas F (2016) Towards automatic wild animal monitoring: Identification of animal species in camera-trap images using very deep convolutional neural networks. arXiv:1603.06169v2.
46. Deng J, et al. (2009) Imagenet: A large-scale hierarchical image database. 2009 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (IEEE, New York).
47. Caruana R (1998) Multitask learning. Learning to Learn (Springer, New York), pp 95-133.
48. Collobert R, Weston J (2008) A unified architecture for natural language processing: Deep neural networks with multitask learning. 2008 International Conference on Machine Learning (ICML) (Association for Computing Machinery, New York).
49. Lin M, Chen Q, Yan S (2013) Network in network. arXiv:1312.4400.
50. Szegedy C, et al. (2015) Going deeper with convolutions. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (IEEE, New York).
51. Dauphin YN, et al. (2014) Identifying and attacking the saddle point problem in high-dimensional non-convex optimization. 2014 Advances in Neural Information Processing Systems (NIPS) (Neural Information Processing Systems Foundation, La Jolla, CA).
52. Chattopadhyay P, Vedantam R, Ramprasaath R, Batra D, Parikh D (2016) Counting everyday objects in everyday scenes. CoRR, abs/1604.03505 1:10.
53. Onoro-Rubio D, López-Sastre RJ (2016) Towards perspective-free object counting with deep learning. 2016 European Conference on Computer Vision (ECCV).
54. Zhang C, Li H, Wang X, Yang X (2015) Cross-scene crowd counting via deep convolutional neural networks. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (IEEE, New York).
55. Tsoumakas G, Katakis I (2006) Multi-label classification: An overview. Int J Data Warehous 3:1-13.
56. Sorower MS (2010) A Literature Survey on Algorithms for Multi-Label Learning (Oregon State University, Corvallis, OR), Vol 18.
57. Read J, Pfahringer B, Holmes G, Frank E (2011) Classifier chains for multi-label classification. Mach Learn 85:333-359.
58. Donahue J, et al. (2014) Decaf: A deep convolutional activation feature for generic visual recognition. 2014 International Conference on Machine Learning (ICML) (Association for Computing Machinery, New York).
59. Russakovsky O, et al. (2015) Imagenet large scale visual recognition challenge. Int J Comput Vis 115:211-252.
60. Settles B (2012) Active learning. Synth Lectures Artif Intelligence Machine Learn 6:1-114.
61. Sener O, Savarese S (2018) Active learning for convolutional neural networks: A core-set approach. International Conference on Learning Representations. Available at https://openreview.net/forum?id=H1aIuk-RW. Accessed May 25, 2018.
62. Ren S, He K, Girshick R, Sun J (2015) Faster R-CNN: Towards real-time object detection with region proposal networks. 2015 Advances in Neural Information Processing Systems (NIPS) (Neural Information Processing Systems Foundation, La Jolla, CA).
63. Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: Unified, real-time object detection. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (IEEE, New York).