Automated image-based tracking and its application in ecology

Trends in Ecology and Evolution

Volumn 29, Issue 7, 2014, Pages 417-428

  Dell, Anthony I ^a   Bender, John A ^b   Branson, Kristin ^c   Couzin, Iain D ^d   de Polavieja, Gonzalo G ^e   Noldus, Lucas P J J ^f   Pérez Escudero, Alfonso ^e   Perona, Pietro ^g   Straw, Andrew D ^h   Wikelski, Martin ^i,j   Brose, Ulrich ^a  

^a UNIVERSITY OF GÖTTINGEN   (Germany)

^b HasOffers Inc   (United States)

^c HOWARD HUGHES MEDICAL INSTITUTE   (United States)

^d PRINCETON UNIVERSITY   (United States)

^e INSTITUTO CAJAL   (Spain)

^f Noldus Information Technology bv   (Netherlands)

^g CALIFORNIA INSTITUTE OF TECHNOLOGY   (United States)

^h RESEARCH INSTITUTE OF MOLECULAR PATHOLOGY   (Austria)

ⁱ MAX PLANCK INSTITUTE FOR ORNITHOLOGY   (Germany)

^j UNIVERSITY OF KONSTANZ   (Germany)

more..

Author keywords

Automated image based tracking; Behavior; Bio logging; Ecological interactions; Tracking

Indexed keywords

BEHAVIORAL RESPONSE; IMAGE ANALYSIS; IMAGING METHOD; TRACKING;

ANIMAL; ANIMAL BEHAVIOR; ANIMAL DISPERSAL; ECOLOGY; TELEMETRY; TRENDS;

ANIMAL DISTRIBUTION; ANIMALS; BEHAVIOR, ANIMAL; ECOLOGY; TELEMETRY;

EID: 84902660161     PISSN: 01695347     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.tree.2014.05.004     Document Type: Review

References (93)

1. Berdahl A., et al. Emergent sensing of complex environments by mobile animal groups. Science 2013, 339:574-576.
2. Simpson S.J., et al. Modelling nutritional interactions: from individuals to communities. Trends Ecol. Evol. 2010, 25:53-60.
3. Pawar S., et al. Dimensionality of consumer search space drives trophic interaction strengths. Nature 2012, 486:485-489.
4. Handegard N.O., et al. The dynamics of coordinated group hunting and collective information transfer among schooling prey. Curr. Biol. 2012, 22:1213-1217.
5. Nathan R., et al. A movement ecology paradigm for unifying organismal movement research. Proc. Natl. Acad. Sci. U.S.A. 2008, 105:19052-19059.
6. Altmann J. Observational study of behavior: sampling methods. Behaviour 1974, 49:227-267.
7. Tinbergen N. On aims and methods of ethology. Zeitschr. Tierpsychol. 1963, 20:410-433.
8. Dankert H., et al. Automated monitoring and analysis of social behavior in Drosophila. Nat. Methods 2009, 6:297-303.
9. Rutz C., Bluff L.A. Animal-borne imaging takes wing, or the dawn of 'wildlife video-tracking'. Trends Ecol. Evol. 2008, 23:292-294. author reply 294.
10. Cooke S.J., et al. Biotelemetry: a mechanistic approach to ecology. Trends Ecol. Evol. 2004, 19:334-343.
11. Krause J., et al. Reality mining of animal social systems. Trends Ecol. Evol. 2013, 28:541-551.
12. Elliott J.P., et al. Prey capture by the African lion. Can. J. Zool. 1977, 55:1811-1828.
13. Bayne B.L., Scullard C. Rates of feeding by Thais-(Nucella)-Lapillus (L). J. Exp. Marine Biol. Ecol. 1978, 32:113-129.
14. Anonymous Geneticist seeks engineer: must like flies and worms. Nat. Methods 2007, 4:463.
15. Balch T., et al. How multirobot systems research will accelerate our understanding of social animal behavior. Proc. IEEE 2006, 94:1445-1463.
16. Noldus L.P.J.J., et al. Computerised video tracking, movement analysis and behaviour recognition in insects. Comput. Electron. Agric. 2002, 35:201-227.
17. Gomez-Marin A., et al. Automated tracking of animal posture and movement during exploration and sensory orientation behaviors. PLoS ONE 2012, 7:e41642.
18. Straw A.D., et al. Multi-camera real-time three-dimensional tracking of multiple flying animals. J. R. Soc. Interface 2011, 8:395-409.
19. Swierczek N.A., et al. High-throughput behavioral analysis in C. elegans. Nat. Methods 2011, 8:592-598.
20. Pérez-Escudero A., et al. idTracker: tracking individuals in a group by automatic identification of unmarked animals. Nat. Methods 2014, 10.1038/nmeth.2994.
21. Kuhl H.S., Burghardt T. Animal biometrics: quantifying and detecting phenotypic appearance. Trends Ecol. Evol. 2013, 28:432-441.
22. Krause J., et al. Interactive robots in experimental biology. Trends Ecol. Evol. 2011, 26:369-375.
23. Ofstad T.A., et al. Visual place learning in Drosophila melanogaster. Nature 2011, 474:204-207.
24. Straw A.D., et al. Visual control of altitude in flying Drosophila. Curr. Biol. 2010, 20:1550-1556.
25. Branson K., Belongie S. Tracking multiple mouse contours (without too many samples). IEEE Comput. Vis. Pattern Recogn. 2005, 1:1039-1046.
26. Strandburg-Peshkin A., et al. Visual sensory networks and effective information transfer in animal groups. Curr. Biol. 2013, 23:R709-R711.
27. Branson K., et al. High-throughput ethomics in large groups of Drosophila. Nat. Methods 2009, 6:451-457.
28. de Chaumont F., et al. Computerized video analysis of social interactions in mice. Nat. Methods 2012, 9:410-417.
29. Matsumoto J., et al. A 3D-video-based computerized analysis of social and sexual interactions in rats. PLoS ONE 2013, 8:e78460.
30. Brose U., et al. Consumer-resource body-size relationships in natural food webs. Ecology 2006, 87:2411-2417.
31. Ohayon S., et al. Automated multi-day tracking of marked mice for the analysis of social behaviour. J. Neurosci. Methods 2013, 219:10-19.
32. Simon J.C., Dickinson M.H. A new chamber for studying the behavior of Drosophila. PLoS ONE 2010, 5:e8793.
33. Katz Y., et al. Inferring the structure and dynamics of interactions in schooling fish. Proc. Natl. Acad. Sci. U.S.A. 2011, 108:18720-18725.
34. Robie A.A., et al. Object preference by walking fruit flies, Drosophila melanogaster, is mediated by vision and graviperception. J. Exp. Biol. 2010, 213:2494-2506.
35. Lacey E.S., Carde R.T. Activation, orientation and landing of female Culex quinquefasciatus in response to carbon dioxide and odour from human feet: 3-D flight analysis in a wind tunnel. Med. Vet. Entomol. 2011, 25:94-103.
36. Spitzen J., et al. A 3D analysis of flight behavior of Anopheles gambiae sensu stricto malaria mosquitoes in response to human odor and heat. PLoS ONE 2013, 8:e62995.
37. Cachat J., et al. Three-dimensional neurophenotyping of adult zebrafish behavior. PLoS ONE 2011, 6:e17597.
38. Ballerini M., et al. Interaction ruling animal collective behavior depends on topological rather than metric distance: evidence from a field study. Proc. Natl. Acad. Sci. U.S.A. 2008, 105:1232-1237.
39. Attanasi A., et al. Tracking in three dimensions via multi-path branching 2013, Cornell University Library. arxiv:1305.1495.
40. Attanasi A., et al. Superfluid transport of information in turning flocks of starlings 2013, Cornell University Library. arxiv:1303.7097.
41. Butail S., et al. Reconstructing the flight kinematics of swarming and mating in wild mosquitoes. J. R. Soc. Interface 2012, 9:2624-2638.
42. Fontaine E.I., et al. Wing and body motion during flight initiation in Drosophila revealed by automated visual tracking. J. Exp. Biol. 2009, 212:1307-1323.
43. Viscido S.V., et al. Individual behavior and emergent properties of fish schools: a comparison of observation and theory. Marine Ecol. Prog. Ser. 2004, 273:239-249.
44. Veeraraghavan A., et al. Motion based correspondence for 3D tracking of multiple dim objects. ICASSP 2006 Proceedings: 2006 IEEE International Conference on Acoustics, Speech and Signal Processing (Vol. 2) 2006.
45. Attanasi A., et al. Wild swarms of midges linger at the edge of an ordering phase transition 2013, Cornell University Library. arxiv:1307.5631.
46. Kanbara M., et al. 3D scene reconstruction from reflection images in a spherical mirror. In 18th International Conference on Pattern Recognition, ICPR 2006 (Vol.) 4 2006, 874-879.
47. Chen Z.H., et al. Single-view reconstruction from an unknown spherical mirror. 18th IEEE International Conference on Image Processing ICIP 2011 2011, 2677-2680.
48. Kalinkat G., et al. Habitat structure alters top-down control in litter communities. Oecologia 2013, 172:877-887.
49. Dell A.I., et al. Systematic variation in the temperature dependence of physiological and ecological traits. Proc. Natl. Acad. Sci. U.S.A. 2011, 108:10591-10596.
50. Hristov N.I., et al. Applications of thermal infrared imaging for research in aeroecology. Integr. Comp. Biol. 2008, 48:50-59.
51. Chiu C., et al. Effects of competitive prey capture on flight behavior and sonar beam pattern in paired big brown bats, Eptesicus fuscus. J. Exp. Biol. 2010, 213:3348-3356.
52. Betke M., et al. Thermal imaging reveals significantly smaller Brazilian free-tailed bat colonies than previously estimated. J. Mammal. 2008, 89:18-24.
53. Mersch D.P., et al. Tracking individuals shows spatial fidelity is a key regulator of ant social organization. Science 2013, 340:1090-1093.
54. Weissbrod A., et al. Automated long-term tracking and social behavioural phenotyping of animal colonies within a semi-natural environment. Nat. Commun. 2013, 4:2018.
55. Johnson S.N., et al. Non-invasive techniques for investigating and modelling root-feeding insects in managed and natural systems. Agric. Forest Entomol. 2007, 9:39-46.
56. Bertolotti J., et al. Non-invasive imaging through opaque scattering layers. Nature 2012, 491:232-234.
57. Gomez-Marin A., et al. Active sampling and decision making in Drosophila chemotaxis. Nat. Commun. 2011, 2:441.
58. Izadi S., et al. KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera. Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology 2011, 559-568.
59. Newcombe R.A., et al. KinectFusion: real-time dense surface mapping and tracking. 10th IEEE International Symposium on Mixed and Augmented Reality 2011, 127-136.
60. ®). Remote Sens. Environ. 2013, 135:36-51.
61. Cagnacci F., et al. Animal ecology meets GPS-based radiotelemetry: a perfect storm of opportunities and challenges. Philos. Trans. R. Soc. Lond. B: Biol. Sci. 2010, 365:2157-2162.
62. Buhl J., et al. From disorder to order in marching locusts. Science 2006, 312:1402-1406.
63. Fergus R., et al. Object class recognition by unsupervised scale-invariant learning. Proceedings of the 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Vol. II) 2003, 264-271.
64. Dollár P., et al. The fastest pedestrian detector in the west. British Machine Vision Conference 2010, 68.1-68.11. BMVA Press. F. Labrosse (Ed.).
65. Burghardt T., Calic J. Real-time face detection and tracking of animals. 8th Seminar on Neural Network Applications in Electrical Engineering, NEUREL 2006 2006, 27-32.
66. Hannuna S.L., et al. Identifying quadruped gait in wildlife video. IEEE International Conference on Image Processing (Vol. 1), ICIP 2005 2005, 713-716.
67. Burgos-Artizzu X.P., et al. Social behavior recognition in continuous video. 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2012, 1322-1329.
68. Berman G.J., et al. Mapping the structure of drosophilid behavior 2013, Cornell University Library. arxiv:1310.4249.
69. Bender J.A., et al. Kinematic and behavioral evidence for a distinction between trotting and ambling gaits in the cockroach Blaberus discoidalis. J. Exp. Biol. 2011, 214:2057-2064.
70. Mirat O., et al. ZebraZoom: an automated program for high-throughput behavioral analysis and categorization. Front. Neural Circuits 2013, 7:107.
71. van Dam E.A., et al. An automated system for the recognition of various specific rat behaviours. J. Neurosci. Methods 2013, 218:214-224.
72. Dollár P., et al. Cascaded pose regression. 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2010, 1078-1085.
73. Kabra M., et al. JAABA: interactive machine learning for automatic annotation of animal behavior. Nat. Methods 2013, 10:64-67.
74. Brown A.E., et al. A dictionary of behavioral motifs reveals clusters of genes affecting Caenorhabditis elegans locomotion. Proc. Natl. Acad. Sci. U.S.A. 2013, 110:791-796.
75. Braun E., et al. Identifying prototypical components in behaviour using clustering algorithms. PLoS ONE 2010, 5:e9361.
76. Fontaine E., et al. Automated visual tracking for studying the ontogeny of zebrafish swimming. J. Exp. Biol. 2008, 211:1305-1316.
77. Censi A., et al. Discriminating external and internal causes for heading changes in freely flying Drosophila. PLoS Comput. Biol. 2013, 9:e1002891.
78. Maimon G., et al. A simple vision-based algorithm for decision making in flying Drosophila. Curr. Biol. 2008, 18:464-470.
79. Pizzo A.B., et al. The membrane raft protein Flotillin-1 is essential in dopamine neurons for amphetamine-induced behavior in Drosophila. Mol. Psychiatry 2013, 18:824-833.
80. van Breugel F., Dickinson M.H. The visual control of landing and obstacle avoidance in the fruit fly Drosophila melanogaster. J. Exp. Biol. 2012, 215:1783-1798.
81. Wu Z., et al. A combinatorial semaphorin code instructs the initial steps of sensory circuit assembly in the Drosophila CNS. Neuron 2011, 70:281-298.
82. Tunstrom K., et al. Collective states, multistability and transitional behavior in schooling fish. PLoS Comput. Biol. 2013, 9:e1002915.
83. Herbert-Read J.E., et al. Inferring the rules of interaction of shoaling fish. Proc. Natl. Acad. Sci. U.S.A. 2011, 108:18726-18731.
84. Fan J., et al. Automatic video-based analysis of animal behaviors. 2010 17th IEEE International Conference on Image Processing (ICIP) 2010, 1513-1516.
85. Wang L., Anderson D.J. Identification of an aggression-promoting pheromone and its receptor neurons in Drosophila. Nature 2010, 463:227-231.
86. Brady D.J., et al. Multiscale gigapixel photography. Nature 2012, 486:386-389.
87. Straw A.D., Dickinson M.H. Motmot, an open-source toolkit for real-time video acquisition and analysis. Source Code Biol. Med. 2009, 4:5.
88. Stuurman N., et al. Micro-Manager: open source software for light microscope imaging. Microsc. Today 2007, 15:42-43.
89. Bar-Shalom Y., Fortmann T.E. Tracking and Data Association 1988, Academic Press.
90. Delcourt J., et al. Video multitracking of fish behaviour: a synthesis and future perspectives. Fish Fisheries 2013, 14:186-204.
91. Hartley R., Zisserman A. Multiple View Geometry in Computer Vision 2004, Cambridge University Press.
92. Zabala F., et al. A simple strategy for detecting moving objects during locomotion revealed by animal-robot interactions. Curr. Biol. 2012, 22:1344-1350.
93. Butail S., Paley D.A. Three-dimensional reconstruction of the fast-start swimming kinematics of densely schooling fish. J. R. Soc. Interface 2012, 9:77-88.