Automated measurement of mouse social behaviors using depth sensing, video tracking, and machine learning

Proceedings of the National Academy of Sciences of the United States of America

Volumn 112, Issue 38, 2015, Pages E5351-E5360

  Hong, Weizhe ^a   Kennedy, Ann ^a   Burgos Artizzu, Xavier P ^b   Zelikowsky, Moriel ^a   Navonne, Santiago G ^b   Perona, Pietro ^b   Anderson, David J ^a  

^a California Institute of Technology   (United States)

^b CALIFORNIA INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

Behavioral tracking; Depth sensing; Machine vision; Social behavior; Supervised machine learning

Indexed keywords

ACCURACY; ANIMAL BEHAVIOR; ANIMAL EXPERIMENT; AUTISM; AUTOMATION; BEHAVIORAL SCIENCE; COMPUTER AIDED DESIGN; COMPUTER PROGRAM; CONFERENCE PAPER; CONTROLLED STUDY; ELECTRONIC SENSOR; EXPERIMENTAL DESIGN; FEMALE; IMAGE ANALYSIS; IMAGE PROCESSING; LEARNING ALGORITHM; MACHINE LEARNING; MALE; MOUSE; NONHUMAN; PREDICTIVE VALUE; PRIORITY JOURNAL; QUANTITATIVE ANALYSIS; REAL TIME TRACKING SYSTEM; SOCIAL BEHAVIOR; SOCIAL ENVIRONMENT; SOCIAL INTERACTION; SUPPORT VECTOR MACHINE; VIDEORECORDING; VISION; ALGORITHM; ANIMAL; ANIMAL MODEL; AUTOMATED PATTERN RECOGNITION; BAGG ALBINO MOUSE; C57BL MOUSE; COMPUTER; OBSERVER VARIATION; REPRODUCIBILITY; SOFTWARE;

ALGORITHMS; ANIMALS; BEHAVIOR, ANIMAL; COMPUTERS; FEMALE; IMAGE PROCESSING, COMPUTER-ASSISTED; MACHINE LEARNING; MALE; MICE; MICE, INBRED BALB C; MICE, INBRED C57BL; MODELS, ANIMAL; OBSERVER VARIATION; PATTERN RECOGNITION, AUTOMATED; REPRODUCIBILITY OF RESULTS; SOCIAL BEHAVIOR; SOFTWARE; VIDEO RECORDING;

EID: 84942914975     PISSN: 00278424     EISSN: 10916490     Source Type: Journal    
DOI: 10.1073/pnas.1515982112     Document Type: Article

References (46)

1. Silverman JL, Yang M, Lord C, Crawley JN (2010) Behavioural phenotyping assays for mouse models of autism. Nat Rev Neurosci 11(7):490-502.
2. Anderson DJ, Perona P (2014) Toward a science of computational ethology. Neuron 84(1):18-31.
3. Gomez-Marin A, Paton JJ, Kampff AR, Costa RM, Mainen ZF (2014) Big behavioral data: Psychology, ethology and the foundations of neuroscience. Nat Neurosci 17(11):1455-1462.
4. Spink AJ, Tegelenbosch RA, Buma MO, Noldus LP (2001) The EthoVision video tracking system-A tool for behavioral phenotyping of transgenic mice. Physiol Behav 73(5):731-744.
5. Noldus LP, Spink AJ, Tegelenbosch RA (2001) EthoVision: A versatile video tracking system for automation of behavioral experiments. Behav Res Methods Instrum Comp. 33(3):398-414.
6. Ou-Yang T-H, Tsai M-L, Yen C-T, Lin T-T (2011) An infrared range camera-based approach for three-dimensional locomotion tracking and pose reconstruction in a rodent. J Neurosci Methods 201(1):116-123.
7. Post AM, et al. (2011) Gene-environment interaction influences anxiety-like behavior in ethologically based mouse models. Behav Brain Res 218(1):99-105.
8. Walf AA, Frye CA (2007) The use of the elevated plus maze as an assay of anxietyrelated behavior in rodents. Nat Protoc 2(2):322-328.
9. Pham J, Cabrera SM, Sanchis-Segura C, Wood MA (2009) Automated scoring of fear-related behavior using EthoVision software. J Neurosci Methods 178(2):323-326.
10. Cai H, Haubensak W, Anthony TE, Anderson DJ (2014) Central amygdala PKC-δ (+) neurons mediate the influence of multiple anorexigenic signals. Nat Neurosci 17(9):1240-1248.
11. Tzschentke TM (2007) Measuring reward with the conditioned place preference (CPP) paradigm: update of the last decade. Addict Biol 12(3-4):227-462.
12. Nestler EJ, Hyman SE (2010) Animal models of neuropsychiatric disorders. Nat Neurosci 13(10):1161-1169.
13. Dankert H, Wang L, Hoopfer ED, Anderson DJ, Perona P (2009) Automated monitoring and analysis of social behavior in Drosophila. Nat Methods 6(4):297-303.
14. Eyjolfsdottir E, Branson S, Burgos-Artizzu XP (2014) Detecting social actions of fruit flies. Computer Vision-ECCV. Lecture Notes in Computer Science (Springer, Cham, Switzerland), Vol 8690, pp 772-787.
15. Kabra M, Robie AA, Rivera-Alba M, Branson S, Branson K (2013) JAABA: Interactive machine learning for automatic annotation of animal behavior. Nat Methods 10(1):64-67.
16. Branson K, Robie AA, Bender J, Perona P, Dickinson MH (2009) High-throughput ethomics in large groups of Drosophila. Nat Methods 6(6):451-457.
17. Tsai H-Y, Huang Y-W (2012) Image tracking study on courtship behavior of Drosophila. PLoS One 7(4):e34784.
18. Iyengar A, Imoehl J, Ueda A, Nirschl J, Wu C-F (2012) Automated quantification of locomotion, social interaction, and mate preference in Drosophila mutants. J Neurogenet 26(3-4):306-316.
19. Gomez-Marin A, Partoune N, Stephens GJ, Louis M, Brembs B (2012) Automated tracking of animal posture and movement during exploration and sensory orientation behaviors. PLoS One 7(8):e41642.
20. Kohlhoff KJ, et al. (2011) The iFly tracking system for an automated locomotor and behavioural analysis of Drosophila melanogaster. Integr Biol (Camb) 3(7):755-760.
21. Fontaine EI, Zabala F, Dickinson MH, Burdick JW (2009) Wing and body motion during flight initiation in Drosophila revealed by automated visual tracking. J Exp Biol 212(Pt 9):1307-1323.
22. Card G, Dickinson M (2008) Performance trade-offs in the flight initiation of Drosophila. J Exp Biol 211(Pt 3):341-353.
23. Wolf FW, Rodan AR, Tsai LT-Y, Heberlein U (2002) High-resolution analysis of ethanolinduced locomotor stimulation in Drosophila. J Neurosci 22(24):11035-11044.
24. Burgos-Artizzu XP, Perona P, Dollár P (2013) Robust face landmark estimation under occlusion. Proceedings of the 2013 IEEE International Conference on Computer Vision (IEEE, Washington, DC), pp 1513-1520.
25. de Chaumont F, et al. (2012) Computerized video analysis of social interactions in mice. Nat Methods 9(4):410-417.
26. Burgos-Artizzu XP, Dollár P, Lin D, Anderson DJ, Perona P (2012) Social behavior recognition in continuous video. Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition (IEEE, Washington, DC), pp 1322-1329.
27. Dollár P, Welinder P, Perona P (2010) Cascaded pose regression. Proceedings of the 2010 IEEE Conference on Computer Vision and Pattern Recognition (IEEE, Washington, DC), pp 1078-1085.
28. Ohayon S, Avni O, Taylor AL, Perona P, Roian Egnor SE (2013) Automated multi-day tracking of marked mice for the analysis of social behaviour. J Neurosci Methods 219(1):10-19.
29. Weissbrod A, et al. (2013) Automated long-term tracking and social behavioural phenotyping of animal colonies within a semi-natural environment. Nat Commun 4:2018.
30. Shemesh Y, et al. (2013) High-order social interactions in groups of mice. eLife 2:e00759.
31. Matsumoto J, et al. (2013) A 3D-video-based computerized analysis of social and sexual interactions in rats. PLoS One 8(10):e78460.
32. Iddan GJ, Yahav G (2001) Three-dimensional imaging in the studio and elsewhere. Photonics West 2001: Electronic Imaging, eds Corner BD, Nurre JH, Pargas RP (SPIE, Bellingham, WA), pp 48-55.
33. Lyons DM, MacDonall JS, Cunningham KM (2014) A Kinect-based system for automatic recording of some pigeon behaviors. Behav Res Methods, 10.3758/s13428-014-0531-6.
34. Kulikov VA, et al. (2014) Application of 3-D imaging sensor for tracking minipigs in the open field test. J Neurosci Methods 235:219-225.
35. Gonçalves N, Rodrigues JL, Costa S, Soares F (2012) Preliminary study on determining stereotypical motor movements. Proceedings of the 2012 International Conference of the IEEE Engineering in Medicine and Biology Society (Piscataway, NJ), pp 1598-1601.
36. Guillot PV, Chapouthier G (1996) Intermale aggression and dark/light preference in ten inbred mouse strains. Behav Brain Res 77(1-2):211-213.
37. Ellegood J, Crawley JN (2015) Behavioral and neuroanatomical phenotypes in mouse models of autism. Neurotherapeutics 12(3):521-533.
38. McFarlane HG, et al. (2008) Autism-like behavioral phenotypes in BTBR T+tf/J mice. Genes Brain Behav 7(2):152-163.
39. Bolivar VJ, Walters SR, Phoenix JL (2007) Assessing autism-like behavior in mice: Variations in social interactions among inbred strains. Behav Brain Res 176(1):21-26.
40. Luo L, Callaway EM, Svoboda K (2008) Genetic dissection of neural circuits. Neuron 57(5):634-660.
41. Deisseroth K, Schnitzer MJ (2013) Engineering approaches to illuminating brain structure and dynamics. Neuron 80(3):568-577.
42. Zhang F, Aravanis AM, Adamantidis A, de Lecea L, Deisseroth K (2007) Circuit-breakers: optical technologies for probing neural signals and systems. Nat Rev Neurosci 8(8):577-581.
43. Desland FA, Afzal A, Warraich Z, Mocco J (2014) Manual versus automated rodent behavioral assessment: Comparing efficacy and ease of bederson and garcia neurological deficit scores to an open field video-tracking system. J Cent Nerv Syst Dis 6:7-14.
44. Lin D, et al. (2011) Functional identification of an aggression locus in the mouse hypothalamus. Nature 470(7333):221-226.
45. Button KS, et al. (2013) Power failure: Why small sample size undermines the reliability of neuroscience. Nat Rev Neurosci 14(5):365-376.
46. Chen JL, Andermann ML, Keck T, Xu N-L, Ziv Y (2013) Imaging neuronal populations in behaving rodents: Paradigms for studying neural circuits underlying behavior in the mammalian cortex. J Neurosci 33(45):17631-17640.