Multimodal integration of micro-doppler sonar and auditory signals for behavior classification with convolutional networks

International Journal of Neural Systems

Volumn 23, Issue 5, 2013, Pages

  Dura Bernal, Salvador ^a   Garreau, Guillaume ^b   Georgiou, Julius ^b   Andreou, Andreas G ^c   Denham, Susan L ^d   Wennekers, Thomas ^d  

^a SUNY DOWNSTATE MEDICAL CENTER   (United States)

^b UNIVERSITY OF CYPRUS   (Cyprus)

^c Johns Hopkins University   (United States)

^d UNIVERSITY OF PLYMOUTH   (United Kingdom)

Author keywords

convolutional network; human action recognition; Multimodal integration; sonar

Indexed keywords

ACOUSTIC INFORMATION; BEHAVIOR CLASSIFICATION; CLASSIFICATION PERFORMANCE; CONVOLUTIONAL NETWORKS; CONVOLUTIONAL NEURAL NETWORK; HUMAN-ACTION RECOGNITION; MULTIMODAL INTEGRATION; SYSTEM ARCHITECTURES;

ACOUSTIC FIELDS; GESTURE RECOGNITION; INTEGRATION; NEURAL NETWORKS; SONAR;

BEHAVIORAL RESEARCH;

ARTIFICIAL NEURAL NETWORK; COMPUTER SYSTEM; DOPPLER FLOWMETRY; HUMAN; PROCEDURES; SOCIAL BEHAVIOR; SOUND DETECTION; ARTICLE; METHODOLOGY;

COMPUTER SYSTEMS; DOPPLER EFFECT; HUMANS; NEURAL NETWORKS (COMPUTER); SOCIAL BEHAVIOR; SOUND SPECTROGRAPHY;

EID: 84882261563     PISSN: 01290657     EISSN: None     Source Type: Journal    
DOI: 10.1142/S0129065713500214     Document Type: Article

References (40)

1. T. Teixeira, G. Dublon and A. Savvides, A surveyof human-sensing: Methods for detecting presence,count, location, track, and identity, Tech. Rep. 09-2010, ENALAB, Yale University (2010).
2. M. Yguel, O. Aycard, D. Raulo and C. Laugier, Gridbased fusion of off-board cameras, Intelligent VehiclesSymp. (2006), pp. 276-281.
3. T. Ellis, Multi-camera video surveillance, Int. CarnahanConf. Security Technology (2002), pp. 228-233.
4. C. Wallraven, M. Schultze, B. Mohler, A. Vatakisand K. Pastra, The poeticon enacted scenario corpusa tool for human and computational experimentson action understanding, Int. Conf. Automatic FaceGesture Recognition and Workshops (FG 2011)(2011), pp. 484-491.
5. F. Brémond, M. Thonnat and M. Zúniga, Videounderstandingframework for automatic behaviourrecognition, Behav. Res. Meth. 38 (2006) 416-426.
6. J. Candamo, M. Shreve, D. B. Goldgof, D. B. Sapper and R. Kasturi, Understanding transit scenes:A survey on human behavior-recognition algorithms,IEEE Trans. Intell. Transp. Syst. 11 (2010) 206-224.
7. K. Subramanian and S. Suresh, Human action recognitionusing meta-cognitive neuro-fuzzy inferencesystem, Int. J. Neural Syst. 22 (2012) 1250028.
8. Z. Zhang, P. O. Pouliquen, A. Waxman and A. G.Andreou, Acoustic micro-Doppler radar for humangait imaging, J. Acoust. Soc. Am. 121 (2009)EL110-EL113.
9. H.-U. Schnitzler and A. Denzinger, Auditory foveaand Doppler shift compensation: Adaptations forflutter detection in echolocating bats using CF-FMsignals, J. Comp. Physiol. A 197 (2011) 541-559.
10. R. Kober and H. Schnitzler, Information in sonarechoes of fluttering insects available for echolocatingbats, J. Acoust. Soc. Am. 87 (1990) 882-896.
11. G. Neuweiler, How bats detect flying insects, Phys.Today 33 (1980) 34-40.
12. Z. Zhang, P. Pouliquen, A. Waxman and A.Andreou, Acoustic micro-doppler gait signatures ofhumans and animals, Annual Conf. Information Sciencesand Systems, CISS '07 (2007), pp. 627-630.
13. Z. Zhang and A. Andreou, Human identificationexperiments using acoustic micro-doppler signatures,Argentine School of Micro-Nanoelectronics,Technology and Applications, EAMTA 2008 (2008),pp. 81-86.
14. G. Garreau, C. Andreou, A. Andreou, J. Georgiou,S. Dura-Bernal, T.Wennekers and S. Denham, Gaitbasedperson and gender recognition using microdopplersignatures, Biomedical Circuits and SystemsConf. (BioCAS) (2011), pp. 444-447.
15. S. Dura-Bernal, G. Garreau, C. M. Andreou, A. G.Andreou, J. Georgiou, T. Wennekers and S. L. Denham,Human action categorization using ultrasoundmicro-doppler signatures, in HBU (Springer, 2011),pp. 18-28.
16. Y. LeCun, K. Kavukcuoglu and C. Farabet, Convolutionalnetworks and applications in vision, inProc. 2010 IEEE Int. Symp. on Circuits and Systems(ISCAS) (2010), pp. 253-256.
17. D. H. Hubel and T. N. Wiesel, Receptive fields andfunctional architecture in two nonstriate visual areas(18 and 19) of the cat, J. Neurophysiol. 28 (1965)229-289.
18. Y. LeCun and Y. Bengio, Convolutional networks for images, speech and time-series, The Handbook of Brain Theory and Neural Networks, ed. M. Arbib(MIT Press, 1995), pp. 255-258.
19. M. A. Giese and T. Poggio, Neural mechanisms forthe recognition of biological movements, Nat Rev.Neurosci. 4 (2003) 179-192.
20. N. T. N. Wi, C. K. Loo and L. Chockalingam,Biologically inspired face recognition: Toward poseinvariance,Int. J. Neural Syst. 22 (2012) 1250029.
21. B. Boser, E. Sackinger, J. Bromley, Y. Le Cun and L.Jackel, An analog neural network processor with programmabletopology, IEEE J. Solid-State Circuits26 (1991) 2017-2025.
22. C. Farabet, Y. LeCun, K. Kavukcuoglu, E. Culurciello,B. Martini, P. Akselrod and S. Talay,Large-scale FPGA-based convolutional networks, inScaling up Machine Learning: Parallel and DistributedApproaches, eds. R. Bekkerman,M. Bilenkoand J. Langford (Cambridge University Press, 2011),pp. 399-419.
23. J. H. Kaas and T. A. Hackett, Subdivisions of auditorycortex and processing streams in primates,Proc. Natl. Acad. Sci. U.S.A. 97 (2000) 11793-11799.
24. M. Chevillet, M. Riesenhuber and J. P. Rauschecker,Functional correlates of the anterolateral processinghierarchy in human auditory cortex, J. Neurosci. 31(2011) 9345-9352.
25. S. Kumar, K. E. Stephan, J. D. Warren, K. J. Fristonand T. D. Griffiths, Hierarchical processing ofauditory objects in humans, PLoS Comput. Biol. 3(2007) e100.
26. N. Suga, Principles of auditory informationprocessingderived from neuroethology, J. Exp. Biol.146 (1989) 277-286.
27. S. Sukittanon, A. C. Surendran, J. C. Platt andC. J. C. Burges, Convolutional networks for speechdetection, Interspeech'04 (2004), pp. 433-445.
28. T. L. H. Li, A. B. Chan and A. H. W. Chun, Automaticmusical pattern feature extraction using convolutionalneural network, in Proc. Int. MultiConf.Engineers and Computer Scientists, IMECS, Vol. I(2010), pp. 546-550.
29. M. Henaff, K. Jarrett, K. Kavukcuoglu and Y.LeCun, Unsupervised learning of sparse features forscalable audio classification, in Proc. Int. Symp.Music Information Retrieval (ISMIR'11), eds. A.Klapuri and C. Leider (University of Miami, 2011),pp. 681-686.
30. S. Dieleman, P. Brakel and B. Schrauwen,Audio-based music classification with a pretrainedconvolutional network, in Proc. Int. Symp. MusicInformation Retrieval (ISMIR'11), eds. A. Klapuriand C. Leider (University of Miami, 2011), pp. 669-674.
31. G. Garreau, N. Nicolaou, C. Andreou, C. D'Urbal,G. Stuarts and J. Georgiou, Computationally efficientclassification of human transport mode usingmicro-doppler signatures, Annual Conf. InformationSciences and Systems (CISS) (IEEE, 2011), pp. 1-4.
32. M. Carandini and D. J. Heeger, Normalization as acanonical neural computation, Nat. Rev. Neurosci. (2012) 51-62.
33. C.-C. Chang and C.-J. Lin, LIBSVM: A library forsupport vector machines, ACM Trans. Intell. Syst.Technol. 2 (2011) 27:1-27:27.
34. T.-F. Wu, C.-J. Lin and R. C. Weng, Probabilityestimates for multi-class classification by pairwisecoupling, J. Mach. Learn. Res. 5 (2004) 975-1005.
35. S. A. Fulopa and K. Fitz, Algorithms for computingthe time-corrected instantaneous frequency (reassigned)spectrogram, with applications, J. Acoust.Soc. Am. 119 (2006) 360-371.
36. P. S. Bradley and U. M. Fayyad, Refining initialpoints for K-means clustering, in ICML '98: Proc.Fifteenth Int. Conf. Machine Learning, San Francisco,CA, USA (Morgan Kaufmann Publishers Inc.,1998), pp. 91-99.
37. N. Mesgarani, S. V. David, J. B. Fritz and S. A.Shamma, Phoneme representation and classificationin primary auditory cortex, J. Acoust. Soc. Am. 123(2008) 899-909.
38. D. A. Reynolds and R. C. Rose, Robust textindependentspeaker identification using gaussianmixture speaker models, IEEE Trans. Speech AudioProcess. 1 (1995) 72-83.
39. K. Kalgaonkar and B. Raj, Acoustic doppler sonarfor gait recognition, Advanced Video and SignalBased Surveillance, AVSS 2007 (2007), pp. 27-32.
40. K. Kalgaonkar and B. Raj, One-handed gesturerecognition using ultrasonic doppler sonar, Acoustics,Speech and Signal Processing, ICASSP 2009(2009), pp. 1889-1892