Continuous human action recognition using depth-MHI-HOG and a spotter model

Sensors (Switzerland)

Volumn 15, Issue 3, 2015, Pages 5197-5227

  Eum, Hyukmin ^a   Yoon, Changyong ^b   Lee, Heejin ^c   Park, Mignon ^a  

^a Yonsei University   (South Korea)

^b Suwon Science College   (South Korea)

^c Hankyong National University   (South Korea)

Author keywords

Action modeling; Action spotting; Continuous human action recognition; Depth MHI HOG (DMH); Hidden Markov model; Spotter model

Indexed keywords

GESTURE RECOGNITION; HIDDEN MARKOV MODELS; IMAGE RECOGNITION; MARKOV PROCESSES; POINT CONTACTS;

ACTION MODELING; ACTION RECOGNITION; ACTION SPOTTING; CONTINUOUS ACTIONS; DEPTH-MHI-HOG (DMH); EXTRACTING FEATURES; HUMAN-ACTION RECOGNITION; K-MEANS CLUSTERING;

MOTION ESTIMATION;

ARTIFICIAL INTELLIGENCE; AUTOMATED PATTERN RECOGNITION; HUMAN; HUMAN ACTIVITIES; MARKOV CHAIN; THEORETICAL MODEL; VIDEORECORDING;

ARTIFICIAL INTELLIGENCE; HUMAN ACTIVITIES; HUMANS; MARKOV CHAINS; MODELS, THEORETICAL; PATTERN RECOGNITION, AUTOMATED; VIDEO RECORDING;

EID: 84928672305     PISSN: 14248220     EISSN: None     Source Type: Journal    
DOI: 10.3390/s150305197     Document Type: Article

References (46)

1. Mitra, V.; Franco, H.; Graciarena, M.; Vergyri, D. Medium duration modulation cepstral feature for robust speech recognition. In Proceedings of the IEEE ICASSP, Florence, Italy, 4–9 May 2014; pp. 1749–1753.
2. Chaaraoui, A.A.; Padilla-López, J.R.; Ferrández-Pastor, F.J.; Nieto-Hidalgo, M.; Flórez-Revuelta, F. A Vision-Based System for Intelligent Monitoring: Human Behaviour Analysis and Privacy by Context. Sensors 2014, 14, 8895–8925.
3. Siddiqi, M.H.; Lee, S.; Lee, Y.-K.; Khan, A.M.; Truc, P.T.H. Hierarchical recognition scheme for human facial expression recognition systems. Sensors 2013, 13, 16682–16713.
4. Lee, L.; Grimson, W.E.L. Gait analysis for recognition and classification. In Proceedings of the Fifth IEEE International Conference on Automatic Face and Gesture Recognition, Washington, DC, USA, 20–21 May 2002; pp. 148–155.
5. Tao, D.; Li, X.; Wu, X.; Maybank, S.J. General tensor discriminant analysis and gabor features for gait recognition. IEEE Trans. Pattern Anal. Mach. Intell. 2007, 29, 1700–1715.
6. Cazzato, D.; Leo, M.; Distante, C. An Investigation on the Feasibility of Uncalibrated and Unconstrained Gaze Tracking for Human Assistive Applications by Using Head Pose Estimation. Sensors 2014, 14, 8363–8379.
7. Ren, Z.; Yuan, J.; Meng, J.; Zhang, Z. Robust part-based hand gesture recognition using kinect sensor. IEEE Trans. Multimed. 2013, 15, 1110–1120.
8. Chen, S.; Tian, Y.; Liu, Q.; Metaxas, D.N. Recognizing expressions from face and body gesture by temporal normalized motion and appearance features. Image Vis. Comput. 2013, 31, 175–185.
9. Kern, N.; Schiele, B.; Schmidt, A. Multi-sensor activity context detection for wearable computing. In Proceedings of the First European Symposium (EUSAI 2003), Veldhoven, The Netherlands, 3–4 November 2003; pp. 220–232.
10. Kim, D.; Song, J.; Kim, D. Simultaneous gesture segmentation and recognition based on forward spotting accumulative HMMs. Pattern Recognit. 2007, 40, 3012–3026.
11. Wu, X.; Xu, D.; Duan, L.; Luo, J.; Jia, Y. Action Recognition Using Multilevel Features and Latent Structural SVM. IEEE Trans. Circuits Syst. Video Techn. 2013, 23, 1422–1431.
12. Poppe, R. A survey on vision-based human action recognition. Image Vis. Comput. 2010, 28, 976–990.
13. Ahad, M.A. R.; Tan, J.; Kim, H.; Ishikawa, S. Action recognition by employing combined directional motion history and energy images. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), San Francisco, CA, USA, 13–18 June 2010; pp. 73–78.
14. Kim, E.; Helal, S.; Cook, D. Human activity recognition and pattern discovery. IEEE Pervasive Comput. 2010, 9, 48–53.
15. Weinland, D.; Ronfard, R.; Boyer, E. Motion history volumes for free viewpoint action recognition. In Proceedings of the Workshop on Modeling People and Human Interaction (PHI), Beijing, China, 15 October 2005.
16. Ahad, M.A. Motion History Image. In Motion History Images for Action Recognition and Understanding; Springer: London, UK, 2013; pp. 31–76.
17. Bobick, A.F.; Davis, J.W. The recognition of human movement using temporal templates. IEEE Trans. Pattern Anal. Mach. Intell. 2001, 23, 257–267.
18. Rabiner, L. A tutorial on hidden Markov models and selected applications in speech recognition. IEEE Proc. 1989, 77, 257–286.
19. Dugad, R.; Desai, U. A Tutorial on Hidden Markov Models; Technical Report No. SPANN-96.1; Indian Institute of Technology: Bombay Powai, India, May 1996.
20. Li, X. HMM based action recognition using oriented histograms of optical flow field. Electron. Lett. 2007, 43, 560–561.
21. Ali, A.; Aggarwal, J. Segmentation and recognition of continuous human activity. In Proceedings of the IEEE Workshop on Detection and Recognition of Events in Video, Vancouver, BC, Canada, 8 July 2001; pp. 28–35.
22. Morency, L.; Quattoni, A.; Darrell, T. Latent-dynamic discriminative models for continuous gesture recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’07), Minneapolis, MN, USA, 17–22 June 2007; pp. 1–8.
23. Ning, H.; Xu, W.; Gong, Y.; Huang, T. Latent pose estimator for continuous action recognition. In Proceedings of the 10th European Conference on Computer Vision, Marseille, France, 12–18 October 2008; pp. 419–433.
24. Singh, V.K.; Nevatia, R. Action recognition in cluttered dynamic scenes using pose-specific part models. In Proceedings of the IEEE International Conference on Computer Vision (ICCV’11), Barcelona, Spain, 6–13 November 2011; pp. 113–120.
25. Chaudhry, R.; Ofli, F.; Kurillo, G.; Bajcsy, R.; Vidal, R. Bio-inspired dynamic 3D discriminative skeletal features for human action recognition. In Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW’13), Portland, OR, USA, 23–28 June 2013; pp. 471–478.
26. Yu, G.; Liu, Z.; Yuan, J. Discriminative orderlet mining for real-time recognition of human-object interaction. In Proceedings of the Asian Conference on Computer Vision (ACCV’14), Singapore, 1–5 November 2014.
27. Wang, J.; Liu, Z.; Wu, Y. Learning actionlet ensemble for 3D human action recognition. In Human Action Recognition with Depth Cameras; Springer: Los Alamitos, CA, USA, 2014; pp. 11–40.
28. Li, W.; Zhang, Z.; Liu, Z. Expandable data-driven graphical modeling of human actions based on salient postures. IEEE Trans. Circuits Syst. Video Technol. 2008, 18, 1499–1510.
29. Wang, L.; Tan, T.; Ning, H.; Hu, W. Silhouette analysis-based gait recognition for human identification. IEEE Trans. Pattern Anal. Mach. Intell. 2003, 25, 1505–1518.
30. Lv, F.; Nevatia, R. Single view human action recognition using key pose matching and viterbi path searching. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’07), Minneapolis, MN, USA, 17–22 June 2007; pp. 1–8.
31. Jalal, A.; Uddin, M.Z.; Kim, T.-S. Depth video-based human activity recognition system using translation and scaling invariant features for life logging at smart home. IEEE Trans. Consum. Electron. 2012, 58, 863–871.
32. Jalal, A.; Uddin, M.Z.; Kim, J.T.; Kim, T.-S. Recognition of Human Home Activities via Depth Silhouettes and ℜ Transformation for Smart Homes. Indoor Built Environ. 2011, doi:10.1177/1420326X11423163.
33. Frinken, V.; Fischer, A.; Baumgartner, M.; Bunke, H. Keyword spotting for self-training of BLSTM NN based handwriting recognition systems. Pattern Recognit. 2014, 47, 1073–1082.
34. Elmezain, M.; Al-Hamadi, A.; Michaelis, B. Real-time capable system for hand gesture recognition using hidden markov models in stereo color image sequences. J. WSCG 2008, 16, 65–72.
35. Elmezain, M.; Al-Hamadi, A.; Michaelis, B. Hand trajectory-based gesture spotting and recognition using HMM. In Proceedings of the 2009 16th IEEE International Conference on Image Processing (ICIP’09), Cairo, Egypt, 7–10 November 2009; pp. 3577–3580.
36. Tian, Y.; Cao, L.; Liu, Z.; Zhang, Z. Hierarchical filtered motion for action recognition in crowded videos. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 2012, 42, 313–323.
37. Xia, L.; Chen, C.-C.; Aggarwal, J. Human detection using depth information by kinect. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW’11), Colorado Springs, CO, USA, 20–25 June 2011; pp. 15–22.
38. Dalal, N.; Triggs, B. Histograms of oriented gradients for human detection. In Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), San Diego, CA, USA, 25 June 2005; pp. 886–893.
39. Ordonez, C. Clustering binary data streams with K-means. In Proceedings of the 8th ACM SIGMOD workshop on Research issues in data mining and knowledge discovery, San Diego, CA, USA, 13 June 2003; pp. 12–19.
40. Lee, H.-K.; Kim, J.-H. An HMM-based threshold model approach for gesture recognition. IEEE Trans. Pattern Anal. Mach. Intell. 1999, 21, 961–973.
41. Park, A.-Y.; Lee, S.-W. Gesture spotting in continuous whole body action sequences using discrete hidden markov models. In Proceedings of Gesture in Human-Computer Interaction and Simulation, Berder Island, France, 18–20 May 2005; Springer: pp. 100–111.
42. Yang, H.-D.; Park, A.-Y.; Lee, S.-W. Gesture spotting and recognition for human–robot interaction. IEEE Trans. Robot. 2007, 23, 256–270.
43. Bradski, G. R.; Davis, J. W. Motion segmentation and pose recognition with motion history gradients. Mach. Vis. Appl. 2002, 13, 174–184.
44. Yang, X.; Zhang, C.; Tian, Y. Recognizing actions using depth motion maps-based histograms of oriented gradients. In Proceedings of the 20th ACM international conference on Multimedia, Nara Japan, 29 October–2 November 2012; pp. 1057–1060.
45. Danafar, S.; Gheissari, N. Action recognition for surveillance applications using optic flow and SVM. In Proceedings of the Computer Vision-ACCV 2007, Tokyo, Japan, 18–22 November 2007; pp. 457–466.
46. Gonzalez-Pacheco, V.; Malfaz, M.; Fernandez, F.; Salichs, M.A. Teaching Human Poses Interactively to a Social Robot. Sensors 2013, 13, 12406–12430.