Multilevel depth and image fusion for human activity detection

IEEE Transactions on Cybernetics

Volumn 43, Issue 5, 2013, Pages 1383-1394

  Ni, Bingbing ^a   Pei, Yong ^a   Moulin, Pierre ^b   Yan, Shuicheng ^c  

^a ADVANCED DIGITAL SCIENCES CENTER   (Singapore)

^b University of Illinois at Urbana Champaign   (United States)

^c NATIONAL UNIVERSITY OF SINGAPORE   (Singapore)

Author keywords

Action recognition and localization; Depth sensor; Spatial and temporal context

Indexed keywords

CHEMICAL DETECTION; IMAGE FUSION; IMAGE PROCESSING; PIPELINE PROCESSING SYSTEMS; VIDEO SIGNAL PROCESSING;

ACTION RECOGNITION; ACTIVITY RECOGNITION; DEPTH SENSORS; HUMAN-ACTIVITY DETECTION; INTEGRATING INFORMATION; LOCALIZATION ACCURACY; SPATIAL AND TEMPORAL CONTEXT; STRUCTURAL MODELING;

FEATURE EXTRACTION;

ACTIMETRY; ALGORITHM; ARTIFICIAL INTELLIGENCE; AUTOMATED PATTERN RECOGNITION; COMPUTER; COMPUTER SIMULATION; COMPUTER SYSTEM; DEVICES; HUMAN; IMAGE ENHANCEMENT; IMAGE SUBTRACTION; PROCEDURES; RECREATION; THREE DIMENSIONAL IMAGING; TRANSDUCER; WHOLE BODY IMAGING; ARTICLE; EQUIPMENT; METHODOLOGY;

ACTIGRAPHY; ALGORITHMS; ARTIFICIAL INTELLIGENCE; COMPUTER PERIPHERALS; COMPUTER SIMULATION; COMPUTER SYSTEMS; HUMANS; IMAGE ENHANCEMENT; IMAGING, THREE-DIMENSIONAL; PATTERN RECOGNITION, AUTOMATED; SUBTRACTION TECHNIQUE; TRANSDUCERS; VIDEO GAMES; WHOLE BODY IMAGING;

EID: 84890390556     PISSN: 21682267     EISSN: None     Source Type: Journal    
DOI: 10.1109/TCYB.2013.2276433     Document Type: Article

References (31)

1. F. Lv and R. Nevatia, "Single view human action recognition using key pose matching and Viterbi path searching," in Proc. IEEE Int. Conf. Comput. Vision Pattern Recognit., Jun. 2007, pp. 1-8.
2. D. Weinland and E. Boyer, "Action recognition using exemplar-based embedding," in Proc. IEEE Int. Conf. Comput. Vision Pattern Recognit., Jun. 2008, pp. 1-7.
3. A. Bobick and J. Davis, "The representation and recognition of action using temporal templates," IEEE Trans. Pattern Anal. Mach. Intell., vol. 23, no. 3, pp. 257-267, Mar. 2001.
4. K. Guo, P. Ishwar, and J. Konrad, "Action recognition in video by sparse representation on covariance manifolds of silhouette tunnels," in Proc. IEEE Int. Conf. Pattern Recognit., 2010.
5. L. Gorelick, M. Blank, E. Shechtman, M. Irani, and R. Basri, "Actions as space-time shapes," IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 12, pp. 2247-2253, Dec. 2007.
6. I. Laptev and T. Lindeberg, "Space-time interest points," in Proc. IEEE Int. Conf. Comput. Vision, Oct. 2003, pp. 432-439.
7. A. Klaser, M. Marszalek, and C. Schmid, "A spatio-temporal descriptor based on 3D gradients," in Proc. British Mach. Vision Conf., 2008.
8. P. Dollar, V. Rabaud, G. Cottrell, and S. Belongie, "Behavior recognition via sparse spatio-temporal features," in Proc. IEEE Int. Workshop Visual Surveillance Performance Evaluation Tracking Surveillance, Oct. 2005, pp. 65-72.
9. H. Wang, A. Kläser, C. Schmid, and L. Cheng-Lin, "Action recognition by dense trajectories," in Proc. IEEE Int. Conf. Comput. Vision Pattern Recognit., Jun. 2011, pp. 3169-3176.
10. M. Raptis, I. Kokkinos, and S. Soatto, "Discovering discriminative action parts from mid-level video representations," in Proc. IEEE Int. Conf. Comput. Vision Pattern Recognit., 2012.
11. T. Lan, Y.Wang,W. Yang, and G. Mori, "Beyond actions: Discriminative models for contextual group activities," in Proc. Adv. Neural Inform. Process. Syst., 2010, pp. 1216-1224.
12. B. Yao and F.-F. Li, "Recognizing human-object interactions in still images by modeling the mutual context of objects and human poses," IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 9, pp. 1691-1703, Sep. 2012.
13. J. Han, L. Shao, D. Xu, and J. Shotton, "Enhanced computer vision with microsoft kinect sensor: A review," IEEE Trans. Cybern., vol. 43, no. 5, pp. 1318-1334, Oct. 2013.
14. W. Li, Z. Zhang, and Z. Liu, "Action recognition based on a bag of 3D points," in Proc. CVPR Workshop Human Commun. Behavior Anal., 2010.
15. B. Ni, G. Wang, and P. Moulin, "RGBD-HuDaAct: A color-depth video database for human daily activity recognition," in Proc. ICCV Workshops Consum. Depth Cameras, 2011, pp. 1147-1153.
16. L. Liu and L. Shao, "Learning discriminative representations from RGBD video data," in Proc. Int. Joint Conf. Artif. Intell., 2013.
17. J. Wang, Z. Liu, Y. Wu, and J. Yuan, "Mining actionlet ensemble for action recognition with depth cameras," in Proc. IEEE Int. Conf. Comput. Vision Pattern Recognit., 2012, pp. 1290-1297.
18. J. Sung, C. Ponce, B. Selman, and A. Saxena, "Human activity detection from RGBD images," in Proc. AAAI Workshop Pattern, Activity, Intent Recognit., 2011.
19. N. Dalal and B. Triggs, "Histograms of oriented gradients for human detection," in Proc. IEEE Int. Conf. Comput. Vision Pattern Recognit., vol. 1. Jun. 2005, pp. 886-893.
20. P. F. Felzenszwalb, R. B. Girshick, D. McAllester, and D. Ramanan, "Object detection with discriminatively trained part based models," IEEE Trans. Pattern Anal. Machine Intell., vol. 32, no. 9, pp. 1627-1645, Jul. 2010.
21. K. Mikolajczyk and H. Uemura, "Action recognition with motion-appearance vocabulary forest," in Proc. IEEE Int. Conf. Comput. Vision Pattern Recognit., Jun. 2008, pp. 1-8.
22. A. Yao, J. Gall, and L. Van Gool, "A Hough transform-based voting framework for action recognition," in Proc. IEEE Int. Conf. Comput. Vision Pattern Recognit., Jun. 2010, pp. 2061-2068.
23. C.-C. Chang and C.-J. Lin, "LIBSVM: A library for support vector machines," ACM Trans. Intell. Syst. Technol., vol. 2, pp. 27:1-27:27, Sep. 2011.
24. Y. Wang and G. Mori, "Hidden part models for human action recognition: Probabilistic versus max margin," IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 7, pp. 1310-1323, May 2011.
25. [Online]. Available: http://pr.cs.cornell.edu/humanactivities/results. php."
26. H. S. Koppula, R. Gupta, and A. Saxena, "Learning human activities and object affordances from RGB-D videos," Int. J. Robot. Res., vol. 32, no. 8, pp. 951-970, Jul. 2013.
27. X. Yang and Y. Tian, "Effective 3D action recognition using eigenjoints," J. Visual Commun. Image Representation, to be published.
28. W. Choi, K. Shahid, and S. Savarese, "What are they doing? Collective activity classification using spatio-temporal relationship among people," in Proc. Int. Workshop Visual Surveillance, 2009.
29. J. C. Niebles, H. Wang, and L. Fei-fei, "Unsupervised learning of human action categories using spatial-temporal words," in Proc. BMVC, 2006.
30. C. Wolf, J. Mille, L. Lombardi, O. Celiktutan, M. Jiu, M. Baccouche, E. Dellandrea, C. Bichot, C. Garcia, and B. Sankur, "The LIRIS human activities dataset and the ICPR human activities recognition and localization competition," Tech. Rep. RR-LIRIS-2012-004, LIRIS Laboratory, 2012.
31. J. Yuan, Z. Liu, and Y. Wu, "Discriminative video pattern search for efficient action detection," IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 9, pp. 1728-1743, Feb. 2011.