Latent hierarchical model of temporal structure for complex activity classification

IEEE Transactions on Image Processing

Volumn 23, Issue 2, 2014, Pages 810-822

  Wang, Limin ^a   Qiao, Yu ^b   Tang, Xiaoou ^a  

^a CHINESE UNIVERSITY OF HONG KONG   (Hong Kong)

^b SHENZHEN INSTITUTES OF ADVANCED TECHNOLOGY   (China)

Author keywords

Activity classification; Cascade inference; Deep structure; Hierarchical model; Latent learning

Indexed keywords

ACTIVITY CLASSIFICATIONS; DEEP STRUCTURE; HIERARCHICAL MODEL; INFERENCE PROCESS; LATENT LEARNING; STATE-OF-THE-ART PERFORMANCE; TEMPORAL DISPLACEMENT; TEMPORAL STRUCTURES;

SPORTS;

HIERARCHICAL SYSTEMS;

EID: 84892588122     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2013.2295753     Document Type: Article

References (53)

1. J. K. Aggarwal and M. S. Ryoo. Human activity analysis: A review. ACM Comput. Surv., vol. 43, no. 3, p. 16, 2011
2. P. K. Turaga, R. Chellappa, V. S. Subrahmanian, and O. Udrea. Machine recognition of human activities: A survey. IEEE Trans. Circuits Syst. Video Technol., vol. 18, no. 11, pp. 1473-1488, Nov. 2008
3. D. A. Forsyth, O. Arikan, L. Ikemoto, J. F. O'Brien, and D. Ramanan. Computational studies of human motion: Part 1, tracking and motion synthesis. Found. Trends Comput. Graph. Vis., vol. 1, nos. 2-3, pp. 77-254, 2005
4. J. C. Nascimento, M. A. T. Figueiredo, and J. S. Marques. Trajectory classification using switched dynamical hidden Markov models. IEEE Trans. Image Process., vol. 19, no. 5, pp. 1338-1348, May 2010
5. F. I. Bashir, A. A. Khokhar, and D. Schonfeld. Object trajectory-based activity classification and recognition using hidden Markov models. IEEE Trans. Image Process., vol. 16, no. 7, pp. 1912-1919, Jul. 2007 (Pubitemid 47018887)
6. A. Jaimes and N. Sebe. Multimodal human-computer interaction: A survey. Comput. Vis. Image Understand., vol. 108, nos. 1-2, pp. 116-134, 2007 (Pubitemid 47333395)
7. A. Ekin, A. M. Tekalp, and R. Mehrotra. Automatic soccer video analysis and summarization. IEEE Trans. Image Process., vol. 12, no. 7, pp. 796-807, Jul. 2003
8. C. G. M. Snoek and M. Worring. Concept-based video retrieval. Found. Trends Inf. Retr., vol. 2, no. 4, pp. 215-322, 2009
9. N. Oliver, B. Rosario, and A. Pentland. A Bayesian computer vision system for modeling human interactions. IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 8, pp. 831-843, Aug. 2000
10. S. B. Wang, A. Quattoni, L-P. Morency, D. Demirdjian, and T. Darrell. Hidden conditional random fields for gesture recognition. in Proc. IEEE Comput. Soc. Conf. CVPR, Jun. 2006, pp. 1521-1527 (Pubitemid 44931500)
11. Y. Wang and G. Mori. Hidden part models for human action recognition: Probabilistic versus max margin. IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 7, pp. 1310-1323, Jul. 2011
12. J. C. Niebles, C-W. Chen, and F-F. Li. Modeling temporal structure of decomposable motion segments for activity classification. in Proc. ECCV, 2010, pp. 392-405
13. J. Wang, Z. Chen, and Y. Wu. Action recognition with multiscale spatio-Temporal contexts. in Proc. IEEE Conf. CVPR, Jun. 2011, pp. 3185-3192
14. K. Tang, F-F. Li, and D. Koller. Learning latent temporal structure for complex event detection. in Proc. IEEE Conf. CVPR, Jun. 2012, pp. 1250-1257
15. L. Wang, Y. Qiao, and X. Tang. Mining motion atoms and phrases for complex action recognition. in Proc. IEEE Conf. ICCV, Dec. 2013, pp. 2680-2687
16. P. F. Felzenszwalb, R. B. Girshick, D. A. McAllester, and D. Ramanan. Object detection with discriminatively trained part-based models. IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 9, pp. 1627-1645, Sep. 2010
17. M. Rohrbach, M. Regneri, M. Andriluka, S. Amin, M. Pinkal, and B. Schiele. Script data for attribute-based recognition of composite activities. in Proc. 12th ECCV, 2012, pp. 144-157
18. H. Wang, M. M. Ullah, A. Kläser, I. Laptev, and C. Schmid. Evaluation of local spatio-Temporal features for action recognition. in Proc. BMVC, 2009, pp. 1-11
19. I. Laptev. On space-Time interest points. Int. J. Comput. Vis., vol. 64, nos. 2-3, pp. 107-123, 2005 (Pubitemid 41314645)
20. G. Willems, T. Tuytelaars, and L. J. V. Gool. An efficient dense and scale-invariant spatio-Temporal interest point detector. in Proc. ECCV, 2008, pp. 650-663
21. P. Dollar, V. Rabaud, G. Cottrell, and S. Belongie. Behavior recognition via sparse spatio-Temporal features. in Proc. 2nd Joint IEEE Int. Workshop VS-PETS, Oct. 2005, pp. 65-72 (Pubitemid 46178025)
22. H. Wang, A. Kläser, C. Schmid, and C-L. Liu. Action recognition by dense trajectories. in Proc. IEEE Conf. CVPR, Jun. 2011, pp. 3169-3176
23. I. Laptev, M. Marszalek, C. Schmid, and B. Rozenfeld. Learning realistic human actions from movies. in Proc. IEEE Conf. CVPR, Jun. 2008, pp. 1-8
24. A. Kläser, M. Marszalek, and C. Schmid. A spatio-Temporal descriptor based on 3D-gradients. in Proc. BMVC, 2008, pp. 1-10
25. X. Peng, Y. Qiao, Q. Peng, and X. Qi. Exploring motion boundary based sampling and spatial-Temporal context descriptors for action recognition. in Proc. BMVC, 2013, pp. 1-11
26. X. Wang, L. Wang, and Y. Qiao. A comparative study of encoding, pooling and normalization methods for action recognition. in Proc. ACCV, 2012, pp. 572-585
27. G. Csurka, C. Dance, L. Fan, J. Willamowski, and C. Bray. Visual categorization with bags of keypoints. in Proc. ECCV Workshop Statist. Learn. Comput. Vis., vol. 1. 2004, pp. 1-22
28. J. V. Gemert, J-M. Geusebroek, C. J. Veenman, and A. W. M. Smeulders. Kernel codebooks for scene categorization. in Proc. ECCV, 2008, pp. 696-709
29. J. Sánchez, F. Perronnin, T. Mensink, and J. J. Verbeek. Image classification with the fisher vector: Theory and practice. Int. J. Comput. Vis., vol. 105, no. 3, pp. 222-245, 2013
30. J. Yang, K. Yu, Y. Gong, and T. S. Huang. Linear spatial pyramid matching using sparse coding for image classification. in Proc. IEEE Conf. CVPR, Jun. 2009, pp. 1794-1801
31. J. Wang, J. Yang, K. Yu, F. Lv, T. S. Huang, and Y. Gong. Localityconstrained linear coding for image classification. in Proc. IEEE Conf. CVPR, Jun. 2010, pp. 3360-3367
32. L. Wang, Y. Qiao, and X. Tang. Motionlets: Mid-level 3D parts for human motion recognition. in Proc. IEEE Conf. CVPR, Jun. 2013, pp. 2674-2681
33. S. Sadanand and J. J. Corso. Action bank: A high-level representation of activity in video. in Proc. IEEE Conf. CVPR, Jun. 2012, pp. 1234-1241
34. B. Laxton, J. Lim, and D. J. Kriegman. Leveraging temporal, contextual and ordering constraints for recognizing complex activities in video. in Proc. IEEE Conf. CVPR, Jun. 2007, pp. 1-8
35. C. Bishop, Pattern Recognition and Machine Learning, vol. 4. New York, NY, USA: Springer-Verlag, 2006
36. T. Serre, G. Kreiman, M. Kouh, C. Cadieu, U. Knoblich, and T. Poggio. A quantitative theory of immediate visual recognition. Progr. Brain Res., vol. 165, pp. 33-56, 2007 (Pubitemid 47520714)
37. N. Krüger, P. Janssen, S. Kalkan, M. Lappe, A. Leonardis, J. H. Piater, et al. Deep hierarchies in the primate visual cortex: What can we learn for computer vision? IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 8, pp. 1847-1871, Aug. 2013
38. Y. Bengio. Learning deep architectures for AI. Found. Trends Mach. Learn., vol. 2, no. 1, pp. 1-127, 2009
39. A. Kovashka and K. Grauman. Learning a hierarchy of discriminative space-Time neighborhood features for human action recognition. in Proc. IEEE Conf. CVPR, Jun. 2010, pp. 2046-2053
40. L. Zhu, Y. Chen, A. L. Yuille, and W. T. Freeman. Latent hierarchical structural learning for object detection. in Proc. IEEE Conf. CVPR, Jun. 2010, pp. 1062-1069
41. Y. Chen, L. Zhu, and A. L. Yuille. Active mask hierarchies for object detection. in Proc. ECCV, 2010, pp. 43-56
42. L. Zhu, Y. Chen, Y. Lu, C. Lin, and A. L. Yuille. Max margin AND/OR graph learning for parsing the human body. in Proc. IEEE Conf. CVPR, Jun. 2008, pp. 1-8
43. S. C. Zhu and D. Mumford. A stochastic grammar of images. Found. Trends Comput. Graph. Vis., vol. 2, no. 4, pp. 259-362, 2006
44. A. Gupta, P. Srinivasan, J. Shi, and L. S. Davis. Understanding videos, constructing plots learning a visually grounded storyline model from annotated videos. in Proc. IEEE Conf. CVPR, Jun. 2009, pp. 2012-2019
45. B. Leibe, A. Leonardis, and B. Schiele. An implicit shape model for combined object categorization and segmentation. in Toward Category-Level Object Recognition. New York, NY, USA: Springer-Verlag, 2006, pp. 508-524
46. C. J. C. Burges. A tutorial on support vector machines for pattern recognition. Data Mining Knowl. Discovery, vol. 2, no. 2, pp. 121-167, 1998 (Pubitemid 128695475)
47. P. F. Felzenszwalb, R. B. Girshick, and D. A. McAllester. Cascade object detection with deformable part models. in Proc. IEEE Conf. CVPR, Jun. 2010, pp. 2241-2248
48. C. Schüldt, I. Laptev, and B. Caputo. Recognizing human actions: A local SVM approach. in Proc. 17th ICPR, Aug. 2004, pp. 32-36
49. M. Marszalek, I. Laptev, and C. Schmid. Actions in context. in Proc. IEEE Conf. CVPR, Jun. 2009, pp. 2929-2936
50. C-C. Chang and C-J. Lin. LIBSVM: A library for support vector machines. ACM Trans. Intell. Syst. Technol., vol. 2, no. 3, p. 27, 2011
51. G. W. Taylor, R. Fergus, Y. LeCun, and C. Bregler. Convolutional learning of spatio-Temporal features. in Proc. ECCV, 2010, pp. 140-153
52. H. Pirsiavash and D. Ramanan. Detecting activities of daily living in first-person camera views. in Proc. IEEE Conf. CVPR, Jun. 2012, pp. 2847-2854
53. H. Wang, A. Kläser, C. Schmid, and C-L. Liu. Dense trajectories and motion boundary descriptors for action recognition. Int. J. Comput. Vis., vol. 103, no. 1, pp. 60-79, 2013