Action recognition using nonnegative action component representation and sparse basis selection

IEEE Transactions on Image Processing

Volumn 23, Issue 2, 2014, Pages 570-581

  Wang, Haoran ^a   Yuan, Chunfeng ^b   Hu, Weiming ^b   Ling, Haibin ^c   Yang, Wankou ^a   Sun, Changyin ^a  

^a SOUTHEAST UNIVERSITY   (China)

^b INSTITUTE OF AUTOMATION   (China)

^c Temple University   (United States)

Author keywords

action recognition; Action unit; nonnegative matrix factorization; sparse representation

Indexed keywords

ACTION RECOGNITION; ACTION REPRESENTATIONS; ACTION UNIT; GEOMETRICAL INFORMATIONS; HUMAN-ACTION RECOGNITION; NONNEGATIVE MATRIX FACTORIZATION; PART-BASED REPRESENTATION; SPARSE REPRESENTATION;

GESTURE RECOGNITION; IMAGE RECOGNITION; SEMANTICS;

MOTION ESTIMATION;

ALGORITHM; AUTOMATED PATTERN RECOGNITION; COMPUTER ASSISTED DIAGNOSIS; HUMAN; IMAGE ENHANCEMENT; IMAGE SUBTRACTION; MOTOR ACTIVITY; MOVEMENT (PHYSIOLOGY); PHOTOGRAPHY; PHYSIOLOGY; PROCEDURES; REPRODUCIBILITY; SENSITIVITY AND SPECIFICITY; THREE DIMENSIONAL IMAGING; WHOLE BODY IMAGING;

ALGORITHMS; HUMANS; IMAGE ENHANCEMENT; IMAGE INTERPRETATION, COMPUTER-ASSISTED; IMAGING, THREE-DIMENSIONAL; MOTOR ACTIVITY; MOVEMENT; PATTERN RECOGNITION, AUTOMATED; PHOTOGRAPHY; REPRODUCIBILITY OF RESULTS; SENSITIVITY AND SPECIFICITY; SUBTRACTION TECHNIQUE; WHOLE BODY IMAGING;

EID: 84891781429     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2013.2292550     Document Type: Article

References (84)

1. P. Turaga, R. Chellappa, V. S. Subrahmanian, and O. Udrea, Machine recognition of human activities: A survey, IEEE Trans. Circuits Syst. Video Technol., vol. 18, no. 11, pp. 1473-1488, Sep. 2008.
2. J. C. Niebles, H.Wang, and L. Fei-Fei, Unsupervised learning of human action categories using spatial-temporal words, in Proc. Int. J. Comput. Vis., vol. 79, no. 3, pp. 299-318, Sep. 2008.
3. J. Liu, J. Luo, and M. Shah, Recognizing realistic actions from videos 'in the wild', in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2009, pp. 1996-2003.
4. J. Niebles, C. Chen, and L. Fei-Fei, Modeling temporal structure of decomposable motion segments for activity classification, in Proc. Eur. Conf. Comput. Vis., 2010, pp. 392-405.
5. H. Wang, C. Yuan, W. Hu, and C. Sun, Supervised class-specific dictionary learning for sparse modeling in action recognition, Pattern Recognit., vol. 45, no. 11, pp. 3902-3911, 2012.
6. J. Liu, B. Kuipers, and S. Savarese, Recognizing human actions by attributes, in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2011, pp. 3337-3344.
7. J. Liu, S. Ali, and M. Shah, Recognizing human actions using multiple features, in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2008, pp. 1-8.
8. K. Rapantzikos, Y. Avrithis, and S. Kollias, Dense saliency-based spatiotemporal feature points for action recognition, in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2009, pp. 1454-1461.
9. W. Lee and H. Chen, Histogram-based interest point detectors, in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2009, pp. 1590-1596.
10. M. D. Rodriguez, J. Ahmed, and M. Shah, Action MACH a spatiotemporal maximum average correlation height filter for action recognition, in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2008, pp. 1-8.
11. H. Wang, M. M. Ullah, A. Klaser, I. Laptev, and C. Schmid, Evaluation of local spatio-temporal features for action recognition, in Proc. Brit. Mach. Vis. Conf., 2009, pp. 1-11.
12. P. Dollar, V. Rabaud, G. Cottrell, and S. Belongie, Behavior recognition via sparse spatiotemporal features, in Proc. 2nd Joint IEEE Int. Workshop Vis. Surveill. Perform. Eval. Track. Surveill., Oct. 2005, pp. 65-72.
13. I. Laptev, M. Marszalek, C. Schmid, and B. Rozenfeld, Learning realistic human actions from movies, in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2008, pp. 1-8.
14. S. Ali and M. Shah, Human action recognition in videos using kinematic features and multiple instance learning, IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 2, pp. 288-303, Feb. 2010.
15. A. F. Bobick and J. W. Davis, The recognition of human movement using temporal templates, IEEE Trans. Pattern Anal. Mach. Intell., vol. 23, no. 3, pp. 1257-1265, Mar. 2001.
16. A. Yilmaz and M. Shah, Actions sketch: A novel action representation, in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2005, pp. 984-989.
17. M. Blank, M. Irani, and R. Basri, Actions as space-time shapes, in Proc. 10th IEEE ICCV, Oct. 2005, pp. 1395-1402.
18. Z. Lin, Z. Jiang, and L. S. Davis, Recognizing actions by shapemotion prototype trees, in Proc. IEEE 12th Int. Conf. Comput. Vis., Sep./Oct. 2009, pp. 444-451.
19. F. Lv and R. Nevatia, Single view human action recognition using key pose matching and viterbi path searching, in Proc. IEEE Int. Conf. CVPR, Jun. 2007, pp. 1-8.
20. H. Wang, C. Yuan, G. Luo, W. Hu, and C. Sun, Action recognition using linear dynamic systems, Pattern Recognit., vol. 46, no. 6, pp. 1710-1718, 2013.
21. M. S. Ryoo and J. K. Aggarwal. (2010). An Overview of Contest on Semantic Description of Human Activities (SDHA)
22. [Online]. Available: http://cvrc.ece.utexas.edu/SDHA2010/Human- Interaction.html
23. M. Raptis and S. Soatto, Tracklet descriptors for action modeling and video analysis, in Proc. ECCV, 2010, pp. 577-590.
24. J. Sun, X. Wu, S. Yan, L. F. Cheong, T. S. Chua, and J. Li, Hierarchical spatio-temporal context modeling for action recognition, in Proc. IEEE Int. Conf. CVPR, Jun. 2009, pp. 2004-2011.
25. M. Bregonzio, S. Gong, and T. Xiang, Recognising action as clouds of space-time interest points, in Proc. IEEE Int. Conf. CVPR, Jun. 2009, pp. 1948-1955.
26. S. F. Wong, T. K. Kim, and R. Cipolla, Learning motion categories using both semantic and structural information, in Proc. IEEE Int. Conf. CVPR, Jun. 2007, pp. 1-6.
27. S. Savarese, A. Delpozo, J. Niebles, and L. Fei-Fei, Spatial-temporal correlations for unsupervised action classification, in Proc. IEEE Workshop Motion Video Comput., Jan. 2008, pp. 1-8.
28. I. Kotsia, S. Zafeiriou, and I. Pitas, Texture and shape information fusion for facial expression and facial action unit recognition, Pattern Recognit., vol. 41, no. 3, pp. 833-851, 2008.
29. H. Jiang, M. Crew, and Z. Li, Successive convex matching for action detection, in Proc. IEEE Int. Conf. CVPR, Jun. 2006, pp. 1646-1653.
30. A. Klaser, M. Marszalek, and C. Schmid, A spatio-temporal descriptor based on 3D-gradients, in Proc. Brit. Mach. Vis. Conf., 2008, pp. 1-10.
31. P. Scovanner, S. Ali, and M. Shah, A 3-dimensional SIFT descriptor and its application to action recognition, in Proc. ACM 15th Int. Conf. Multimedia, 2007, pp. 357-360.
32. C. Schuldt, I. Laptive, and B. Caputo, Recognizing human actions: A local SVM approach, in Proc. IEEE 17th ICPR, vol. 3. Aug. 2004, pp. 32-36.
33. Z. Zhang, Y. Hu, S. Chan, and L. Chia, Motion context: A new representation for human action recognition, in Proc. ECCV, 2008, pp. 817-829.
34. Y. Wang and G. Mori, Human action recognition by semi-latent topic models, IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 10, pp. 1762-1774, Oct. 2009.
35. J. Liu and M. Shah, Learning human actions via information maximization, in Proc. IEEE Int. Conf. CVPR, Jun. 2008, pp. 1-8.
36. S. Ali, A. Basharat, and M. Shah, Chaotic invariants for human action recognition, in Proc. IEEE 11th ICCV, Oct. 2007, pp. 1-8.
37. H. Jhuang, T. Serre, L. Wolf, and T. Poggio, A biologically inspired system for action recognition, in Proc. IEEE 11th ICCV, Oct. 2007, pp. 1-8.
38. K. Schindler and L. Gool
39. T. Berg, A. Berg, and J. Shih, Automatic attribute discovery and characterization from noisy web data, in Proc. ECCV, 2010, pp. 663-676.
40. A. Farhadi, I. Endres, D. Hoiem, and D. Forsyth, Describing objects by their attributes, in Proc. IEEE Int. Conf. CVPR, Jun. 2009, pp. 1778-1785.
41. S. E. Palmer, Hierarchical structure in perceptual representation, Cognit. Psychol., vol. 9, no. 4, pp. 441-474, 1977.
42. E. Wachsmuth, M. W. Oram, and D. I. Perrett, Recognition of objects and their component parts: Responses of single units in the temporal cortex of the macaque, Cereb. Cortex, vol. 4, no. 5, pp. 509-522, 1994.
43. P. Paatero and U. Tapper, Positive matrix factorization: A nonnegative factor model with optimal utilization of error estimates of data values, Environmetrics, vol. 5, no. 2, pp. 111-126, 1994.
44. D. D. Lee and H. S. Seung, Learning the parts of objects by nonnegative matrix factorization, Nature, vol. 401, pp. 788-791, Oct. 1999.
45. D. Cai, X. He, J. Han, and T. S. Huang, Graph regularized nonnegative matrix factorization for data representation, IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 8, pp. 1548-1560, Aug. 2011.
46. Y. Wang and G. Mori, Max-margin hidden conditional random fields for human action recognition, in Proc. IEEE Int. Conf. CVPR, Jun. 2009, pp. 872-879.
47. D. Ramanan and D. Forsyth, Automatic annotation of everyday movements, in Advances in Neural Information Processing Systems. Cambridge, MA, USA: MIT Press, 2003.
48. J. Liu, Y. Yang, and M. Shah, Learning semantic visual vocabularies using diffusion distance, in Proc. IEEE Int. Conf. CVPR, Jun. 2009, pp. 461-468.
49. A. Fathi and G. Mori, Action recognition by learning mid-level motion features, in Proc. IEEE Int. Conf. CVPR, Jun. 2008, pp. 1-8.
50. H. Wang, A. Klaser, C. Schmid, and C. Liu Dense trajectories and motion boundary descriptors for action recognition, Int. J. Comput. Vis., vol. 103, no. 1, pp. 60-79, 2013.
51. C. Lampert, H. Nickisch, and S. Harmeling, Learning to detect unseen object classes by between-class attribute transfer, in Proc. IEEE Int. Conf. CVPR, Jun. 2009, pp. 951-958.
52. J. Vogel and B. Schiele, Semantic modeling of natural scenes for content-based image retrieval, Int. J. Comput. Vis., vol. 72, no. 2, pp. 133-157, 2007.
53. G. Wang, D. Hoiem, and D. Forsyth, Learning image similarity from Flickr groups using stochastic intersection kernel machines, in Proc. IEEE 12th Int. Conf. Comput. Vis., Sep./Oct. 2009, pp. 428-435.
54. L. J. Li, C. Wang, Y. Lim, D. Blei, and F. Li, Building and using a semantivisual image hierarchy, in Proc. IEEE Int. Conf. CVPR, Jun. 2010, pp. 3336-3343.
55. L. Torresani, M. Szummer, and A. Fitzgibbon, Efficient object category recognition using classemes, in Proc. Eur. Conf. Comput. Vis., 2010, pp. 776-789.
56. A. Gilbert, J. Illingworth, and R. Bowden, Fast realistic multi-action recognition using mined dense spatio-temporal features, in Proc. IEEE 12th Int. Conf. Comput. Vis., Sep./Oct. 2009, pp. 925-931.
57. A. Kovashka and K. Grauman, Learning a hierarchy of discriminative space-time neighborhood features for human action recognition, in Proc. IEEE Int. Conf. CVPR, Jun. 2010, pp. 2046-2053.
58. C. Yuan, X. Li, W. Hu, H. Ling, and S. Maybank, 3D R transform on spatio-temporal interest points for action recognition, in Proc. IEEE Conf. CVPR, Jun. 2013, pp. 724-730.
59. C. Ding, D. Zhou, X. He, and H. Zha, R1-PCA: Rotational invariant L1-norm principal component analysis for robust subspace factorization, in Proc. 23rd ICML, 2006, pp. 281-288.
60. A. Argyriou, T. Evgeniou, and M. Pontil, Multi-task feature learning, in Advances in Neural Information Processing Systems. Cambridge, MA, USA: MIT Press, 2007.
61. G. Obozinski, B. Taskar, and M. Jordan, Multi-task feature selection, Dept. Statist., Univ. California, Berkeley, CA, USA, Tech. Rep., 2006.
62. H. Huang and C. Ding, Robust tensor factorization using R1 norm, in Proc. IEEE Int. Conf. CVPR, Jun. 2008, pp. 1-8.
63. F. R. K. Chung, Spectral Graph Theory. Providence, RI, USA: AMS, 1997.
64. W. Brendel and S. Todorovic, Activities as time series of human postures, in Proc. ECCV, 2010, pp. 721-734.
65. B. Li, M. Ayazoglu, T. Mao, O. Camps, and M. Sznaier, Activity recognition using dynamic subspace angles, in Proc. IEEE Int. Conf. CVPR, Jun. 2011, pp. 3193-3200.
66. Q. V. Le, W. Y. Zou, S. Y. Yeung, and A. Y. Ng, Learning hierarchical invariant spatio-temporal features for action recognition with independent subspace analysis, in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2011, pp. 3361-3368.
67. L. Yeffet and L. Wolf, Local trinary patterns for human action recognition, in Proc. IEEE 12th ICCV, Sep./Oct. 2009, pp. 492-497.
68. A. Yao, J. Gall, and L. Van Gool, A Hough transform-based voting framework for action recognition, in Proc. IEEE Int. Conf. CVPR, Jun. 2010, pp. 2061-2068.
69. Q. Qiu, Z. Jiang, and R. Chellappa, Sparse dictionary-based representation and recognition of action attributes, in Proc. IEEE ICCV, Nov. 2011, pp. 707-714.
70. J. Mairal, F. Bach, J. Ponce, G. Sapiro, and A. Zisserman, Supervised dictionary learning, in Advances in Neural Information Processing Systems. Cambridge, MA, USA: MIT Press, 2008.
71. D. M. Bradley and J. A. Bagnell, Differential sparse coding, in Advances in Neural Information Processing Systems. Cambridge, MA, USA: MIT Press, 2008.
72. J. Liu, Y. Yang, I. Saleemi, and M. Shah, Learning semantic features for action recognition via diffusion maps, Comput. Vis. Image Understand., vol. 116, no. 3, pp. 361-377, 2012.
73. N. Ikizler-Cinbis and S. Sclaroff, Object, scene and actions: Combining multiple features for human action recognition, in Proc. ECCV, 2010, pp. 494-507.
74. M. Marszalek, I. Laptev, and C. Schmid, Actions in context, in Proc. IEEE Int. Conf. CVPR, Jun. 2009, pp. 2929-2936.
75. S. Sadanand and J. Corso, Action bank: A high-level representation of activity in video, in Proc. IEEE Int. Conf. CVPR, Jun. 2012, pp. 1234-1241.
76. M. Raptis and L. Sigal, Poselet key-framing: A model for human activity recognition, in Proc. IEEE Int. Conf. CVPR, Jun. 2013, pp. 2650-2657.
77. T. Lan, Y. Wang, and G. Mori, Discriminative figure-centric models for joint action localization and recognition, in Proc. IEEE ICCV, Nov. 2011, pp. 2003-2010.
78. Y. Tian, R. Sukthankar, and M. Shah, Spatiotemporal deformable part models for action detection, in Proc. IEEE Int. Conf. CVPR, Jun. 2013, pp. 1-8.
79. M. Ryoo, Human activity prediction: Early recognition of ongoing activities from streaming videos, in Proc. IEEE ICCV, Nov. 2011, pp. 1036-1043.
80. Y. Zhang, X. Liu, M. Chang, X. Ge, and T. Chen, Spatio-temporal phrases for activity recognition, in Proc. ECCV, 2012, pp. 707-721.
81. A. Vahdat, B. Gao, M. Ranjbar, and G. Mori, A discriminative key pose sequence model for recognizing human interactions, in Proc. IEEE ICCV Workshops, Nov. 2011, pp. 1729-1736.
82. A. Gaidon, Z. Harchaoui, and C. Schmid, Temporal localization of actions with actoms, IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 11, pp. 2782-2795, Nov. 2013.
83. M. Ullah, S. Parizi, and I. Laptev, Improving bag-of-features action recognition with non-local cues, in Proc. Brit. Mach. Vis. Conf., 2010, pp. 1-11.
84. M. Raptis, I. Kokkinos, and S. Soatto, Discovering discriminative action parts from mid-level video representations, in Proc. IEEE Int. Conf. CVPR, Jun. 2012, pp. 1242-1249.