Action Recognition from Arbitrary Views Using Transferable Dictionary Learning

IEEE Transactions on Image Processing

Volumn 27, Issue 10, 2018, Pages 4709-4723

  Zhang, Jingtian ^a   Shum, Hubert P H ^a   Han, Jungong ^b   Shao, Ling ^c  

^a NORTHUMBRIA UNIVERSITY   (United Kingdom)

^b LANCASTER UNIVERSITY   (United Kingdom)

^c Rutgers University   (United Arab Emirates)

Author keywords

3D dense trajectories; Action recognition; transfer dictionary learning; view invariance

Indexed keywords

FEATURE EXTRACTION; HUMAN COMPUTER INTERACTION; PERSONNEL TRAINING; SECURITY SYSTEMS; TRAJECTORIES;

ACTION RECOGNITION; DICTIONARY LEARNING; SHAPE; SOLID MODEL; TWO-DIMENSIONAL DISPLAYS; VIEW INVARIANCE;

THREE DIMENSIONAL DISPLAYS;

EID: 85047017388     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2018.2836323     Document Type: Article

References (69)

1. M. Blank, L. Gorelick, E. Shechtman, M. Irani, and R. Basri, "Actions as space-Time shapes, " in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), vol. 2, Oct. 2005, pp. 1395-1402.
2. P. Dollar, V. Rabaud, G. Cottrell, and S. Belongie, "Behavior recognition via sparse spatio-Temporal features, " in Proc. 2nd Joint IEEE Int. Workshop Vis. Surveill. Perform. Eval. Tracking Surveill., Oct. 2005, pp. 65-72.
3. I. Laptev, "On space-Time interest points, " Int. J. Comput. Vis., vol. 64, nos. 2-3, pp. 107-123, 2005.
4. J. Liu and M. Shah, "Learning human actions via information maximization, " in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2008, pp. 1-8.
5. J. Liu, Y. Yang, and M. Shah, "Learning semantic visual vocabularies using diffusion distance, " in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2009, pp. 461-468.
6. Z. Lin, Z. Jiang, and L. S. Davis, "Recognizing actions by shapemotion prototype trees, " in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), Sep. 2009, pp. 444-451.
7. F. Lv and R. Nevatia, "Single view human action recognition using key pose matching and Viterbi path searching, " in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2007, pp. 1-8.
8. S. Wu, O. Oreifej, and M. Shah, "Action recognition in videos acquired by a moving camera using motion decomposition of Lagrangian particle trajectories, " in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), Nov. 2011, pp. 1419-1426.
9. H. Wang, A. Kläser, C. Schmid, and C.-L. Liu, "Dense trajectories and motion boundary descriptors for action recognition, " Int. J. Comput. Vis., vol. 103, no. 1, pp. 60-79, 2013.
10. H. Wang, A. Kläser, C. Schmid, and C.-L. Liu, "Action recognition by dense trajectories, " in Proc. IEEE Int. Conf. Comput. Vis. (CVPR), Jun. 2011, pp. 3169-3176.
11. H. Wang and C. Schmid, "Action recognition with improved trajectories, " in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), Dec. 2013, pp. 3551-3558.
12. A. Karpathy, G. Toderici, S. Shetty, T. Leung, R. Sukthankar, and L. Fei-Fei, "Large-scale video classification with convolutional neural networks, " in Proc. IEEE Int. Conf. Comput. Vis. (CVPR), Jun. 2014, pp. 1725-1732.
13. K. Simonyan and A. Zisserman, "Two-stream convolutional networks for action recognition in videos, " in Proc. Adv. Neural Inf. Process. Syst., 2014, pp. 568-576.
14. D. Tran, L. Bourdev, R. Fergus, L. Torresani, and M. Paluri, "Learning spatiotemporal features with 3D convolutional networks, " in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), Dec. 2015, pp. 4489-4497.
15. L. Wang et al., "Temporal segment networks: Towards good practices for deep action recognition, " in Proc. Eur. Conf. Comput. Vis., 2016, pp. 20-36.
16. A. Gupta, J. Martinez, J. J. Little, and R. J. Woodham, "3D pose from motion for cross-view action recognition via non-linear circulant temporal encoding, " in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), 2014, pp. 2601-2608.
17. H. Rahmani and A. Mian, "Learning a non-linear knowledge transfer model for cross-view action recognition, " in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2015, pp. 2458-2466.
18. A. Kovashka and K. Grauman, "Learning a hierarchy of discriminative space-Time neighborhood features for human action recognition, " in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2010, pp. 2046-2053.
19. A. Farhadi and M. K. Tabrizi, "Learning to recognize activities from the wrong view point, " in Proc. Eur. Conf. Comput. Vis., 2008, pp. 154-166.
20. J. Liu, M. Shah, B. Kuipers, and S. Savarese, "Cross-view action recognition via view knowledge transfer, " in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2011, pp. 3209-3216.
21. J. Wang, X. Nie, Y. Xia, Y. Wu, and S.-C. Zhu, "Cross-view action modeling, learning, and recognition, " in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), 2014, pp. 2649-2656.
22. C.-H. Huang, Y.-R. Yeh, and Y.-C. F. Wang, "Recognizing actions across cameras by exploring the correlated subspace, " in Proc. Eur. Conf. Comput. Vis., 2012, pp. 342-351.
23. D. Weinland, E. Boyer, and R. Ronfard, "Action recognition from arbitrary views using 3D exemplars, " in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), Oct. 2007, pp. 1-7.
24. P. Yan, S. M. Khan, and M. Shah, "Learning 4D action feature models for arbitrary view action recognition, " in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2008, pp. 1-7.
25. H. Rahmani, A. Mian, and M. Shah, "Learning a deep model for human action recognition from novel viewpoints, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 40, no. 3, pp. 667-681, Mar. 2018.
26. N. Iwamoto, H. P. H. Shum, L. Yang, and S. Morishima, "Multi-layer lattice model for real-Time dynamic character deformation, " Comput. Graph. Forum, vol. 34, no. 7, pp. 99-109, Oct. 2015.
27. C. Hecker, B. Raabe, R. W. Enslow, J. DeWeese, J. Maynard, and K. van Prooijen, "Real-Time motion retargeting to highly varied usercreated morphologies, " ACM Trans. Graph., vol. 27, no. 3, 2008, Art. no. 27.
28. J. Zhang, L. Zhang, H. P. H. Shum, and L. Shao, "Arbitrary view action recognition via transfer dictionary learning on synthetic training data, " in Proc. IEEE Int. Conf. Robot. Automat. (ICRA), May 2016, pp. 1678-1684.
29. M. Ankerst, G. Kastenmöller, H.-P. Kriegel, and T. Seidl, "3D shape histograms for similarity search and classification in spatial databases, " in Proc. Int. Symp. Spatial Databases, 1999, pp. 207-226.
30. M. Körtgen, G.-J. Park, M. Novotni, and R. Klein, "3D shape matching with 3D shape contexts, " in Proc. 7th Central Eur. Seminar Comput. Graph., vol. 3, 2003, pp. 5-17.
31. P. Huang and A. Hilton, "Shape-colour histograms for matching 3D video sequences, " in Proc. IEEE 12th Int. Conf. Comput. Vis. Workshops (ICCV Workshops), Sep. 2009, pp. 1510-1517.
32. S. Pehlivan and P. Duygulu, "A new pose-based representation for recognizing actions from multiple cameras, " Comput. Vis. Image Understand., vol. 115, no. 2, pp. 140-151, 2011.
33. D. Weinland, R. Ronfard, and E. Boyer, "Free viewpoint action recognition using motion history volume, " Comput. Vis. Image Understand., vol. 104, nos. 2-3, pp. 249-257, 2006.
34. M. Pierobon, M. Marcon, A. Sarti, and S. Tubaro, "3-D body posture tracking for human action template matching, " in Proc. IEEE Int. Conf. Acoust., Speech Signal Process. (ICASSP), vol. 2, 2006, pp. 501-504.
35. I. Cohen and H. Li, "Inference of human postures by classification of 3D human body shape, " in Proc. IEEE Int. Workshop Anal. Modeling Faces Gestures (AMFG), Oct. 2003, pp. 74-81.
36. B. Zhang, Y. Yang, C. Chen, L. Yang, J. Han, and L. Shao, "Action recognition using 3D histograms of texture and a multi-class boosting classifier, " IEEE Trans. Image Process., vol. 26, no. 10, pp. 4648-4660, Oct. 2017.
37. M. Liu, H. Liu, C. Chen, and M. Najafian, "Energy-based global ternary image for action recognition using sole depth sequences, " in Proc. 4th Int. Conf. 3D Vis. (3DV), 2016, pp. 47-55.
38. M. Liu, H. Liu, and C. Chen, "Enhanced skeleton visualization for view invariant human action recognition, " Pattern Recognit., vol. 68, pp. 346-362, Aug. 2017.
39. K. Wang, G. Zhang, and S. Xia, "Templateless non-rigid reconstruction and motion tracking with a single RGB-D camera, " IEEE Trans. Image Process., vol. 26, no. 12, pp. 5966-5979, Dec. 2017.
40. C. Jia and Y. Fu, "Low-rank tensor subspace learning for RGB-D action recognition, " IEEE Trans. Image Process., vol. 25, no. 10, pp. 4641-4652, Oct. 2016.
41. Y. Kong and Y. Fu, "Discriminative relational representation learning for RGB-D action recognition, " IEEE Trans. Image Process., vol. 25, no. 6, pp. 2856-2865, Jun. 2016.
42. P. Scovanner, S. Ali, and M. Shah, "A 3-dimensional sift descriptor and its application to action recognition, " in Proc. 15th ACM Int. Conf. Multimedia, 2007, pp. 357-360.
43. G. Willems, T. Tuytelaars, and L. Van Gool, "An efficient dense and scale-invariant spatio-Temporal interest point detector, " in Proc. Eur. Conf. Comput. Vis., 2008, pp. 650-663.
44. R. Messing, C. Pal, and H. Kautz, "Activity recognition using the velocity histories of tracked keypoints, " in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), Sep. 2009, pp. 104-111.
45. P. Matikainen, M. Hebert, and R. Sukthankar, "Trajectons: Action recognition through the motion analysis of tracked features, " in Proc. IEEE 12th Int. Conf. Comput. Vis. Workshops (ICCV Workshops), Sep. 2009, pp. 514-521.
46. J. Sun, X. Wu, S. Yan, L.-F. Cheong, T.-S. Chua, and J. Li, "Hierarchical spatio-Temporal context modeling for action recognition, " in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2009, pp. 2004-2011.
47. J. Sun, Y. Mu, S. Yan, and L.-F. Cheong, "Activity recognition using dense long-duration trajectories, " in Proc. IEEE Int. Conf. Multimedia Expo (ICME), Jul. 2010, pp. 322-327.
48. F. Liu, X. Xu, S. Qiu, C. Qing, and D. Tao, "Simple to complex transfer learning for action recognition, " IEEE Trans. Image Process., vol. 25, no. 2, pp. 949-960, Feb. 2016.
49. T. Xu, F. Zhu, E. K. Wong, and Y. Fang, "Dual many-To-one-encoderbased transfer learning for cross-dataset human action recognition, " Image Vis. Comput., vol. 55, pp. 127-137, Nov. 2016.
50. M. Zhou, H. Chen, L. Ren, G. Sapiro, L. Carin, and J. W. Paisley, "Non-parametric Bayesian dictionary learning for sparse image representations, " in Proc. Adv. Neural Inf. Process. Syst., 2009, pp. 2295-2303.
51. J. Wright, A. Y. Yang, A. Ganesh, S. S. Sastry, and Y. Ma, "Robust face recognition via sparse representation, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 2, pp. 210-227, Feb. 2009.
52. Y. Wang and G. Mori, "Hidden part models for human action recognition: Probabilistic versus max margin, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 7, pp. 1310-1323, Jul. 2010.
53. M. Aharon, M. Elad, and A. Bruckstein, "K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation, " IEEE Trans. Signal Process., vol. 54, no. 11, pp. 4311-4322, Nov. 2006.
54. Q. Qiu, V. M. Patel, P. Turaga, and R. Chellappa, "Domain adaptive dictionary learning, " in Proc. Eur. Conf. Comput. Vis., 2012, pp. 631-645.
55. J. Zheng, Z. Jiang, and R. Chellappa, "Cross-view action recognition via transferable dictionary learning, " IEEE Trans. Image Process., vol. 25, no. 6, pp. 2542-2556, Jun. 2016.
56. R. Gross and J. Shi, "The CMU motion of body (MoBo) database, " Carnegie Mellon Univ., Pittsburgh, PA, USA, Tech. Rep. CMU-RI-TR-01-18, Jun. 2001.
57. J. Tilmanne, R. Sebbe, and T. Dutoit, "A database for stylistic human gait modeling and synthesis, " in Proc. eNTERFACE Workshop Multimodal Interfaces, Paris, France, 2008, pp. 91-94.
58. J. Mairal, F. Bach, J. Ponce, and G. Sapiro, "Online dictionary learning for sparse coding, " in Proc. 26th Annu. Int. Conf. Mach. Learn., 2009, pp. 689-696.
59. F. Zhu and L. Shao, "Weakly-supervised cross-domain dictionary learning for visual recognition, " Int. J. Comput. Vis., vol. 109, nos. 1-2, pp. 42-59, Aug. 2014.
60. Z. Jiang, Z. Lin, and L. S. Davis, "Learning a discriminative dictionary for sparse coding via label consistent K-SVD, " in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2011, pp. 1697-1704.
61. G. H. Golub, P. C. Hansen, and D. P. OLeary, "Tikhonov regularization and total least squares, " SIAM J. Matrix Anal. Appl., vol. 21, no. 1, pp. 185-194, 1999.
62. R. Li and T. Zickler, "Discriminative virtual views for crossview action recognition, " in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2012, pp. 2855-2862.
63. Z. Zhang, C. Wang, B. Xiao, W. Zhou, S. Liu, and C. Shi, "Crossview action recognition via a continuous virtual path, " in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2013, pp. 2690-2697.
64. B. Li, O. I. Camps, and M. Sznaier, "Cross-view activity recognition using Hankelets, " in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2012, pp. 1362-1369.
65. N. Gkalelis, H. Kim, A. Hilton, N. Nikolaidis, and I. Pitas, "The i3Dpost multi-view and 3D human action/interaction database, " in Proc. IEEE Conf. Vis. Media Prod. (CVMP), Nov. 2009, pp. 159-168.
66. G. Gkioxari and J. Malik, "Finding action tubes, " in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2015, pp. 759-768.
67. J. Donahue et al., "Long-Term recurrent convolutional networks for visual recognition and description, " in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2015, pp. 2625-2634.
68. M. B. Holte, T. B. Moeslund, N. Nikolaidis, and I. Pitas, "3D human action recognition for multi-view camera systems, " in Proc. Int. Conf. 3D Imag., Modeling, Process., Vis. Transmiss. (3DIMPVT), 2011, pp. 342-349.
69. A. Iosifidis, N. Nikolaidis, and I. Pitas, "Movement recognition exploiting multi-view information, " in Proc. IEEE Int. Workshop Multimedia Signal Process. (MMSP), Oct. 2010, pp. 427-431.