Hankelet-based dynamical systems modeling for 3D action recognition

Image and Vision Computing

Volumn 44, Issue , 2015, Pages 29-43

  Lo Presti, Liliana ^a   La Cascia, Marco ^a   Sclaroff, Stan ^b   Camps, Octavia ^c  

^a UNIVERSITY OF PALERMO   (Italy)

^b BOSTON UNIVERSITY   (United States)

^c Northeastern University   (United States)

Author keywords

Action; Discriminative learning; Hankel Matrix; Hidden Markov Model; Linear time invariant system

Indexed keywords

ATOMS; CLASSIFICATION (OF INFORMATION); DYNAMICAL SYSTEMS; HIDDEN MARKOV MODELS; INVARIANCE; LINEAR SYSTEMS; MARKOV PROCESSES; TIME VARYING CONTROL SYSTEMS;

ACTION; ACTION REPRESENTATIONS; DISCRIMINATIVE APPROACH; DISCRIMINATIVE LEARNING; DYNAMICAL SYSTEMS MODEL; HANKEL MATRIX; LINEAR TIME INVARIANT SYSTEMS; LINEAR TIME-INVARIANT SYSTEM;

MATRIX ALGEBRA;

EID: 84946152186     PISSN: 02628856     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.imavis.2015.09.007     Document Type: Article

References (86)

1. S. Kwak, B. Han, and J. Han Scenario-based video event recognition by constraint flow Proc. of Conference on Computer Vision and Pattern Recognition (CVPR) 2011 IEEE 3345 3352 10.1109/CVPR.2011.5995435
2. U. Gaur, Y. Zhu, B. Song, and A. Roy-Chowdhury A "string of feature graphs" model for recognition of complex activities in natural videos Proc. of International Conference on Computer Vision (ICCV) 2011 IEEE 2595 2602 10.1109/ICCV.2011.6126548
3. S. Park, and J. Aggarwal Recognition of two-person interactions using a hierarchical Bayesian network First ACM SIGMM international workshop on Video surveillance 2003 ACM 65 76 10.1145/982452.982461
4. I. Junejo, E. Dexter, I. Laptev, and P. Pérez View-independent action recognition from temporal self-similarities IEEE Trans. Pattern Anal. Mach. Intell. (PAMI) 33 1 2011 172 185 10.1109/TPAMI.2010.68
5. Z. Duric, W. Gray, R. Heishman, F. Li, A. Rosenfeld, M. Schoelles, C. Schunn, and H. Wechsler Integrating perceptual and cognitive modeling for adaptive and intelligent human-computer interaction Proc. IEEE 90 2002 1272 1289 10.1109/JPROC.2002.801449
6. Y.-J. Chang, S.-F. Chen, and J.-D. Huang A Kinect-based system for physical rehabilitation: a pilot study for young adults with motor disabilities Res. Dev. Disabil. 32 6 2011 2566 2570 10.1016/j.ridd.2011.07.002
7. A. Thangali, J.P. Nash, S. Sclaroff, and C. Neidle Exploiting phonological constraints for handshape inference in ASL video Proc. of Conference on Computer Vision and Pattern Recognition (CVPR) 2011 IEEE 521 528 10.1109/CVPR.2011.5995718
8. A. Thangali Varadaraju Exploiting phonological constraints for handshape recognition in sign language video Ph.D. thesis 2013 Boston University MA, USA
9. H. Cooper, and R. Bowden Large lexicon detection of sign language Proc. of International Conference on Human-Computer Interaction Lecture Notes in Computer Science 2007 Springer Berlin Heidelberg 88 97 10.1007/978-3-540-75773-3-10
10. J.M. Rehg, G.D. Abowd, A. Rozga, M. Romero, M.A. Clements, S. Sclaroff, I. Essa, O.Y. Ousley, Y. Li, C. Kim, and et al. Decoding children's social behavior Proc. of Conference on Computer Vision and Pattern Recognition (CVPR) 2013 IEEE 3414 3421 10.1109/CVPR.2013.438
11. L. Lo Presti, S. Sclaroff, and A. Rozga Joint alignment and modeling of correlated behavior streams Proc. of International Conference on Computer Vision-Workshops (ICCVW) 2013 730 737 10.1109/ICCVW.2013.100
12. H. Moon, R. Sharma, N. Jung, Method and system for measuring shopper response to products based on behavior and facial expression, uS Patent 8,219,438 (Jul. 10 2012). URL http://www.google.com/patents/US8219438
13. J. Shotton, T. Sharp, A. Kipman, A. Fitzgibbon, M. Finocchio, A. Blake, M. Cook, and R. Moore Real-time human pose recognition in parts from single depth images Commun. ACM 56 1 2013 116 124 10.1145/2398356.2398381
14. L. Xia, C.-C. Chen, and J. Aggarwal View invariant human action recognition using histograms of 3D joints Proc. of Computer Vision and Pattern Recognition Workshops (CVPRW) 2012 IEEE 20 27 10.1109/CVPRW.2012.6239233
15. X. Yang, and Y. Tian EigenJoints-based action recognition using Naive-Bayes-Nearest-Neighbor Proc. of Computer Vision and Pattern Recognition Workshops (CVPRW) 2012 IEEE 14 19 10.1109/CVPRW.2012.6239232
16. O. Oreifej, Z. Liu, and W. Redmond HON4D: histogram of oriented 4D normals for activity recognition from depth sequences Proc. of Conference on Computer Vision and Pattern Recognition (CVPR) 2013 716 723 10.1109/CVPR.2013.98
17. G. Evangelidis, G. Singh, R. Horaud, and et al. Skeletal quads: human action recognition using joint quadruples Proc. of International Conference on Pattern Recognition (ICPR) 2014 IEEE 4513 4518 10.1109/ICPR.2014.772
18. R. Slama, H. Wannous, M. Daoudi, and A. Srivastava Accurate 3D action recognition using learning on the Grassmann manifold Pattern Recogn. (PR) 48 2 2015 556 567 10.1016/j.patcog.2014.08.011
19. J. Wang, Z. Liu, J. Chorowski, Z. Chen, and Y. Wu Robust 3D action recognition with random occupancy patterns Proc. of European Conference on Computer Vision (ECCV) 2012 Springer 872 885 10.1007/978-3-642-33709-3-62
20. J. Wang, Z. Liu, Y. Wu, and J. Yuan Mining actionlet ensemble for action recognition with depth cameras Proc. of Conference on Computer Vision and Pattern Recognition (CVPR) 2012 IEEE 1290 1297 10.1109/CVPR.2012.6247813
21. N. Ikizler, and D. Forsyth Searching video for complex activities with finite state models Proc. of Conference on Computer Vision and Pattern Recognition (CVPR) 2007 IEEE 1 8 10.1109/CVPR.2007.383168
22. R. Li, T.-P. Tian, S. Sclaroff, and M.-H. Yang 3D human motion tracking with a coordinated mixture of factor analyzers Int. J. Comput. Vis. (IJCV) 87 1-2 2010 170 190 10.1007/s11263-009-0283-4
23. R. Li, T.-P. Tian, and S. Sclaroff Simultaneous learning of nonlinear manifold and dynamical models for high-dimensional time series Proc. of International Conference on Computer Vision (ICCV) 2007 IEEE 1 8 10.1109/ICCV.2007.4409044
24. B. Li, O.I. Camps, and M. Sznaier Cross-view activity recognition using Hankelets Proc. of Conference on Computer Vision and Pattern Recognition (CVPR) 2012 IEEE 1362 1369 10.1109/CVPR.2012.6247822
25. B. Li, M. Ayazoglu, T. Mao, O.I. Camps, and M. Sznaier Activity recognition using dynamic subspace angles Proc. of Conference on Computer Vision and Pattern Recognition (CVPR) 2011 IEEE 3193 3200 10.1109/CVPR.2011.5995672
26. L. Lo Presti, M. La Cascia, S. Sclaroff, and O. Camps Gesture modeling by Hanklet-based hidden Markov model D. Cremers, I. Reid, H. Saito, M.-H. Yang, Computer Vision - ACCV 2014 Vol. 9005 of Lecture Notes in Computer Science 2015 Springer International Publishing 529 546 10.1007/978-3-319-16811-1-35
27. T.B. Moeslund, and E. Granum A survey of computer vision-based human motion capture Comp. Vision Image Underst. (CVIU) 81 3 2001 231 268 10.1006/cviu.2000.0897
28. S. Mitra, and T. Acharya Gesture recognition: a survey IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 37 3 2007 311 324 10.1109/TSMCC.2007.893280
29. R. Poppe A survey on vision-based human action recognition Image Vis. Comput. (IMAVIS) 28 6 2010 976 990 10.1016/j.imavis.2009.11.014
30. D. Weinland, R. Ronfard, and E. Boyer A survey of vision-based methods for action representation, segmentation and recognition Comp. Vision Image Underst. (CVIU) 115 2 2011 224 241 10.1016/j.cviu.2010.10.002
31. J. Klamka Controllability of dynamical systems. A survey Bull. Pol. Acad. Sci. Tech. Sci. 61 2 2013 335 342 10.2478/bpasts-2013-0031
32. N. Dalal, and B. Triggs Histograms of oriented gradients for human detection Proc. of Conference on Computer Vision and Pattern Recognition (CVPR) vol. 1 2005 IEEE 886 893 10.1109/CVPR.2005.177
33. P. Scovanner, S. Ali, and M. Shah A 3-dimensional SIFT descriptor and its application to action recognition Proc. of Conference on Multimedia (MM) 2007 ACM 357 360 10.1145/1291233.1291311
34. V. Kellokumpu, G. Zhao, and M. Pietikäinen Human activity recognition using a dynamic texture based method Proc. of British Machine Vision Conference (BMVC) vol. 1 2008 BMVA Press 2 (doi:10.1.1.165.2894)
35. J.C. Niebles, H. Wang, and L. Fei-Fei Unsupervised learning of human action categories using spatial-temporal words Int. J. Comput. Vis. (IJCV) 79 3 2008 299 318 10.1007/s11263-007-0122-4
36. I. Laptev On space-time interest points Int. J. Comput. Vis. (IJCV) 64 2 2005 107 123 10.1007/s11263-005-1838-7
37. H. Wang, A. Kläser, C. Schmid, and C.-L. Liu Dense trajectories and motion boundary descriptors for action recognition Int. J. Comput. Vis. (IJCV) 103 1 2013 60 79 10.1007/s11263-012-0594-8
38. N. Dalal, B. Triggs, and C. Schmid Human detection using oriented histograms of flow and appearance Proc. of European Conference on Computer Vision (ECCV) 2006 Springer 428 441 10.1007/11744047-33
39. W. Li, Z. Zhang, and Z. Liu Action recognition based on a bag of 3D points Proc. of Computer Vision and Pattern Recognition Workshops (CVPRW) 2010 IEEE 9 14 10.1109/CVPRW.2010.5543273
40. A.W. Vieira, E.R. Nascimento, G.L. Oliveira, Z. Liu, and M.F. Campos STOP: space-time occupancy patterns for 3D action recognition from depth map sequences Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications 2012 252 259 10.1007/978-3-642-33275-3-31
41. A. Yao, J. Gall, G. Fanelli, and L.J. Van Gool Does human action recognition benefit from pose estimation? Proc. of the British Machine Vision Conference (BMVC) vol. 3 2011 BMVA Press 67.1 67.11 10.5244/C.25.67
42. M. Devanne, H. Wannous, S. Berretti, P. Pala, M. Daoudi, and A. Del Bimbo Space-time pose representation for 3D human action recognition Proc. of the International Conference on Image Analysis and Processing (ICIAP) 2013 Springer 456 464 10.1007/978-3-642-41190-8-49
43. J. Sung, C. Ponce, B. Selman, and A. Saxena Unstructured human activity detection from RGBD images Proc. of International Conference on Robotics and Automation (ICRA) 2012 IEEE 842 849 10.1109/ICRA.2012.6224591
44. Y. Zhu, W. Chen, and G. Guo Fusing spatiotemporal features and joints for 3D action recognition Proc. of Computer Vision and Pattern Recognition Workshops (CVPRW) 2013 IEEE 486 491 10.1109/CVPRW.2013.78
45. B. Bamieh, and L. Giarre Identification of linear parameter varying models Int. J. Robust Nonlinear Control 12 9 2002 841 853 10.1002/rnc.706
46. S. Paoletti, A.L. Juloski, G. Ferrari-Trecate, and R. Vidal Identification of hybrid systems: a tutorial Eur. J. Control. 13 2 2007 242 260 10.3166/ejc.13.242-260
47. E.D. Sontag Nonlinear regulation: the piecewise linear approach IEEE Trans. Autom. Control 26 2 1981 346 358 10.1109/TAC.1981.1102596
48. V. Gupta, R.M. Murray, L. Shi, and B. Sinopoli Networked sensing, estimation and control systems California Institute of Technology Report 2009
49. G. Doretto, A. Chiuso, Y.N. Wu, and S. Soatto Dynamic textures Int. J. Comput. Vis. (IJCV) 51 2 2003 91 109 10.1023/A:1021669406132
50. C. Dicle, O.I. Camps, and M. Sznaier The way they move: tracking multiple targets with similar appearance Proc. of International Conference on Computer Vision (ICCV) 2013 IEEE 2304 2311 10.1109/ICCV.2013.286
51. A.C. Sankaranarayanan, P.K. Turaga, R.G. Baraniuk, and R. Chellappa Compressive acquisition of dynamic scenes Proc. of European Conference on Computer Vision (ECCV) 2010 Springer 129 142 10.1007/978-3-642-15549-9-10
52. L. Lo Presti, and M. La Cascia Using Hankel matrices for dynamics-based facial emotion recognition and pain detection Proc. of Computer Vision and Pattern Recognition Workshops (AMFG-CVPRW) 2015 IEEE 36-33 (URL http://www.cv-foundation.org/openaccess/content-cvpr-workshops-2015/W08/papers/Presti-Using-Hankel-Matrices-2015-CVPR-paper.pdf)
53. L. Lo Presti, and M. La Cascia Ensemble of Hankel Matrices for Face Emotion Recognition, Proc. of International Conference on Image Analysis and Processing (ICIAP) Lecture Notes in Computer Science 9280 2015 Springer International Publishing 586 597 10.1007/978-3-319-23234-8-54
54. F. Cuzzolin, and M. Sapienza Learning pullback HMM distances IEEE Trans. Pattern Anal. Mach. Intell. (PAMI) 36 7 2014 1483 1489 10.1109/TPAMI.2013.181
55. H. Jiang Discriminative training of HMMs for automatic speech recognition: a survey Comput. Speech Lang. 24 4 2010 589 608 10.1016/j.csl.2009.08.002
56. T. Jebara, and A. Pentland Maximum conditional likelihood via bound maximization and the cem algorithm Advances in Neural Information Processing Systems (NIPS) vol. 1 1998 The MIT Press 494 500
57. L. Bahl, P. Brown, P. De Souza, and R. Mercer Maximum mutual information estimation of hidden markov model parameters for speech recognition Proc. of International Conference on Acoustics, Speech, and Signal Processing (ICASSP) 1986 IEEE 49 52 10.1109/ICASSP.1986.1169179
58. R. Meir Empirical risk minimization versus maximum-likelihood estimation: a case study Neural Comput. 7 1 1995 144 157 10.1162/neco.1995.7.1.144
59. A.J. Smola Advances in large margin classifiers 2000 The MIT Press
60. J. Lafferty, A. McCallum, and F.C. Pereira Conditional random fields: probabilistic models for segmenting and labeling sequence data Proc. of International Conference on Machine Learning (ICML) 2001 Morgan Kaufmann Publishers Inc. 282 289
61. A. Quattoni, S. Wang, L.-P. Morency, M. Collins, and T. Darrell Hidden conditional random fields IEEE Trans. Pattern Anal. Mach. Intell. (PAMI) 29 10 2007 1848 1852 10.1109/TPAMI.2007.1124
62. P. Felzenszwalb, D. McAllester, and D. Ramanan A discriminatively trained, multiscale, deformable part model Proc. of Conference on Computer Vision and Pattern Recognition (CVPR) 2008 IEEE 1 8 10.1109/CVPR.2008.4587597
63. I. Tsochantaridis, T. Joachims, T. Hofmann, and Y. Altun Large margin methods for structured and interdependent output variables J. Mach. Learn. Res. (JMLR) 6 2005 1453 1484
64. Y. Wang, and G. Mori Max-margin Hidden Conditional Random Fields for human action recognition Proc. of Conference on Computer Vision and Pattern Recognition (CVPR) 2009 IEEE 872 879 10.1109/CVPR.2009.5206709
65. D. Weinland, E. Boyer, and R. Ronfard Action recognition from arbitrary views using 3D exemplars Proc. of International Conference on Computer Vision (ICCV) 2007 IEEE 1 7 10.1109/ICCV.2007.4408849
66. F. Martinez-Contreras, C. Orrite-Urunuela, E. Herrero-Jaraba, H. Ragheb, and S.A. Velastin Recognizing human actions using silhouette-based HMM Proc. of International Conference on Advanced Video and Signal Based Surveillance (AVSS) 2009 IEEE 43 48 10.1109/AVSS.2009.46
67. F. Lv, and R. Nevatia Recognition and segmentation of 3-D human action using HMM and multi-class AdaBoost Proc. of European Conference on Computer Vision (ECCV) 2006 Springer 359 372 10.1007/11744085-28
68. T. Lan, Y. Wang, W. Yang, and G. Mori Beyond actions: discriminative models for contextual group activities Advances in Neural Information Processing Systems (NIPS) vol. 4321 2010 The MIT Press 4322 4325
69. A.D. Wilson, and A.F. Bobick Parametric Hidden Markov Models for gesture recognition IEEE Trans. Pattern Anal. Mach. Intell. (PAMI) 21 9 1999 884 900 10.1109/34.790429
70. L.E. Baum, T. Petrie, G. Soules, and N. Weiss A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains Ann. Math. Stat. 41 1 1970 164 171 10.1214/aoms/1177697196
71. F. Sha, and L.K. Saul Large margin Hidden Markov Models for automatic speech recognition Advances in Neural Information Processing Systems (NIPS) vol. 19 2007 The MIT Press 1249
72. M. Collins Discriminative training methods for hidden Markov models: theory and experiments with perceptron algorithms Proc. of Conference on Empirical Methods in Natural Language Processing (EMNLP) vol. 10 2002 Association for Computational Linguistics 1 8
73. B. Taskar, C. Guestrin, and D. Koller Max-margin Markov networks Advances in Neural Information Processing Systems (NIPS) vol. 16 2004 25
74. Y. Altun, I. Tsochantaridis, T. Hofmann, and et al. Hidden Markov support vector machines Proc. of International Conference on Machine Learning (ICML) vol. 3 2003 3 10
75. L. Rabiner A tutorial on Hidden Markov Models and selected applications in speech recognition Proc. IEEE 77 2 1989 257 286 10.1109/5.18626
76. J. Nocedal, and S.J. Wright Numerical optimization 2nd edition 2006 Springer
77. S.Z. Masood, C. Ellis, M.F. Tappen, J.J. LaViola, and R. Sukthankar Exploring the trade-off between accuracy and observational latency in action recognition Int. J. Comput. Vis. (IJCV) 101 3 2013 420 436 10.1007/s11263-012-0550-7
78. F. Ofli, R. Chaudhry, G. Kurillo, R. Vidal, and R. Bajcsy Sequence of the most informative joints (SMIJ): a new representation for human skeletal action recognition J. Vis. Commun. Image Represent. 25 1 2014 24 38 10.1016/j.jvcir.2013.04.007
79. J. Martens, and I. Sutskever Learning recurrent neural networks with Hessian-free optimization Proc. of International Conference on Machine Learning (ICML) 2011 1033 1040
80. R. Vemulapalli, F. Arrate, and R. Chellappa Human action recognition by representing 3D skeletons as points in a Lie Group Proc. of Conference on Computer Vision and Pattern Recognition (CVPR) 2014 IEEE 588 595 10.1109/CVPR.2014.82
81. C. Wang, Y. Wang, and A.L. Yuille An approach to pose-based action recognition Proc. of Conference on Computer Vision and Pattern Recognition (CVPR) 2013 IEEE 915 922 10.1109/CVPR.2013.123
82. X. Yang, C. Zhang, and Y. Tian Recognizing actions using depth motion maps-based histograms of oriented gradients Proc. of International Conference on Multimedia (MM) 2012 ACM 1057 1060 10.1145/2393347.2396382
83. E. Ohn-Bar, and M.M. Trivedi Joint angles similarities and HOG2 for action recognition Proc. of Computer Vision and Pattern Recognition Workshops (CVPRW) 2013 IEEE 465 470 10.1109/CVPRW.2013.76
84. N. Raman, and S.J. Maybank Action classification using a discriminative multilevel HDP-HMM Neurocomputing 2015 10.1016/j.neucom.2014.12.009 (in press)
85. M.E. Hussein, M. Torki, M.A. Gowayyed, and M. El-Saban Human action recognition using a temporal hierarchy of covariance descriptors on 3D joint locations Proc. of International Joint Conference on Artificial Intelligence (IJCAI) 2013 AAAI Press 2466 2472
86. J.R. Padilla-López, A.A. Chaaraoui, and F. Flórez-Revuelta A discussion on the validation tests employed to compare human action recognition methods using the MSR Action3D dataset, CoRR abs/1407.7390 2015 URL http://arxiv.org/abs/1407.7390