Spatiotemporal localization and categorization of human actions in unsegmented image sequences

IEEE Transactions on Image Processing

Volumn 20, Issue 4, 2011, Pages 1126-1140

  Oikonomopoulos, Antonios ^a   Patras, Ioannis ^b   Pantic, Maja ^a,c  

^a IMPERIAL COLLEGE LONDON   (United Kingdom)

^b QUEEN MARY UNIVERSITY OF LONDON   (United Kingdom)

^c UNIVERSITY OF TWENTE   (Netherlands)

Author keywords

Action detection; space time voting

Indexed keywords

ACTION DETECTION; CODEBOOKS; CODEWORD; DATA SETS; FEATURE DESCRIPTORS; HUMAN ACTIONS; HUMAN ACTIVITIES; IMAGE SEQUENCE; IMPLICIT REPRESENTATION; MEAN SHIFT; MODE ESTIMATION; REFERENCE POINTS; SPATIAL POSITIONS; TEMPORAL POSITION; TIME VOTING; VOTING SCHEMES;

CLASSIFICATION (OF INFORMATION);

CLUTTER (INFORMATION THEORY);

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; AUTOMATED PATTERN RECOGNITION; COMPUTER ASSISTED DIAGNOSIS; HUMAN; IMAGE ENHANCEMENT; IMAGE SUBTRACTION; METHODOLOGY; MOVEMENT (PHYSIOLOGY); PHOTOGRAPHY; PHYSIOLOGY; REPRODUCIBILITY; SENSITIVITY AND SPECIFICITY; VIDEORECORDING;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; HUMANS; IMAGE ENHANCEMENT; IMAGE INTERPRETATION, COMPUTER-ASSISTED; MOVEMENT; PATTERN RECOGNITION, AUTOMATED; PHOTOGRAPHY; REPRODUCIBILITY OF RESULTS; SENSITIVITY AND SPECIFICITY; SUBTRACTION TECHNIQUE; VIDEO RECORDING;

EID: 79952952363     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2010.2076821     Document Type: Article

References (46)

1. R. Poppe, "A survey on vision-based human action recognition," Image Vis. Comput. J., vol. 28, pp. 976-990, 2010.
2. M. Pantic, A. Pentland, A. Nijholt, and T. Huang, "Human computing and machine understanding of human behavior: A survey," Lecture Notes in Artificial Intelligence, vol. 4451, pp. 47-71, 2007.
3. K. Mikolajczyk and C. Schmid, "A performance evaluation of local descriptors," IEEE Trans. Pattern Anal. Machine Intell., vol. 27, pp. 1615-1630, 2005. (Pubitemid 41555406)
4. I. Laptev and T. Lindeberg, "Space-time interest points," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2003, pp. 432-439.
5. D. Han, L. Bo, and C. Sminchisescu, "Selection and context for action recognition," in Proc. IEEE Int. Conf. Comput. Vis., 2009, pp. 1933-1940.
6. P. Dollar, V. Rabaud, G. Cottrell, and S. Belongie, "Behavior recognition via sparse spatio-temporal features," in VS-PETS, 2005, pp. 65-72. (Pubitemid 46178025)
7. M. Bregonzio, S. Gong, and T. Xiang, "Recognising action as clouds of space-time interest points," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2009, pp. 1948-1955.
8. A. Oikonomopoulos, I. Patras, and M. Pantic, "Spatiotemporal salient points for visual recognition of human actions," IEEE Trans. Syst., Man Cybern. B, vol. 36, pp. 710-719, 2006. (Pubitemid 43811326)
9. T. Kadir and M. Brady, "Scale saliency: A novel approach to salient feature and scale selection," in Int. Conf. Vis. Inf. Eng., 2000, pp. 25-28.
10. D. Lowe, "Distinctive image features from scale-invariant keypoints," Int. J. Comput. Vis., vol. 60, pp. 91-110, 2004.
11. A. Agarwal and B. Triggs, "Hyperfeatures multilevel local coding for visual recognition," in Eur. Conf. Comput. Vis., 2006, vol. 1, pp. 30-43. (Pubitemid 44029766)
12. l. J. Li and L. Fei-Fei, "What, where and who? classifying events by scene and object recognition," in Proc. ICCV, 2007, pp. 1-8.
13. J. Sivic and A. Zisserman, "Video Google: Efficient visual search of videos," LNCS, vol. 4170, pp. 127-144, 2006.
14. H. Bay, T. Tuytelaars, and L. V. Gool, "Surf: Speeded up robust features," Comput. Vis. Image Understand., vol. 110, pp. 346-359, 2008.
15. H. Jhuang, T. Serre, L. Wolf, and T. Poggio, "A biologically inspired system for action recognition," in Proc. IEEE Int. Conf. Comput. Vis., 2007, pp. 1-8.
16. K. Schindler and L. V. Gool, "Action snippets: How many frames does human action require?," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2008, pp. 1-8.
17. S. Ali and M. Shah, "Human action recognition in videos using kinematic features and multiple instance learning," IEEE Trans. Pattern Anal. Machine Intell., vol. 32, no. 2, pp. 288-303, 2010.
18. I. Laptev, M. Marszalek, C. Schmid, and B. Rozenfeld, "Learning realistic human actions from movies," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2008, pp. 1-8.
19. M. Marszalek, I. Laptev, and C. Schmid, "Actions in context," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2009, pp. 2929-2936.
20. J. C. Niebles, H. Wang, and L. Fei-Fei, "Unsupervised learning of human action categories using spatial-temporalwords," in Proc. British Machine Vis. Conf., Edinburgh, U.K., 2006, vol. 1, Online.
21. T. Hofmann, "Probabilistic latent semantic indexing," in Proc. SIGIR, 1999, pp. 50-57.
22. H. Ning, Y. Hu, and T. Huang, "Searching human behaviors using spatial- temporal words," in Proc. IEEE Int. Conf. Image Process., 2007, vol. 6, pp. 337-340.
23. A. Oikonomopoulos, M. Pantic, and I. Patras, "Sparse b-spline polynomial descriptors for human activity recognition," Image Vis. Comput. J., vol. 27, pp. 1814-1825, 2009.
24. N. Ikizler and P. Duygulu, "Human action recognition using distribution of oriented rectangular patches," Lecture Notes in Computer Science, vol. 4814, pp. 271-284, 2007.
25. B. Leibe, A. Leonardis, and B. Schiele, "Combined object categorization and segmentation with an implicit shape model," in Proc. ECCV'04 Workshop on Statistical Learning in Computer Vision, 2004, pp. 17-32.
26. A. Opelt, A. P. , and A. Zisserman, "A boundary-fragment-model for object detection," in Proc. Eur. Conf. Comput. Vis., 2006, pp. 575-588. (Pubitemid 44029756)
27. K. Mikolajczyk and H. Uemura, "Action recognition with motion-appearance vocabulary forest," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2008, pp. 1-8.
28. J. Sivic, B. Russell, A. Efros, A. Zisserman, and W. Freeman, "Discovering objects and their location in images," in Proc. IEEE Int. Conf. Comput. Vis., 2005, vol. 1, pp. 370-377. (Pubitemid 44054982)
29. O. Boiman and M. Irani, "Detecting irregularities in images and in video," in Proc. IEEE Int. Conf. Comput. Vis., 2005, vol. 1, pp. 462-469. (Pubitemid 44054993)
30. J. Niebles and L. Fei-Fei, "A hierarchical model of shape and appearance for human action classification," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2007, pp. 1-8.
31. E. Shechtman and M. Irani, "Matching local self-similarities across images and videos," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2007, pp. 1-8.
32. I. Junejo, E. Dexter, I. Laptev, and P. Prez, "Cross-view action recognition from temporal self-similarities," in Eur. Conf. Comput. Vis., 2008, vol. 2, pp. 293-306.
33. A. Gilbert, J. Illingworth, and R. Bowden, "Fast realistic multi-action recognition using mined dense spatio-temporal features," in Proc. IEEE Int. Conf. Comput. Vis., 2009, pp. 925-931.
34. Y. Cheng, "Mean shift, mode seeking, and clustering," IEEE Trans. Pattern Anal. Machine Intell., vol. 17, pp. 790-799, 1995.
35. M. Tipping, "The relevance vector machine," Advances in Neural Information Processing Systems, pp. 652-658, 1999.
36. A. Oikonomopoulos, I. Patras, and M. Pantic, "An implicit spatiotemporal shape model for human activity localization and recognition," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit. (CVPR09),Workshop on Human Communicative Behavior Analysis, 2009, vol. 3, pp. 27-33.
37. C. Schuldt, I. Laptev, and B. Caputo, "Recognizing human actions: A local svm approach," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2004, vol. 3, pp. 32-36.
38. M. Blank, L. Gorelick, E. Shechtman, M. Irani, and R. Basri, "Actions as space-time shapes," in Proc. IEEE Int. Conf. Comput. Vis., 2005, vol. 2, pp. 1395-1402. (Pubitemid 44042342)
39. M. Black and P. Anandan, "A framework for the robust estimation of optical flow," in Proc. ICCV, 1993, pp. 231-236. (Pubitemid 23667848)
40. J. Friedman, T. Hastie, and R. Tibshirani, "Additive Logistic Regression: A Statistical View of Boosting," Stanford Univ., Stanford, CA, 1993, Tech. Report.
41. Y. B. Shalom and T. Fortmann, Tracking and Data Association. New York: Academic Press, 1988.
42. Y. Ke,M. Ans, and R. S. Hebert, "Efficient visual event detection using volumetric features," in Proc. IEEE Int. Conf. Comput. Vis., 2005, vol. 1, pp. 166-173. (Pubitemid 44054955)
43. K. Rapantzikos, Y.Avrithis, and S.Kollias, "Dense saliency-based spatiotemporal feature points for action recognition,," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2009, pp. 1-8.
44. M. Ahmad and S. Lee, "Human action recognition using shape and clg-motion flow from multi-view image sequences," Pattern Recognit., vol. 41, pp. 2237-2252, 2008.
45. A. Fathi and G. Mori, "Action recognition by learning midlevel motion features,," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2008, pp. 1-8.
46. S. Wong and R. Cipolla, "Extracting spatiotemporal interest points using global information,," in Proc. IEEE Int. Conf. Comput. Vis., 2007, pp. 1-8.