Realistic human action recognition with multimodal feature selection and fusion

IEEE Transactions on Systems, Man, and Cybernetics Part A:Systems and Humans

Volumn 43, Issue 4, 2013, Pages 875-885

  Wu, Qiuxia ^a,b,c   Wang, Zhiyong ^b   Deng, Feiqi ^a   Chi, Zheru ^d   Feng, David Dagan ^b,d  

^a SOUTH CHINA UNIVERSITY OF TECHNOLOGY   (China)

^b UNIVERSITY OF SYDNEY   (Australia)

^c Guangzhou Institute of Modern Industrial Technology   (China)

^d HONG KONG POLYTECHNIC UNIVERSITY   (Hong Kong)

Author keywords

Fuzzy integral; Multimodal fusion; Multiple kernel learning (MKL); Realistic human action recognition

Indexed keywords

ACTION RECOGNITION; DYNAMIC BACKGROUND; FUZZY INTEGRAL; HUMAN-ACTION RECOGNITION; MULTI-MODAL FUSION; MULTIMODAL FEATURES; MULTIPLE KERNEL LEARNING; REALISTIC SCENARIO; CONTROLLED CONDITIONS;

GESTURE RECOGNITION; IMAGE RECOGNITION; INTEGRAL EQUATIONS; SUPPORT VECTOR MACHINES; TELEPHONE SETS; VECTOR SPACES;

MOTION ESTIMATION; LEARNING ALGORITHMS;

EID: 84887050628     PISSN: 10834427     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSMCA.2012.2226575     Document Type: Article

References (51)

1. P. Turaga, R. Chellappa, V. S. Subrahmanian, and O. Udrea, Machine recognition of human activities: A survey, IEEE Trans. Circuits Syst. Video Technol., vol. 18, no. 11, pp. 1473-1488, Nov. 2008.
2. R. Poppe, A survey on vision-based human action recognition, Image Vis. Comput., vol. 28, no. 6, pp. 976-990, Jun. 2010.
3. J. Aggarwal and M. Ryoo, Human activity analysis: A review, ACM Comput. Surv., vol. 43, no. 3, p. 16, Apr. 2011.
4. C. Schüldt, I. Laptev, and B. Caputo, Recognizing human actions: A local SVM approach, in Proc. IEEE ICPR, 2004, vol. 3, pp. 32-36.
5. M. Blank, L. Gorelick, E. Shechtman, M. Irani, and R. Basri, Actions as space-time shapes, in Proc. IEEE ICCV, 2005, vol. 2, pp. 1395-1402. (Pubitemid 44042342)
6. I. Laptev, M. Marszalek, C. Schmid, and B. Rozenfeld, Learning realistic human actions from movies, in Proc. IEEE CVPR, 2008, pp. 1-8.
7. K. Rapantzikos, Y. Avrithis, and S. Kollias, Dense saliency-based spatiotemporal feature points for action recognition, in Proc. IEEE CVPR, 2009, pp. 1454-1461.
8. J. Sun, X. Wu, S. Yan, L.-F. Cheong, T.-S. Chua, and J. Li, Hierarchical spatio-temporal context modeling for action recognition, in Proc. IEEE CVPR, 2009, pp. 2004-2011.
9. J. Liu, J. Luo, and M. Shah, Recognizing realistic actions from videos ?in the wild?, in Proc. IEEE CVPR, 2009, pp. 1996-2003.
10. G. Zhu, M. Yang, K. Yu, W. Xu, and Y. Gong, Detecting video events based on action recognition in complex scenes using spatio-temporal descriptor, in Proc. ACM Int. Conf. Multimedia, Oct. 2009, pp. 165-174.
11. D. Han, L. Bo, and C. Sminchisescu, Selection and context for action recognition, in Proc. IEEE ICCV, Oct. 2009, pp. 1933-1940.
12. M. Marszalek, I. Laptev, and C. Schmid, Actions in context, in Proc. IEEE CVPR, 2009, pp. 2929-2936.
13. R. Arrighi, F. Marini, and D. Burr, Meaningful auditory information enhances perception of visual biological motion, J. Vis., vol. 9, no. 4, pp. 25-1-25-7, Apr. 2009.
14. H.-G. Kim, J. Jeong, J.-H. Kim, and J. Kim, Real-time highlight detection in baseball video for TVs with time-shift function, IEEE Trans. Consum. Electron., vol. 54, no. 2, pp. 831-838, May 2008.
15. Y. Gong, M. Han, W. Hua, and W. Xu, Maximum entropy modelbased baseball highlight detection and classification, Comput. Vis. Image Understand., vol. 96, no. 2, pp. 181-199, Nov. 2004.
16. R. Leonardi, P. Migliorati, and M. Prandini, Semantic indexing of soccer audio-visual sequences: A multimodal approach based on controlled markov chains, IEEE Trans. Circuits Syst. Video Technol., vol. 14, no. 5, pp. 634-643, May 2004.
17. M. Cristani, M. Bicego, and V. Murino, Audio-visual event recognition in surveillance video sequences, IEEE Trans. Multimedia, vol. 9, no. 2, pp. 257-267, Feb. 2007.
18. C. Liu, Q. Huang, S. Jiang, L. Xing, Q. Ye, andW. Gao, A framework for flexible summarization of racquet sports video using multiple modalities, Comput. Vis. Image Understand., vol. 113, no. 3, pp. 415-424, Mar. 2009.
19. X. Zhu, X.Wu, A. Elmagarmid, Z. Feng, and L.Wu, Video data mining: Semantic indexing and event detection from the association perspective, IEEE Trans. Knowl. Data Eng., vol. 17, no. 5, pp. 665-677, May 2005.
20. Y. Li, S. Narayanan, and C.-C. J. Kuo, Content-based movie analysis and indexing based on audiovisual cues, IEEE Trans. Circuits Syst. Video Technol., vol. 14, no. 8, pp. 1073-1085, Aug. 2004.
21. Q. Wu, Z. Wang, F. Deng, and D. D. Feng, Realistic human action recognition with audio context, in Proc. Int. Conf. DICTA, Dec. 2010, pp. 288-293.
22. H. Liu and L. Yu, Toward integrating feature selection algorithms for classification and clustering, IEEE Trans. Knowl. Data Eng., vol. 17, no. 4, pp. 491-502, Apr. 2005.
23. G. Muhammad, Y. A. Alotaibi, M. Alsulaiman, and M. N. Huda, Environment recognition using selected MPEG-7 audio features and mel-frequency cepstral coefficients, in Proc. IEEE ICDT, Jun. 2010, pp. 11-16.
24. Y. Gu, S. Tan, K. Wong, M. Ho, and L. Qu, Using GA-based feature selection for emotion recognition from physiological signals, in Proc. ISPACS, Feb. 2009, pp. 1-4.
25. M. Varma and B. R. Babu, More generality in efficient multiple kernel learning, in Proc. ICML, 2009, pp. 1065-1072.
26. A. Temko, D. Macho, and C. Nadeu, Fuzzy integral based information fusion for classification of highly confusable non-speech sounds, Pattern Recognit., vol. 41, no. 5, pp. 1814-1823, May 2008.
27. D. Mcennis, C. Mckay, and I. Fujinaga, JAudio: A feature extraction library, in Proc. Int. Conf. Music Inf. Retrieval, 2005, pp. 600-603.
28. K. Mikolajczyk and C. Schmid, A performance evaluation of local descriptors, IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 10, pp. 1615-1630, Oct. 2005.
29. I. Laptev and T. Lindeberg, Space-time interest points, in Proc. IEEE ICCV, 2003, vol. 1, pp. 432-439.
30. P. Dollár, V. Rabaud, G. Cottrell, and S. Belongie, Behavior recognition via sparse spatio-temporal features, in Proc. IEEE Int. Workshop Vis. Surveillance Perform. Eval. Tracking Surveillance, Oct. 2005, pp. 65-72.
31. M. Bregonzio, S. Gong, and T. Xiang, Recognising action as clouds of space-time interest points, in Proc. IEEE CVPR, 2009, pp. 1948-1955.
32. C. G. Snoek and M. Worring, Multimodal video indexing: A review of the state-of-the-art, Multimedia Tools Appl., vol. 25, no. 1, pp. 5-35, Jan. 2005.
33. W. Jiang, C. Cotton, S.-F. Chang, D. Ellis, and A. Loui, Short-term audiovisual atoms for generic video concept classification, in Proc. ACM Int. Conf. Multimedia, Beijing, China, Oct. 2009, pp. 5-14.
34. M. Gurban and J.-P. Thiran, Information theoretic feature extraction for audio-visual speech recognition, IEEE Trans. Signal Process., vol. 57, no. 12, pp. 4765-4776, Dec. 2009.
35. S. Chu, S. Narayanan, and C.-C. Kuo, Environmental sound recognition with time-frequency audio features, IEEE Trans. Audio, Speech, Lang. Process., vol. 17, no. 6, pp. 1142-1158, Aug. 2009.
36. A. D. Dileep and C. C. Sekhar, Representation and feature selection using multiple kernel learning, in Proc. IJCNN, Jun. 2009, pp. 717-722.
37. S. Das, Filters wrappers and a boosting-based hybrid for feature selection, in Proc. IEEE ICML, 2001, pp. 98-101.
38. E. P. Xing, M. I. Jordan, and R. M. Karp, Feature selection for highdimensional genomic microarray data, in Proc. IEEE ICML, 2001, pp. 601-608.
39. P. K. Atrey, M. A. Hossain, A. E. Saddik, and M. S. Kankanhalli, Multimodal fusion for multimedia analysis: A survey, Multimedia Syst., vol. 16, no. 6, pp. 345-379, Nov. 2010.
40. S. T. Shivappa, M. M. Trivedi, and B. D. Rao, Audiovisual information fusion in human-computer interfaces and intelligent environments: A survey, Proc. IEEE, vol. 98, no. 10, pp. 1692-1715, Oct. 2010.
41. X. Zou and B. Bhanu, Tracking humans using multi-modal fusion, in Proc. IEEE Int. Workshop CVPR, Jun. 2005, p. 4.
42. J. Liu, S. Ali, and M. Shah, Recognizing human actions using multiple features, in Proc. IEEE Int. Conf. CVPR, Jun. 2008, pp. 1-8.
43. A. Vedaldi, V. Gulshan, M. Varma, and A. Zisserman, Multiple kernels for object detection, in Proc. IEEE ICCV, Sep. 2009, pp. 606-613.
44. M. Guillaumin, J. Verbeek, and C. Schmid, Multimodal semi-supervised learning for image classification, in Proc. IEEE Int. Conf. CVPR, Jun. 2010, pp. 902-909.
45. Z. Zeng, Z. Zhang, B. Pianfetti, J. Tu, and T. S. Huang, Audio-visual affect recognition in activation-evaluation space, in Proc. IEEE ICME, 2005, pp. 828-831. (Pubitemid 44668995)
46. M. Grabisch and E. Raufaste, An empirical study of statistical properties of the Choquet and Sugeno integrals, IEEE Trans. Fuzzy Syst., vol. 16, no. 4, pp. 839-850, Aug. 2008.
47. I. Laptev, On space-time interest points, Int. J. Comput. Vis., vol. 64, no. 2, pp. 107-123, Sep. 2005.
48. J. Yang, Y.-G. Jiang, A. G. Hauptmann, and C.-W. Ngo, Evaluating bagof-visual-words representations in scene classification, in Proc. ACM Int. Workshop Multimedia Inf. Retrieval, 2007, pp. 197-206.
49. M. Varma and D. Ray, Learning the discriminative power-invariance trade-off, in Proc. IEEE ICCV, Oct. 2007, pp. 1-8.
50. C.-C. Chang and C.-J. Lin, LIBSVM: A library for support vector machines, ACM Transactions on Intelligent Systems and Technology, vol. 2, no. 3, pp. 27:1-27, Apr. 2011.
51. M. Grabisch, A new algorithm for identifying fuzzy measures and its application to pattern recognition, in Proc. IEEE Int. Conf. Fuzzy Syst., Mar. 1995, vol. 1, pp. 145-150.