Human behavior analysis for highlight ranking in broadcast racket sports video

IEEE Transactions on Multimedia

Volumn 9, Issue 6, 2007, Pages 1167-1181

  Zhu, Guangyu ^b   Huang, Qingming ^a,c   Xu, Changsheng ^a,d   Xing, Liyuan ^c   Gao, Wen ^a,b   Yao, Hongxun ^a,b  

^a IEEE

^b HARBIN INSTITUTE OF TECHNOLOGY   (China)

^c UNIVERSITY OF CHINESE ACADEMY OF SCIENCES   (China)

^d INSTITUTE FOR INFOCOMM RESEARCH   (Singapore)

Author keywords

Action recognition; Affective analysis; Highlight ranking; Semantic analysis; Sports video analysis

Indexed keywords

FEATURE EXTRACTION; IMAGE ANALYSIS; MATHEMATICAL MODELS; SEMANTICS;

ACTION RECOGNITION; AFFECTIVE ANALYSIS; HIGHLIGHT RANKING; MOTION ANALYSIS; SEMANTIC ANALYSIS; SPORTS VIDEO ANALYSIS; SUPPORT VECTOR REGRESSION;

BEHAVIORAL RESEARCH;

EID: 34648857539     PISSN: 15209210     EISSN: None     Source Type: Journal    
DOI: 10.1109/TMM.2007.902847     Document Type: Article

References (53)

1. Y. Gong, T. S. Lim, H. C. Chua, H. J. Zhang, and M. Sakauchi, "Automatic parsing of TV soccer programs," in Proc. Int. Conf. Multimedia Computing and System, Washington, DC, 1995, pp. 167-174.
2. A. Ekin, A. M. Tekalp, and R. Mehrotra, "Automatic soccer video analysis and summarization," IEEE Trans. Image. Processing, vol. 12, no. 7, pp. 796-807, Jul. 2003.
3. L. Xie, P. Xu, S. F. Chang, A. Divakaran, and H. Sun, "Structure analysis of soccer video with domain knowledge and hidden Markov models," Pattern Recognit. Lett., vol. 25, no. 7, pp. 767-775, 2004.
4. Y. Rui, A. Gupta, and A. Acero, "Automatically extracting highlights for TV baseball programs," in Proc. ACM Multimedia, Los Angeles, CA, 2000, pp. 105-115.
5. J. Assfalg, M. Bertini, C. Colombo, A. Delbimbo, and W. Nunziati, "Semantic annotation of soccer video: Automatic highlights identification," Comput. Vis. Image. Understand., vol. 92, no. 2-3, pp. 285-305, 2003.
6. N. Babaguchi, Y. Kawai, T. Ogura, and T. Kitahashi, "Personalized abstraction of broadcasted American, football video by highlight selection," IEEE Trans. Multimedia, vol. 6, no. 4, pp. 575-586, Aug. 2004.
7. K. Wan, J. Wang, C. Xu, and Q. Tian, "Automatic sports highlights extraction, with content augmentation," in Proc. Pacific-Rim Conf. Multimedia, Tokyo, 2004, vol. 3332, pp. 19-26.
8. D. Liang, Y. Liu, Q. Huang, and G. Zhu, "Video2Cartoon: Generating 3-D cartoon from broadcast soccer video," in Proc. ACM Multimedia, Singapore, 2005, pp. 217-218.
9. G. S. Pingali, Y. Jean, A. Opalach, and I. Carlbom, "LucentVision: Converting real world events into multimedia experiences," in Proc. Int. Conf. Multimedia & Expo, New York, 2000, vol. 3, pp. 1433-1436.
10. C. C. Cheng and C. T. Hsu, "Fusion of audio and motion information on HMM-based highlight extraction," IEEE Trans. Multimedia, vol. 8, no. 3, pp. 585-599, 2006.
11. H. L. Wang and L. F. Cheong, "Affective understanding in film," IEEE Trans. Circuits Syst. Video Technol., vol. 16, no. 6, pp. 689-704, 2006.
12. [Online]. Available: http://en.wikipedia.org/wiki/List-of-sports#Racket- sports
13. G. Sudhir, J. C. M. Lee, and A. K. Jain, "Automatic classification of tennis video for high-level content-based retrieval," in Proc. Int. Workshop on Content-Based Access of Image and Video Databases, Bombay, 1998, pp. 81-90.
14. G. S. Pingali, Y. Jean, and I. Carlbom, "Real time tracking for enhanced tennis broadcasts," in Proc. Conf. Computer Vision and Pattern Recognition, Santa Barbara, CA, 1998, pp. 260-265.
15. X. Yu, C. H. Sim, J. R. Wang, and L. F. Cheong, "A trajectory-based ball detection and tracking algorithm in broadcast tennis video," in Proc. Int. Conf. Image Processing, Singapore, 2004, vol. 2, pp. 1049-1052.
16. H. Miyamori and S. Iisaku, "Video annotation for content-based retrieval using human behavior analysis and domain knowledge," in Proc. Int. Conf. Automatic Face and Gesture Recognition, Grenoble, France, 2000, pp. 320-325.
17. H. Miyamori, "Improving accuracy in behavior identification for content-based retrieval by using audio and video information," in Proc. Int. Conf. Pattern Recognition, 2002, vol. 2, pp. 826-830.
18. E. Kijak, G. Gravier, P. Gros, L. Oisel, and F. Bimbot, "HMM based structuring of tennis videos using visual and audio cues," in Proc. Int. Conf. Multimedia Expo, Baltimore, MD, 2003, vol. 3, pp. 309-312.
19. M. Xu, L. Y. Duan, C. S. Xu, and Q. Tian, "A fusion scheme of visual and auditory modalities for event detection in sports video," in Proc. Int. Conf. Acoustics, Speech, and Signal. Processing, Hong Kong, 2003, vol. 3, pp. 189-192.
20. X. Tong, Q. Liu, Y. Zhang, and H. Lu, "Highlight ranking for sports video browsing," in Proc. ACM Multimedia, Singapore, 2005, pp. 519-522.
21. L. Xing, H. Yu, Q. Huang, Q. Ye, and A. Divakaran, "Subjective evaluation criterion for selecting affective features and modeling highlights," Proc. SPIE Multimedia Content Analysis, Management, and Retrieval, vol. 6073, 2006.
22. L. Y. Duan, M. Xu, T. S. Chua, Q. Tian, and C. S. Xu, "A mid-level representation framework for semantic sports video analysis," in Proc. ACM Multimedia, Berkeley, CA, 2003, pp. 33-44.
23. Q. Ye, Q. Huang, W. Gao, and S. Jiang, "Exciting event detection in broadcast soccer video with, mid-level description and incremental learning," in. Proc. ACM Multimedia, Singapore, 2005, pp. 455-458.
24. D. Tjondronegoro, Y. P. P. Chen, and B. Pham, "Content-based video indexing for sports applications using integrated multi-modal approach," in Proc. ACM Multimedia, Singapore, 2005, pp. 1035-1036.
25. Q. Ye, Q. Huang, S. Jiang, Y. Liu, and W. Gao, "Jersey number detection in sports video for athlete identification," in Proc. SPIE Visual. Communications & Image Processing, Beijing, China, 2005, pp. 1599-1606.
26. G. Zhu, C. Xu, Q. Huang, W. Gao, and L. Xing, "Player action recognition in broadcast tennis video with applications to semantic analysis of sports game," in Proc. ACM Multimedia, Santa Barbara, CA, 2006, pp. 431-440.
27. M. Shall and R. Jain, Motion-Based Recognition. Norwell, MA: Kluwer, 1997.
28. D. M. Gavrila, "The visual analysis of human movement: A survey," Comput. Vis. Image. Understand., vol. 73, no. 1, pp. 82-98, 1999.
29. A. F. Bobick and J. W. Davis, "The recognition, of human movement using temporal templates," IEEE Trans. Pattern Anal. Machine Intell., vol. 23, no. 3, pp. 257-267, 2001.
30. Y. Song, L. Goncalves, and P. Perona, "Unsupervised learning of human motion," IEEE Trans. Pattern Anal Machine Intell., vol. 25, no. 7, pp. 814-827, 2003.
31. R. Cutler and L. S. Davis, "Robust real-time periodic motion detection, analysis, and application," IEEE Trans. Pattern Anal Machine Intell., vol. 22, no. 8, pp. 781-796, 2000.
32. A. A. Efros, A. C. Berg, G. Mori, and J. Malik, "Recognizing action at a distance," in Proc. Int. Conf. Computer Vision, Nice, France, 2003, vol. 2, pp. 726-733.
33. M. C. Roh, B. Christmas, J. Kittler, and S. W. Lee, "Robust player gesture spotting and recognition in low-resolution sports video," in Proc. Eur. Conf. Computer Vision, 2006, pp. 347-358.
34. W. L. Lu and J. J. Little, "Tracking and recognizing actions at a distance," in Proc. Eur. Conf. Computer Vision Workshop on Computer Vision Based Analysis in Sport Envimnments, 2006, pp. 49-60.
35. R. W. Picard, Affective Computing. Cambridge: MIT Press, 2000.
36. A. Hanjalic and L. Q. Xu, "Affective video content representation and modeling," IEEE Trans. Multimedia, vol. 7, no. 1, pp. 143-154, 2005.
37. A. Hanjalic, "Adaptive extraction of highlights from a sport video based on excitement modeling," IEEE Trans. Multimedia, vol. 7, no. 6, pp. 1114-1122, 2005.
38. Z. Xiong, R. Radhakrishnan, and A. Divakaran, "Generation of sports highlights using motion activity in combination with a common audio feature extraction framework," in Proc. Int. Conf. Image Processing, Barcelona, Spain, 2003, vol. 1, pp. 5-8.
39. M. S. Lew, N. Sebe, C. Djeraba, and R. Jain, "Content-based multimedia information retrieval: State of the art and challenges," ACM Trans. Multimedia Comput., Commun., Applicat., vol. 2, no. 1, pp. 1-19, 2006.
40. G. Zhu, D. Liang, Y. Liu, Q. Huang, and W. Gao, "Improving particle filter with support vector regression for efficient visual tracking," in Proc. Int. Conf. Image. Processing, Genova, Italy, 2005, vol. 2, pp. 422-425.
41. S. Jiang, Q. Ye, W. Gao, and T. Huang, "A new method to segment playfield and its applications in match analysis in. sports video," in Proc. ACM Multimedia, New York, 2004, pp. 292-295.
42. Q. Ye, W. Gao, and W. Zeng, "Color image segmentation using density-based clustering," in Proc. Int. Conf. Acoustics, Speech, and Signal Processing, Hong Kong, 2003, vol. 3, pp. 345-348.
43. B. K. P. Horn and B. G. Schunck, "Determining optical flow," Artif. Intell., vol. 17, pp. 185-203, 1981.
44. N. Dalal and B. Triggs, "Histograms of oriented gradients for human detection," in Proc. Conf. Computer Vision and Pattern Recognition, San Diego, CA, 2005, pp. 886-893.
45. D. Comaniciu, V. Ramesh, and P. Meer, "Kernel-based object tracking," IEEE Trans. Pattern Anal Machine Intell., vol. 25, no. 5, pp. 564-577, 2003.
46. V. Vapnik, The Nature of Statistical Learning Theory. New York: Springer-Verlag, 1995.
47. B. Schölkopf, A. J. Smola, R. C. Williamson, and P. L. Bartlett, "New support vector algorithm," Neural Comput., vol. 12, pp. 1083-1121, 2000.
48. R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision. Cambridge, U.K.: Cambridge Univ. Press, 2003.
49. F. Dufaux and J. Konrad, "Efficient, robust, and fast global motion estimation for video coding," IEEE Trans. Image. Processing, vol. 9, no. 3, pp. 497-501, 2000.
50. K. A. Peker and A. Divakaran, "Framework for measurement of the intensity of motion, activity of video segments," Proc. SPIE, vol. 4862, pp. 126-137, 2002.
51. J. Chin, V. Diehl, and K. Norman, "Development of an instrument measuring user satisfaction of the human-computer interface," in Proc. SIGCHI on Human Factors in CS, Washington, DC, 1988, pp. 213-218.
52. A. K. Jain, "Statistical pattern recognition: A review," IEEE Trans. Pattern Anal. Machine, Intell., vol. 22, no. 1, pp. 4-37, 2000.
53. C. Xu, J. Wang, K. Wan, Y. Li, and L. Duan, "Live sports event detection based on broadcast video and web-casting text," in Proc. ACM Multimedia, Santa Barbara, CA, 2006, pp. 221-230.