Statistical motion information extraction and representation for semantic video analysis

IEEE Transactions on Circuits and Systems for Video Technology

Volumn 19, Issue 10, 2009, Pages 1513-1528

  Papadopoulos, Georgios Th ^a   Briassouli, Alexia ^b   Mezaris, Vasileios ^b   Kompatsiaris, Ioannis ^b   Strintzis, Michael G ^a  

^a ARISTOTLE UNIVERSITY OF THESSALONIKI   (Greece)

^b INFORMATICS AND TELEMATICS INSTITUTE   (Greece)

Author keywords

Hidden Markov models (HMMs); Kurtosis; Motion representation; Semantic video analysis

Indexed keywords

BROADCAST VIDEO; COMPARATIVE EVALUATIONS; COMPLEMENTARY SETS; ENERGY DISTRIBUTIONS; HUMAN ACTIONS; KURTOSIS; MEASUREMENT NOISE; MOTION FEATURES; MOTION INFORMATION; MOTION REPRESENTATION; MOTION SIGNALS; SEMANTIC CLASS; SPATIAL ATTRIBUTE; STATISTICAL PROCESSING; VIDEO ANALYSIS;

COMPUTATIONAL GRAMMARS; DATA PROCESSING; OPTICAL FLOWS; PIXELS; SEMANTICS;

HIDDEN MARKOV MODELS;

EID: 70350060041     PISSN: 10518215     EISSN: None     Source Type: Journal    
DOI: 10.1109/TCSVT.2009.2026932     Document Type: Article

References (59)

1. S. Chang, "The holy grail of content-based media analysis," IEEE Multimedia, vol.9, no.2, pp. 6-10, Jun. 2002.
2. A. Smeulders, M. Worring, S. Santini, A. Gupta, and R. Jain, "Content-based image retrieval at the end of the early years," IEEE Trans. Pattern Anal. Mach. Intell., vol.22, no.12, pp. 1349-1380, Dec. 2000.
3. A. Francois, R. Nevatia, J. Hobbs, R. Bolles, and J. Smith, "VERL: An ontology framework for representing and annotating video events," IEEE Multimedia, vol.12, no.4, pp. 76-86, Oct.-Dec. 2005.
4. M. Naphade, J. Smith, J. Tesic, S. Chang, W. Hsu, L. Kennedy, A. Haupt-mann, and J. Curtis, "Large-scale concept ontology for multimedia," IEEE Multimedia, vol.13, no.3, pp. 86-91, Jul.-Sep. 2006.
5. D. Sadlier and N. O'Connor, "Event detection in field sports video using audio-visual features and a support vector machine," IEEE Trans. Circuit. Syst. Video Technol., vol.15, no.10, pp. 1225-1233, Oct. 2005.
6. W. Adams, G. Iyengar, C. Lin, M. Naphade, C. Neti, H. Nock, and J. Smith, "Semantic indexing of multimedia content using visual, audio, and text cues," EURASIP J. Appl. Signal Process., no.2, pp. 170-185, 2003.
7. L. Rabiner, "A tutorial on hidden Markov models and selected applications in speech recognition," Proc. IEEE, vol.77, no.2, pp. 257-286, Feb. 1989.
8. C. Cheng and C. Hsu, "Fusion of audio and motion information on HMM-based highlight extraction for baseball games," IEEE Trans. Multimedia, vol.8, no.3, pp. 585-599, Jun. 2006.
9. S. Liu, M. Xu, H. Yi, L. Chia, and D. Rajan, "Multimodal semantic analysis and annotation for basketball video," EURASIP J. Appl. Signal Process., pp. 1-13, 2006.
10. C. Snoek and M. Worring, "Multimodal video indexing: A review of the state-of-the-art," Multimedia Tools Appl., vol.25, no.1, pp. 5-35, 2005.
11. K. Peker, A. Alatan, and A. Akansu, "Low-level motion activity features for semantic characterization of video," in Proc. IEEE Int. Conf. Multimedia Expo (ICME), vol.2. 2000, pp. 801-804.
12. X. Sun, B. Manjunath, and A. Divakaran, "Representation of motion activity in hierarchical levels for video indexing and filtering," in Proc. IEEE Int. Conf. Image Processing (ICIP), vol 1. 2002, pp. I-149-I-152.
13. Y. Tan, D. Saur, S. Kulkami, and P. Ramadge, "Rapid estimation of camera motion from compressed video with application to video annotation," IEEE Trans. Circuit. Syst. Video Technol., vol.10, no.1, pp. 133-146, Feb. 2000.
14. L. Duan, J. Jin, Q. Tian, and C. Xu, "Nonparametric motion characterization for robust classification of camera motion patterns," IEEE Trans. Multimedia, vol.8, no.2, pp. 323-340, Apr. 2006.
15. R. Leonardi and P. Migliorati, "Semantic indexing of multimedia documents," IEEE Multimedia, vol.9, no.2, pp. 44-51, Apr.-Jun. 2002.
16. Y. Ma and H. Zhang, "Motion texture: A new motion-based video representation," in Proc. Int. Conf. Pattern Recognition (ICPR), vol.16. 2002, pp. 548-551. (Pubitemid 44844799)
17. B. Adams, C. Dorai, and S. Venkatesh, "Toward automatic extraction of expressive elements from motion pictures: Tempo," IEEE Trans. Multimedia, vol.4, no.4, pp. 472-481, Dec. 2002.
18. S. Dagtas, W. Al-Khatib, A. Ghafoor, R. Kashyap, P. Res, and B. Manor, "Models for motion-based video indexing and retrieval," IEEE Trans. Image Process., vol.9, no.1, pp. 88-101, Jan. 2000.
19. M. Roach, J. Mason, and M. Pawlewski, "Video genre classification using dynamics," in Proc. IEEE Int. Conf. Acoustics Speech Signal Processing (ICASSP), vol.3. 2001, pp. 1557-1560.
20. C. Chen, J.-C. Wang, J.-F. Wang, and Y. Hu, "Motion entropy feature and its applications to event-based segmentation of sports video," EURASIP J. Advances Signal Process., Jan. 2008.
21. H. Pan, P. van Beek, and M. Sezan, "Detection of slow-motion replay segments in sports video for highlights generation," in Proc. IEEE Int. Conf. Acoust. Speech Signal Process. (ICASSP '01), vol.3. May 2001, pp. 1649-1652.
22. L. Wang, X. Liu, S. Lin, G. Xu, and H. Shum, "Generic slow-motion replay detection in sports video," in Proc. IEEE Int. Conf. Image Process. (ICIP '04), vol.3. Oct. 2004, pp. 1585-1588.
23. N. Rea, R. Dahyot, and A. Kokaram, "Modeling high level structure in sports with motion driven HMMs," in Proc. IEEE Int. Conf. Acoust. Speech Signal Process. (ICASSP '04), vol.3. May 2004, pp. 621-624.
24. H. Chen, D. Chen, and S. Lee, "Object-based video similarity retrieval and its application to detecting anchorperson shots in news video," in Proc. Int. Symp. Multimedia Software Eng., 2003, pp. 172-179.
25. L. Chaisorn, T. Chua, and C. Lee, "The segmentation of news video into story units," in Proc. IEEE Int. Conf. Multimedia Expo (ICME '02), vol.1. 2002, pp. 73-76.
26. P. Turaga, R. Chellappa, V. S. Subrahmanian, and O. Udrea, "Machine recognition of human activities: A survey," IEEE Trans. Circuit. Syst. Video Technol., vol.18, no.11, pp. 1473-1488, Nov. 2008.
27. W. Hu, T. Tan, L. Wang, and S. Maybank, "A survey on visual surveillance of object motion and behaviors," IEEE Trans. Syst. Man Cybernetics, Part C, vol.34, no.3, pp. 334-352, Aug. 2004.
28. N. Cuntoor, B. Yegnanarayana, and R. Chellappa, "Activity modeling using event probability sequences," IEEE Trans. Image Process., vol.17, no.4, pp. 594-607, Apr. 2008.
29. A. Chan and N. Vasconcelos, "Modeling, clustering, and segmenting video with mixtures of dynamic textures," IEEE Trans. Pattern Anal. Mach. Intell., vol.30, no.5, pp. 909-926, May 2008.
30. R. Fablet and P. Bouthemy, "Motion recognition using nonparametric image motion models estimated from temporal and multiscale cooccurrence statistics," IEEE Trans. Pattern Anal. Mach. Intell., vol.25, no.12, pp. 1619-1624, Dec. 2003.
31. W. Hu, X. Xiao, Z. Fu, D. Xie, T. Tan, and S. Maybank, "A system for learning statistical motion patterns," IEEE Trans. Pattern Anal. Mach. Intell., vol.28, no.9, pp. 1450-1464, Sep. 2006.
32. Y. Rui and P. Anandan, "Segmenting visual actions-based on spatio-temporal motion patterns," in Proc. IEEE Int. Conf. Computer Vision Pattern Recognition (CVPR), vol.1. 2000, pp. 111-118.
33. A. Bobick and J. Davis, "The recognition of human movement using temporal templates," IEEE Trans. Pattern Anal. Mach. Intell., vol.23, no.3, pp. 257-267, Mar. 2001.
34. M. Barnard and J. Odobez, "Sports event recognition using layered HMMS," in Proc. IEEE Int. Conf. Multimedia Expo (ICME), 2005, pp. 1150-1153.
35. G. Xu, Y. Ma, H. Zhang, and S. Yang, "An HMM-based framework for video semantic analysis," IEEE Trans. Circuit. Syst. Video Technol., vol.15, no.11, pp. 1422-1433, Nov. 2005.
36. J. Huang, Z. Liu, and Y. Wang, "Joint scene classification and segmentation based on hidden Markov model," IEEE Trans. Multimedia, vol.7, no.3, pp. 538-550, Jun. 2005. (Pubitemid 40795052)
37. J. Wang, C. Xu, and E. Chng, "Automatic sports video genre classification using pseudo-2-D-HMM," in Proc. 18th Int. Conf. Pattern Recognition (ICPR), 2006, pp. 778-781.
38. Y. Haoran, D. Rajan, and C. Liang-Tien, "An efficient video classification system based on HMM in compressed domain," in Proc. Joint Conf. Int. Conf. Inf. Commun. Signal Process. Pacific Rim Conf. Multimedia, vol.3. Dec. 2003, pp. 1546-1550.
39. X. Gibert, H. Li, and D. Doermann, "Sports video classification using HMMS," in Proc. IEEE Int. Conf. Multimedia Expo (ICME), vol.2. Jul. 2003, pp. 345-348.
40. L. Xie, P. Xu, S. Chang, A. Divakaran, and H. Sun, "Structure analysis of soccer video with domain knowledge and hidden Markov models," Pattern Recog. Lett., vol.25, no.7, pp. 767-775, 2004.
41. M. Welling, "Robust higher order statistics," in Proc. Int. Workshop Artif. Intell. Statist. (AISTATS), 2005, pp. 405-412.
42. C.-Y. Chi, C.-Y. Chen, C.-H. Chen, and C.-C Feng, "Batch processing algorithms for blind equalization using higher order statistics," IEEE Signal Process. Mag., vol.20, no.1, pp. 25-49, Jan. 2003.
43. J. Kim, J. Choi, J. Yi, and M. Turk, "Effective representation using ICA for face recognition robust to local distortion and partial occlusion," in Proc. IEEE Trans. Pattern Anal. Mach. Intell., 2005, pp. 1977-1981.
44. C. Papadias, "Globally convergent blind source separation based on a multiuser kurtosis maximization criterion," IEEE Trans. Signal Proces., vol.48, no.12, pp. 3508-3519, Dec. 2000.
45. L. Zelnik-Manor and M. Irani, "Event-based analysis of video," in Proc. IEEE Int. Conf. Computer Vision Pattern Recognition (CVPR), vol.2. 2001, pp. 123-130.
46. E. Lee and D. Messerschmitt, "MIMO communications," Digital Communication. Boston, MA: Kluwer, 1994, pp. 517-541.
47. E. Moulines and J. Cardoso, "Second-order versus fourth-order music algorithms: An asymptotical statistical analysis," in Proc. IEEE Signal Process. Workshop Higher Order Stat., Chamrousse, France, Jun. 1991, pp. 121-130.
48. B. Sinha, "Detection of multivariate outliers in elliptically symmetric distributions," Ann. Stat., vol.12, no.4, pp. 1558-1565, 1984.
49. G. T. Papadopoulos, V. Mezaris, I. Kompatsiaris, and M. G. Strintzis, "Accumulated motion energy fields estimation and representation for semantic event detection," in Proc. Int. Conf. Image Video Retrieval (CIVR), 2008, pp. 221-230.
50. V. Kobla, D. Doermann, and K. Lin, "Archiving, indexing, and retrieval of video in the compressed domain," in Proc. SPIE Conf. Multimedia Storage Archiving Systems, 1996, pp. 78-89.
51. M. Proesmans, L. Van Gool, E. Pauwels, and A. Oosterlinck, "Determination of optical flow and its discontinuities using non-linear diffusion," in Proc. Eur. Conf. Comput. Vision (ECCV '94), pp. 294-304.
52. R. Gonzalez and R. Woods, Digital Image Processing. Englewood Cliffs, NJ: Prentice Hall, 2007.
53. G. Giannakis and M. Tsatsanis, "Time-domain tests for Gaussianity and time-reversibility," IEEE Trans. Signal Process., vol.42, no.12, pp. 3460-3472, Dec. 1994.
54. R. Duda, P. Hart, and D. Stork, Pattern Classification. Chichester, U.K.: Wiley-Interscience, 2000.
55. Hidden Markov Model Toolkit (HTK) http://htk.eng.cam.ac.uk
56. S. Zhao and H. Lee, "Human Silhouette extraction based on HMM," in Proc. Int. Conf. Pattern Recognition (ICPR), 2006, pp. 994-997.
57. A. Mittal, L. Zhao, and L. Davis, "Human body pose estimation using silhouette shape analysis," in Proc. IEEE Int. Conf. Advanced Video Signal-Based Surveillance, 2003, pp. 263-270.
58. C. Schuldt, I. Laptev, and B. Caputo, "Recognizing human actions: A local SVM approach," in Proc. Int. Conf. Pattern Recognition (ICPR), vol.3. 2004, pp. 32-36.
59. S. Fu, R. Gutierrez-Osuna, A. Esposito, P. Kakumanu, and O. Garcia, "Audio/visual mapping with cross-modal hidden Markov models," IEEE Trans. Multimedia, vol.7, no.2, pp. 243-252, Apr. 2005.