Evaluating multimedia features and fusion for example-based event detection

Machine Vision and Applications

Volumn 25, Issue 1, 2014, Pages 17-32

  Myers, Gregory K ^a   Nallapati, Ramesh ^a,d   Van Hout, Julien ^a   Pancoast, Stephanie ^a   Nevatia, Ramakant ^b   Sun, Chen ^b   Habibian, Amirhossein ^c   Koelma, Dennis C ^c   Van De Sande, Koen E A ^c   Smeulders, Arnold W M ^c   Snoek, Cees G M ^c  

^a SRI INTERNATIONAL   (United States)

^b UNIVERSITY OF SOUTHERN CALIFORNIA   (United States)

^c UNIVERSITY OF AMSTERDAM   (Netherlands)

^d IBM T J WATSON RESEARCH CENTER   (United States)

Author keywords

Content extraction; Difference coding; Late fusion; Multimedia event detection; Video retrieval

Indexed keywords

EID: 84894905430     PISSN: 09328092     EISSN: 14321769     Source Type: Journal    
DOI: 10.1007/s00138-013-0527-8     Document Type: Article

References (46)

1. Smeaton, A.F.; Over, P.; Kraaij, W.: Evaluation campaigns and TRECVID. In: Proceedings of the 8th ACM international workshop on multimedia information retrieval, Santa Barbara, 26-27 October 2006 (MIR '06). ACM Press, New York, pp. 321-330 (2006)
2. Jiang, Y.-G.; Bhattacharya, S.; Chang, S.-F.; Shah, M.: High-level event recognition in unconstrained videos. Int. J. Multimed. Inform. Retr. 1-29 (2012)
3. Natarajan, P.; Wu, S.; Vitaladevuni, S.; Zhuang, X.; Tsakalidis, S.; Paurk, U.; Prasad.; R.: Multimodal feature fusion for robust event detection in web videos. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition (CVPR), pp. 1298-1305 (2012)
4. Sawhney, H.; Cheng, H.; Divakaran, A.; Javed, O.; Liu, J.; Yu, Q.; Ali, S.; Tamrakar, A.: Evaluation of low-level features and their combinations for complex event detection in open source videos. CVPR, 2496-2499 (2012)
5. Jiang, Y.: Super: towards real-time event recognition in internet videos. ACM Int. Conf. Multimed. Retr. (ICMR) (2012) (article no. 33)
6. Ballan, L.; Bertini, M.; Del Bimbo, A.; Seidenari, L.; Serra, G.: Event detection and recognition for semantic annotation of video. Multimed. Tools Appl. 51(1), 279-302 (2011)
7. Xu, D.; Chang, S.-F.: Video event recognition using kernel methods with multilevel temporal alignment. IEEE Trans. Pattern Anal. Mach. Intell. (IEEE TPAMI) 30(11), 1985-1997 (2008)
8. Snoek, C.G.M.; Worring, M.: Concept-based video retrieval. Found. Trends Inf. Retr. 2(4), 214-322 (2009)
9. Felzenszwalb, P.; Girshick, R.; McAllester, D.; Ramanan, D.: Object detection with discriminatively trained part-based models. IEEE TPAMI 32(9), 1627-1645 (2010)
10. Li, L.; SU, H.; Xing, E.; Fei-Fei, L.: Object bank: a high-level image representation for scene classification and semantic feature sparsification. Adv. Neural Inf. Process. Syst.; 24 (2010)
11. Sadanand, S.; Corso, J.J.: Action bank: a high-level representation of activity in video. CVPR (2012)
12. Snoek, C.G.M.; Smeulders, A.W.M.: Visual-concept search solved? IEEE Comput. 43(6), 76-78 (2010)
13. Merler, M.; Huang, B.; Xie, L.; Hua, G.; Natsev, A.: Semantic model vectors for complex video event recognition. IEEE Trans. Multimed. (TMM) 14(1), 88-101 (2012)
14. Althoff, T.; Song, H.; Darrell, T.: Detection bank: an object detection based video representation for multimedia event recognition. ACM Multimed. (MM) (2012)
15. Tsampoulatidis, I.; Gkalelis, N.; Dimou, A.; Mezaris, V.; Kompatsiaris, I.: High-level event detection in video exploiting discriminant concepts. In: Proceedings of the 1st ACM international conference on multimedia retrieval, pp. 85-90 (2011)
16. Habibian, A.; van de Sande, K.E.A.; Snoek, C.G.M.: Recommendations for video event recognition using concept vocabularies. In: Proceedings of the ACM international conference on multimedia retrieval, pp. 89-96 Dallas (2013)
17. Perera, A.G.A.; Oh, S.; Leotta, M.; Kim, I.; Byun, B.; Lee, C.-H.; McCloskey, S.; Liu, J.; Miller, B.; Huang, Z.F.; Vahdat, A.; Yang, W.; Mori, G.; Tang, K.; Koller, D.; Fei-Fei, L.; Li, K.; Chen, G.; Corso, J.; Fu, Y.; Srihari, R.: GENIE TRECVID 2011 multimedia event detection: late-fusion approaches to combine multiple audio-visual features. In: NIST TRECVID, workshop (2011)
18. Strassel, S.; Morris, A.; Fiscus, J.; Caruso, C.; Lee, H.; Over, P.; Fiumara, J.; Shaw, B.; Antonishek, B.; Michel, M.: Creating HAVIC: heterogeneous audio visual internet collection. In: Calzolari N.; Choukri K.; Declerck T.; Uǧur Doǧan M.; Maegaard B.; Mariani J.; Odijk J.; Piperidis S. (eds.) Proceedings of the eighth international conference on language resources and evaluation, Istanbul (2012)
19. Jaakkola, T.; Haussler. D.: Exploiting generative models in discriminative classifiers. In:Proceedings of the 1998 conference on advances in neural information processing systems II, pp. 489-493 (1999)
20. Jégou, H.; Perronnin, F.; Douze, M.; Sanchez, J.; Pérez, P.; Schmid, C.: Aggregating local image descriptors into compact codes. IEEE TPAMI 34(9), 1704-1716 (2012)
21. Perronnin, F.; Dance, C.: Fisher kernels on visual vocabularies for image categorization. CVPR, (2007)
22. Snoek, C.G.M.; van de Sande, K.E.A.; Habibian, A.; Kordumova, S.; Li, Z.; Mazloom, M.; Pintea, S.L.; Tao, R.; Koelma, D.C.; Smeulders, A.W.M.: The MediaMill TRECVID 2012 semantic video search engine. In: Proceeding of the TRECVID workshop, Gaithersburg (2012)
23. Tuytelaars, T.; Mikolajczyk, K.: Local invariant feature detectors: a survey. Found. Trends. Comput. Graph. Vis. 3(3), 177-280 (2008)
24. Snoek, C.G.M.; Worring, M.; Geusebroek, J.-M.; Koelma, D.C.; Seinstra, F.J.: On the surplus value of semantic video analysis beyond the key frame.In: Proceedings of the IEEE international conference on multimedia and expo (2005)
25. Lazebnik, S.; Schmid, C.; Ponce, J.: Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. CVPR 2, 2169-2178 (2006) (New York)
26. van de Sande, K.E.A.; Gevers, T.; Snoek, C.G.M.: Evaluating color descriptors for object and scene recognition. IEEE TPAMI 32(9), 1582-1596 (2010)
27. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60, 91-110 (2004)
28. Geusebroek, J.-M.; Boomgaard, R.; Smeulders, A.W.M.; Geerts, H.: Color invariance. IEEE TPAMI 23(12), 1338-1350 (2001)
29. van Gemert, J.C.; Snoek, C.G.M.; Veenman, C.J.; Smeulders, A.W.M.; Geusebroek, J.-M.: Comparing compact codebooks for visual categorization. Comput. Vis. Image Underst. 114(4), 450-462 (2010)
30. Maji, S.; Berg, A.C.; Malik, J.: Classification using intersection kernel support vector machines is efficient. In: Proceedings of the IEEE computer society conference on CVPR, pp. 619-626, Anchorage (2008)
31. Wang, H.; Kläser, A.; Schmid, C.; Cheng-Lin, L.: Action recognition by dense trajectories. CVPR, 3169-3176 (2011)
32. Chen, M.-Y.; Hauptmann, A.: MoSIFT: recognizing human actions in surveillance videos. CMU-CS-09-161. Carnegie Mellon Univ. (2009)
33. Laptev, I.: On space-time interest points. Int. J. Comput. Vis. 64(2/3), 107-123 (2005)
34. Sun, C.; Nevatia, R.: Large scale web video classification by use of Fisher vectors. In: Workshop on applications of computer vision, Clearwater (2013) (January)
35. Chechik, G.; Ie, E.; Rehn, M.; Bengio, S.; Lyon, D.: Large-scale content-based audio retrieval from text queries. In: Proceedings of 1st ACM international conference on multimedia information retrieval (MIR '08), pp. 105-112, New York (2008)
36. Uchida, Y.; Sakazawa, S.; Argawal, M.; Akbacak, M.: KDDI labs and SRI international at TRECVID 2010: content-based copy detection. In: NIST TRECVID 2010 evaluation, workshop (2010)
37. Jiang, Y.; Zeng, X.; Ye, G.; Ellis, D.; Shah, M.; Chang, S.: Columbia-UCF TRECVID 2010 multimedia event detection: combining multiple modalities, contextual concepts, and temporal matching. In: NIST TRECVID, workshop (2010)
38. Pancoast, S.; Akbacak, M.: Bag-of-audio-words approach for multimedia event detection. In: Proceedings of interspeech (2012)
39. Merler, M.; Huang, B.; Xie, L.; Hua, G.; Natsev, A.: Semantic model vectors for complex video event recognition. IEEE Trans. Multimed. 14(1), 88-101 (2012)
40. Over, P.; Awad, G.; Michel, M.; Fiscus, J.; Sanders, G.; Shaw, B.; Kraaij, W.; Smeaton, A.F.; Quéenot, G.: TRECVID 2012 - an overview of the goals, tasks, data, evaluation mechanisms, and metrics. In: Proceedings of TRECVID (2012) http://www-nlpir.nist.gov/projects/tvpubs/tv12.papers/ tv12overview.pdf
41. Berg, A.; Deng, J.; Satheesh, S.; Su, H.; Li, F.-F.: Imagenet large scale visual recognition challenge (2011) http://www.image-net.org/challenges/LSVRC/ 2011/
42. Janin, A.; Stolcke, A.; Anguera, X.; Boakye, K.; Çetin, Ö.; Frankel, J.; Zheng, J.: The ICSI-SRI spring 2006 meeting recognition system, MLMI'06. In: Proceedings of the third international conference on machine learning for multimodal, interaction, pp. 444-456 (2006)
43. van Hout, J.; Akbacak, M.; Castaneda, D.; Yeh, E.; Sanchez, M.: Extracting audio and spoken concepts for multimedia event detection. In: International conference on acoustics, speech, and signal processing (ICASSP) (2013)
44. Nallapati, R.; Yeh, E.; Myers, G.: Sparse mixture model: late fusion with missing scores for multimedia event detection. Algorithms and systems VII. SPIE Multimed. Content Access (2012)
45. Fiscus, J.; Michel, M.: TRECVID 2012 multimedia event detection task. In: NIST TRECVID 2012 evaluation, workshop (2012)
46. Hauptmann, A.; Yan, R.; Lin, W.-H.; Christel, M.; Wactlar, H.: Can high-level concepts fill the semantic gap in video retrieval? A case study with broadcast retrieval. IEEE Trans Multimed. 9(5), 958-966 (2007)