Learning automatic concept detectors from online video

Computer Vision and Image Understanding

Volumn 114, Issue 4, 2010, Pages 429-438

  Ulges, Adrian ^a   Schulze, Christian ^a   Koch, Markus ^b   Breuel, Thomas M ^b  

^a GERMAN RESEARCH CENTER FOR ARTIFICIAL INTELLIGENCE DFKI   (Germany)

^b Rheinland Pfälzische Technische Universität Kaiserslautern Landau   (Germany)

Author keywords

Concept detection; Content based video retrieval; Online video

Indexed keywords

COMPLEX DOMAINS; CONCEPT DETECTION; CONTENT-BASED VIDEO RETRIEVAL; DATA SETS; HIGH QUALITY; KEY COMPONENT; MANUAL ANNOTATION; ONLINE VIDEO; PERFORMANCE IMPROVEMENTS; PERFORMANCE LOSS; PRACTICAL USE; QUANTITATIVE EXPERIMENTS; RELATIVE PERFORMANCE; RESEARCH PROTOTYPE; SCALE-UP; SEMANTIC CONCEPT; TARGET DOMAIN; TRAINING MATERIAL; TRAINING SETS; VIDEO CONTENTS; VIDEO SHARING; WEB VIDEO; YOUTUBE;

CONCENTRATION (PROCESS); IMAGE RETRIEVAL; MULTIMEDIA SYSTEMS; STANDARDS; VISUAL COMMUNICATION;

DETECTORS;

EID: 77249168140     PISSN: 10773142     EISSN: 1090235X     Source Type: Journal    
DOI: 10.1016/j.cviu.2009.08.002     Document Type: Article

References (54)

1. R. Akbani, S. Kwek, N. Japkowicz, Applying support vector machines to imbalanced datasets, in: Proc. Europ. Conf. Machine Learning, September 2004, pp. 39-50.
2. A. Amir, M. Berg, S. Chang, W. Hsu, G. Iyengar, C.-Y. Lin, M. Naphade, A. Natsev, C. Neti, H. Nock, J. Smith, B. Tseng, Y. Wu, D. Zhang, IBM research TRECVID-2003 video retrieval system, in: Proc. TRECVID Workshop (unreviewed workshop paper), November 2003.
3. H. Bay, T. Tuytelaars, L. van Gool, SURF: speeded up robust features, in: Proc. Europ. Conf. Computer Vision, May 2006, pp. 404-417.
4. D. Borth, A. Ulges, C. Schulze, T. Breuel, Keyframe extraction for video tagging and summarization, in: Proc. Informatiktage 2008, pp. 45-48.
5. C. Snoek et al., The MediaMill TRECVID 2007 semantic video search engine, in: Proc. TRECVID Workshop (Unreviewed Workshop Paper), November 2007.
6. C. Snoek et al., The MediaMill TRECVID 2008 semantic video search engine, in: Proc. TRECVID Workshop (Unreviewed Workshop Paper), November 2008.
7. M. Campbell, A. Haubold, M. Liu, A. Natsev, J. Smith, J. Tesic, L. Xie, R. Yan, J. Yang, IBM research TRECVID-2007 video retrieval system, in: Proc. TRECVID Workshop (Unreviewed Workshop Paper), November 2007.
8. C.-C. Chang, C.-J. Lin, LIBSVM: A Library for Support Vector Machines, 2001. Software available at %3chttp://www.csie.ntu.edu.tw/cjlin/libsvm%3e.
9. S.-F. Chang, D. Ellis, W. Jiang, K. Lee, A. Yanagawa, A. Loui, J. Luo, Large-scale multimodal semantic concept detection for consumer video, in: Proc. Int. Workshop Multimedia Information Retrieval, September 2007, pp. 255-264.
10. S.-F. Chang, W. Hsu, W. Jiang, L. Kennedy, D. Xu, A. Yanagawa, E. Zavesky, Columbia University TRECVID-2006 video search and high-level feature extraction, in: Proc. TRECVID Workshop (Unreviewed Workshop Paper), November 2006.
11. S.-F. Chang, W. Jiang, A. Yanagawa, E. Zavesky, Columbia University TRECVID2007 high-level feature extraction, in: Proc. TRECVID Workshop (Unreviewed Workshop Paper), November 2007.
12. T. Deselaers, L. Pimenidis, H. Ney, Bag-of-visual-words models for adult image classification and filtering, in: Proc. Int. Conf. Pattern Recognition, December 2008, pp. 1-4.
13. D. Ding, D. Zhang, Probabilistic model supported rank aggregation for the semantic concept detection in video, in: Proc. Int. Conf. Image and Video Retrieval, July 2007, pp. 587-594.
14. M. Everingham, L. Van Gool, C. Williams, J. Winn, A. Zisserman. The PASCAL visual object classes challenge 2007 (VOC2007) results, Technical Report, PASCAL Challenge Workshop, October 2007. %3chttp://pascallin.ecs.soton.ac.uk/challenges/VOC/voc2007/workshop.html%3e.
15. M. Everingham, L. Van Gool, C. Williams, J. Winn, A. Zisserman, The PASCAL Visual Object Classes Challenge 2008 (VOC2008) Results, October 2008.
16. A. Haubold, M. Naphade, Classification of video events using 4-dimensional time-compressed motion features, in: Proc. Int. Conf. Image and Video Retrieval, July 2007, pp. 178-185.
17. A. Hauptmann, R. Yan, W. Lin, How many high-level concepts will fill the semantic gap in news video retrieval? in: Proc. Int. Conf. Image and Video Retrieval, July 2007, pp. 627-634.
18. C. Hsu, C. Chang, C. Lin, A practical guide to support vector classification, Technical Report, Department of Computer Science and Information Engineering, National Taiwan University, 2003.
19. M. Huijbregts, R. Ordelman, F. de Jong. Annotation of heterogeneous multimedia content using automatic speech recognition, in: Proc. Int. Conf. Semantics and Digital Media Technology, December 2007, pp. 78-90.
20. W. Jiang, S.-F. Chang, A. Loui, Context-based concept fusion with boosted conditional random fields, in: Proc. Int. Conf. Acoustics, Speech, and Signal Processing, April 2007.
21. T. Kölsch, Local Features for Image Classification, Diploma thesis, RWTH Aachen, 2003.
22. W. Kraaij, P. Over, TRECVID-2008 high-level feature task: overview, in: Proc. TRECVID Workshop, November 2008. %3chttp://www-nlpir.nist.gov/projects/tvpubs/tv.pubs.org.html%3e.
23. X. Li, C. Snoek, M. Worring, Learning tag relevance by neighbor voting for social image retrieval, in: Proc. Int. Conf. on Multimedia Information Retrieval, October 2008, pp. 180-187.
24. Lienhart R. Reliable transition detection in videos: a survey and Practitioner's guide. Int. J. Img. Graph. 1 3 (2001) 286-469
25. W.-H. Lin, A. Hauptmann, News video classification using SVM-based multimodal classifiers and combination strategies, in: Proc. Int. Conf. on Multimedia, December 2002, pp. 323-326.
26. Lowe D. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60 2 (2004) 91-110
27. LSCOM - Large-scale Concept Ontology for Multimedia, 2008. %3chttp://www.lscom.org%3e (retrieved August 2008).
28. Manjunath B., Ohm J.-R., Vasuvedan V., and Yamada A. Color and texture descriptors. IEEE Trans. Circuit. Syst. Video Technol. 11 6 (2001) 703-715
29. Naphade M., Smith J., Tesic J., Chang S., Hsu W., Kennedy L., Hauptmann A., and Curtis J. Large-scale concept ontology for multimedia. IEEE MultiMedia 13 3 (2006) 86-91
30. P. Over, T. Ianeva, W. Kraaij, A.F. Smeaton, TRECVID 2005 - an overview, in: Proc. TRECVID Workshop (Unreviewed Workshop Paper), November 2005.
31. R. Paredes, A. Perez-Cortes, Local representations and a direct voting scheme for face recognition, in: Proc. Workshop on Pattern Rec. and Inf. Systems, July 2001, pp. 71-79.
32. Quelhas P., Monay F., Odobez J., Gatica-Perez D., and Tuytelaars T. A thousand words in a scene. IEEE Trans. Pattern Anal. Mach. Intell. 29 9 (2007) 1575-1589
33. G. Schindler, L. Zitnick, M. Brown, Internet video category recognition, in: Proc. First Internet Vision Workshop, June 2008, pp. 1-7.
34. Schölkopf B., and Smola A. Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond (2001), MIT Press, Cambridge, MA, USA
35. J. Sivic, A. Zisserman, Video google: a text retrieval approach to object matching in videos, in: Proc. Int. Conf. Computer Vision, October 2003, pp. 1470-1477.
36. A. Smeaton, Large scale evaluations of multimedia information retrieval: the TRECVid experience, in: Proc. Int. Conf. Image and Video Retrieval, July 2005, pp. 11-17.
37. Smeaton A. Techniques used and open challenges to the analysis, indexing retrieval of digital video. Inf. Syst. 32 4 (2007) 545-559
38. A. Smeaton, W. Kraaij, P. Over, TREC video retrieval evaluation: a case study and status report, in: Proc. RIAO: Coupling Approaches, Coupling Media and Coupling Languages for Information Retrieval, April 2004.
39. A. Smeaton, P. Over, W. Kraaij, TRECVID: evaluating the effectiveness of information retrieval tasks on digital video, in: Proc. Int. Conf. on Multimedia, October 2004, pp. 652-655.
40. Snoek C., Worring M., de Rooij O., van de Sande K., Yan R., and Hauptmann A. VideOlympics: real-time evaluation of multimedia retrieval systems. IEEE MultiMedia 15 1 (2008) 86-91
41. C. Snoek, M. Worring, J. van Gemert, J. Geusebroek, A. Smeulders, The challenge problem for automated detection of 101 semantic concepts in multimedia, in: Proc. Int. Conf. on Multimedia, October 2006, pp. 225-226.
42. Tamura H., Mori S., and Yamawaki T. Textural features corresponding to visual perception. IEEE Trans. Syst. Man Cybern. 8 6 (1978) 460-472
43. A. Ulges, C. Schulze, D. Keysers, T. Breuel, Content-based video tagging for online video portals, in: Proc. MUSCLE/ImageCLEF Workshop, September 2007.
44. A. Ulges, C. Schulze, D. Keysers, T. Breuel, A system that learns to tag videos by watching YouTube, in: Proc. Int. Conf. on Vision Systems, May 2008, pp. 415-424.
45. A. Ulges, C. Schulze, D. Keysers, T. Breuel, Identifying relevant frames in weakly labeled videos for training concept detectors, in: Proc. Int. Conf. Image and Video Retrieval, July 2008, pp. 9-16.
46. YouTube Serves up 100 Million Videos a Day Online, USA Today (Garnnett Company, Inc.), July 2006. %3chttp://www.usatoday.com/tech/news/2006-07-16-youtube-views_x.htm%3e (retrieved September 2008).
47. K. van de Sande, T. Gevers, C. Snoek, A comparison of color features for visual concept classification, in: Proc. Int. Conf. Image and Video Retrieval, July 2008, pp. 141-150.
48. D. Wang, X. Liu, L. Luo, J. Li, B. Zhang, Video diver: generic video indexing with diverse features, in: Proc. Int. Workshop Multimedia Information Retrieval, September 2007, pp. 61-70.
49. A. Yanagawa, S.-F. Chang, L. Kennedy, W. Hsu, Columbia University's baseline detectors for 374 LSCOM semantic visual concepts, Technical Report, Columbia University, 2007.
50. A. Yanagawa, W. Hsu, S.-F. Chang, Brief descriptions of visual features for baseline TRECVID concept detectors, Technical Report, Columbia University, 2007.
51. J. Yang, A. Hauptmann, (Un)reliability of video concept detection, in: Proc. Int. Conf. Image and Video Retrieval, July 2008, pp. 85-94.
52. J. Yang, R. Yan, A. Hauptmann, Cross-domain video concept detection using adaptive SVMs, in: Proc. Int. Conf. on Multimedia, September 2007, pp. 188-197.
53. L. Zelnik-Manor, S. Zanetti, P. Perona, A walk through the web's video clips, in: Proc. First Internet Vision Workshop, June 2008, pp. 1-7.
54. Zhang J., Marszalek M., Lazebnik S., and Schmid C. Local features and kernels for classification of texture and object categories: a comprehensive study. Int. J. Comput. Vis. 73 2 (2007) 213-238