Transformation of an uncertain video search pipeline to a sketch-based visual analytics loop

IEEE Transactions on Visualization and Computer Graphics

Volumn 19, Issue 12, 2013, Pages 2109-2118

  Legg, Philip A ^a,b   Chung, David H S ^a   Parry, Matthew L ^a   Bown, Rhodri ^c   Jones, Mark W ^a   Griffiths, Iwan W ^a   Chen, Min ^b  

^a SWANSEA UNIVERSITY   (United Kingdom)

^b UNIVERSITY OF OXFORD   (United Kingdom)

^c WELSH RUGBY UNION   (United Kingdom)

Author keywords

data clustering; machine learning; multimedia visualization; visual knowledge discovery

Indexed keywords

ACTIVE MACHINE LEARNING; AGGREGATED INFORMATION; DATA CLUSTERING; MULTIMEDIA VISUALIZATION; SKETCH-BASED INTERFACES; TRADITIONAL APPROACHES; VISUAL ANALYTICS SYSTEMS; VISUAL KNOWLEDGE DISCOVERY;

CLUSTERING ALGORITHMS; LEARNING SYSTEMS; SEMANTICS; VISUALIZATION;

SEARCH ENGINES;

ALGORITHM; ARTICLE; COMPUTER ASSISTED DIAGNOSIS; COMPUTER GRAPHICS; COMPUTER INTERFACE; INFORMATION RETRIEVAL; METHODOLOGY; REPRODUCIBILITY; SENSITIVITY AND SPECIFICITY; VIDEORECORDING; PROCEDURES;

ALGORITHMS; COMPUTER GRAPHICS; IMAGE INTERPRETATION, COMPUTER-ASSISTED; INFORMATION STORAGE AND RETRIEVAL; REPRODUCIBILITY OF RESULTS; SENSITIVITY AND SPECIFICITY; USER-COMPUTER INTERFACE; VIDEO RECORDING;

EID: 84886680981     PISSN: 10772626     EISSN: None     Source Type: Journal    
DOI: 10.1109/TVCG.2013.207     Document Type: Article

References (42)

1. M. Ankerst, S. Berchtold, and D. Keim. Similarity clustering of dimensions for an enhanced visualization of multidimensional data. In Information Visualization, 1998. Proceedings. IEEE Symposium on, pages 52-60, 153, 1998.
2. R. A. Baeza-Yates and B. Ribeiro-Neto. Modern Information Retrieval. Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA, 1999.
3. S. Baluja, R. Seth, D. Sivakumar, Y. Jing, J. Yagnik, S. Kumar, D. Ravichandran, and M. Aly. Video suggestion and discovery for youtube: taking random walks through the view graph. In Proceedings of the 17th international conference on World Wide Web, WWW '08, pages 895-904, New York, NY, USA, 2008. ACM.
4. J. Blaas, C. Botha, and F. Post. Extensions of parallel coordinates for interactive exploration of large multi-timepoint data sets. Visualization and Computer Graphics, IEEE Transactions on, 14(6):1436-1451, 2008.
5. R. Borgo, M. Chen, B. Daubney, E. Grundy, G. Heidemann, B. Hoferlin, M. Hoferlin, H. Leitte, D. Weiskopf, and X. Xie. State of the art report on video-based graphics and video visualization. Computer Graphics Forum, 31(8):2450-2477, 2012.
6. R. Botchen, F. Schick, and T. Ertl. Action-based multifield video visualization. Visualization and Computer Graphics, IEEE Transactions on, 14(4):885-899, july-aug. 2008. (Pubitemid 351780214)
7. I. Bowman, S. Joshi, and J. Van Horn. Query-based coordinated multiple views with feature similarity space for visual analysis of mri repositories. In Visual Analytics Science and Technology (VAST), 2011 IEEE Conference on, pages 267-268, 2011.
8. S.-F. Chang, W. Chen, H. J. Meng, H. Sundaram, and D. Zhong. Videoq: an automated content based video search system using visual cues. In Proceedings of the fifth ACM international conference on Multimedia, MULTIMEDIA '97, pages 313-324, New York, NY, USA, 1997. ACM.
9. S.-F. Chang, W. Chen, H. J. Meng, H. Sundaram, and D. Zhong. A fully automated content-based video search engine supporting spatiotemporal queries. IEEE Transactions on Circuits and Systems for Video Technology, 8:602-615, 1998. (Pubitemid 128743727)
10. S.-F. Chang, W.-Y. Ma, and A. Smeulders. Recent advances and challenges of semantic image/video search. In Acoustics, Speech and Signal Processing, 2007. ICASSP 2007. IEEE International Conference on, April 2007.
11. M. Chen, R. Hashim, R. Botchen, D. Weiskopf, T. Ertl, and I. Thornton. Visual signatures in video visualization. Visualization and Computer Graphics, IEEE Transactions on, 12(5):1093-1100, sept.-oct. 2006. (Pubitemid 44955642)
12. W. Chen and S.-F. Chang. Vismap: an interactive image/video retrieval system using visualization and concept maps. In ICIP (3), pages 588-591, 2001. (Pubitemid 33058055)
13. G. Daniel and M. Chen. Video visualization. In Visualization, 2003. VIS 2003. IEEE, pages 409-416, oct. 2003.
14. A. Dasgupta and R. Kosara. Pargnostics: Screen-space metrics for parallel coordinates. Visualization and Computer Graphics, IEEE Transactions on, 16(6):1017-1026, 2010.
15. Y. Deng and B. S. Manjunath. Content-based search of video using color, texture, and motion. In Proceedings. International Conference on Image Processing (cat. No.97cb36144). IEEE Signal Process. Soc, IEEE Signal Process. Soc, Jan 1997.
16. N. Dimitrova, H.-J. Zhang, B. Shahraray, I. Sezan, T. Huang, and A. Zakhor. Applications of video-content analysis and retrieval. IEEE Multi-Media, 9(3):42-55, July 2002.
17. G. Ellis and A. Dix. Enabling automatic clutter reduction in parallel coordinate plots. IEEE Transactions on Visualization and Computer Graphics, 12(5):717-724, Sept. 2006.
18. M. Hao, U. Dayal, D. Keim, D. Morent, and J. Schneidewind. Intelligent visual analytics queries. In Visual Analytics Science and Technology, 2007. VAST 2007. IEEE Symposium on, pages 91-98, 2007.
19. A. G. Hauptmann, W.-H. Lin, R. Yan, J. Yang, and M.-Y. Chen. Extreme video retrieval: joint maximization of human and computer performance. In Proceedings of the 14th annual ACM international conference on Multimedia, MULTIMEDIA '06, pages 385-394, New York, NY, USA, 2006. ACM. (Pubitemid 47113554)
20. M. Hoferlin, B. Hoferlin, G. Heidemann, and D. Weiskopf. Interactive schematic summaries for faceted exploration of surveillance video. Multimedia, IEEE Transactions on, PP(99):1, 2013.
21. W. H. Hsu, L. S. Kennedy, and S.-F. Chang. Reranking methods for visual search. IEEE MultiMedia, 14(3):14-22, July 2007. (Pubitemid 47322511)
22. M. D. Hughes and I. M. Franks. Notational analysis of sport. London: E. & F.N. Spon., 1997.
23. J. Lee and T. Funkhouser. Sketch-based search and composition of 3D models. In EUROGRAPHICS Workshop on Sketch-Based Interfaces and Modeling, June 2008.
24. P. A. Legg, D. H. S. Chung, M. L. Parry, M. W. Jones, R. Long, I. W. Griffiths, and M. Chen. Matchpad: Interactive glyph-based visualization for real-time sports performance analysis. Computer Graphics Forum, 31(3pt4):1255-1264, 2012.
25. Y. Liu, T. Mei, X.-S. Hua, J. Tang, X. Wu, and S. Li. Learning to video search rerank via pseudo preference feedback. In Multimedia and Expo, 2008 IEEE International Conference on, pages 297-300, 2008.
26. T. McLoughlin, M. Jones, R. Laramee, R. Malki, I. Masters, and C. Hansen. Similarity measures for enhancing interactive streamline seeding. Visualization and Computer Graphics, IEEE Transactions on, PP(99):1-1, 2012.
27. M. Parry, P. Legg, D. Chung, I. Griffiths, and M. Chen. Hierarchical event selection for video storyboards with a case study on snooker video visualization. Visualization and Computer Graphics, IEEE Transactions on, 17(12):1747-1756, dec. 2011.
28. C. J. V. Rijsbergen. Information Retrieval. Butterworth-Heinemann, Newton, MA, USA, 2nd edition, 1979.
29. J. Rodrigues, L. Romani, A. J. M. Traina, and C. Traina. Combining visual analytics and content based data retrieval technology for efficient data analysis. In Information Visualisation (IV), 2010 14th International Conference, pages 61-67, 2010.
30. K. Schoeffmann, F. Hopfgartner, O. Marques, L. Boeszoermenyi, and J. M. Jose. Video browsing interfaces and applications: a review. Journal of Photonics for Energy, pages 018004-018004-35, 2010.
31. J. Sivic. Efficient Visual Search of Images and Videos. PhD thesis, University of Oxford, 2006.
32. J. Sivic and A. Zisserman. Video Google: Efficient visual search of videos. In J. Ponce, M. Hebert, C. Schmid, and A. Zisserman, editors, Toward Category-Level Object Recognition, volume 4170 of LNCS, pages 127-144. Springer, 2006.
33. J. Sivic and A. Zisserman. Efficient visual search for objects in videos. Proceedings of the IEEE, 96(4):548-566, 2008.
34. C. G. M. Snoek, B. Huurnink, L. Hollink, M. de Rijke, G. Schreiber, and M. Worring. Adding semantics to detectors for video retrieval. Multimedia, IEEE Transactions on, Aug. 2007.
35. C. G. M. Snoek, M. Worring, J. van Gemert, J.-M. Geusebroek, D. Koelma, G. P. Nguyen, O. de Rooij, and F. Seinstra. Mediamill: exploring news video archives based on learned semantics. In Proceedings of the 13th annual ACM international conference on Multimedia, MULTIMEDIA '05, pages 225-226, New York, NY, USA, 2005. ACM.
36. A. Tatu, G. Albuquerque, M. Eisemann, J. Schneidewind, H. Theisel, M. Magnor, and D. Keim. Combining automated analysis and visualization techniques for effective exploration of high-dimensional data. In Visual Analytics Science and Technology, 2009. VAST 2009. IEEE Symposium on, pages 59-66, 2009.
37. J. Tesíc, A. P. Natsev, and J. R. Smith. Cluster-based data modeling for semantic video search. In Proceedings of the 6th ACM international conference on Image and video retrieval, CIVR '07, pages 595-602, New York, NY, USA, 2007. ACM. (Pubitemid 350229677)
38. S. Vrochidis, I. Kompatsiaris, and I. Patras. Optimizing visual search with implicit user feedback in interactive video retrieval. In Proceedings of the ACM International Conference on Image and Video Retrieval, CIVR '10, pages 274-281, New York, NY, USA, 2010. ACM.
39. X. Wang, C. Zhang, and Z. Zhang. Boosted multi-task learning for face verification with applications to web image and video search. In Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, pages 142-149, June 2009.
40. L. Zhang, F. Lin, and B. Zhang. Support vector machine learning for image retrieval. In Image Processing, 2001. Proceedings. 2001 International Conference on, volume 2, pages 721-724 vol.2, Oct.
41. D. Zhong and S.-F. Chang. Spatio-temporal video search using the object based video representation. In Proceedings of the 1997 International Conference on Image Processing (ICIP '97) 3-Volume Set-Volume 1-Volume 1, ICIP '97, pages 21-, Washington, DC, USA, 1997. IEEE Computer Society.
42. D. Zhong and S. fu Chang. Structure analysis of sports video using domain models. IEEE ICME, pages 22-25, 2001.