Automatic detection of salient objects and spatial relations in videos for a video database system

Image and Vision Computing

Volumn 26, Issue 10, 2008, Pages 1384-1396

  Sevilmiş, Tarkan ^a   Baştan, Muhammet ^a   Güdükbay, Uǧur ^a   Ulusoy, Özgür ^a  

^a BILKENT UNIVERSITY   (Turkey)

Author keywords

Camera focus estimation; Knowledge base construction; Multimedia databases; Object labeling; Salient object detection and tracking; Spatio temporal queries

Indexed keywords

ADMINISTRATIVE DATA PROCESSING; DATA PROCESSING; DATA STORAGE EQUIPMENT; FEATURE EXTRACTION; LABELING; LABELS; MANAGEMENT INFORMATION SYSTEMS; VIDEO RECORDING; VIDEO SIGNAL PROCESSING;

AUTOMATIC DETECTION; CAMERA FOCUS ESTIMATION; KNOWLEDGE BASED; KNOWLEDGE-BASE CONSTRUCTION; MULTIMEDIA DATA; MULTIMEDIA DATABASES; OBJECT LABELING; SALIENT OBJECT DETECTION AND TRACKING; SALIENT OBJECT DETECTIONS; SALIENT OBJECTS; SCALABLE ARCHITECTURES; SHOT BOUNDARY DETECTION; SPATIAL RELATIONS; SPATIO-TEMPORAL; SPATIO-TEMPORAL QUERIES; SPATIO-TEMPORAL RELATIONS; VIDEO DATABASE MANAGEMENT; VIDEO DATABASES; VIDEO PROCESSING;

DATABASE SYSTEMS;

CAMERAS; DATA PROCESSING; DATA STORAGE; EXTRACTION; INDEXING; INFORMATION RETRIEVAL; INFORMATION SYSTEMS; LABELING; LABELS; MANAGEMENT; STORAGE;

EID: 49349097727     PISSN: 02628856     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.imavis.2008.01.001     Document Type: Article

References (43)

1. Marcus S., and Subrahmanian V.S. Foundations of multimedia database systems. Journal of the ACM 43 3 (1996) 474-523
2. Flickner M., Sawhney H., Niblack W., Ashley J., Huang Q., Dom B., Gorkani M., Hafner J., Lee D., Petkovic D., Steele D., and Yanker P. Query by image and video content: the QBIC system. IEEE Computer 28 9 (1995) 23-32
3. J.R. Smith, S.F. Chang, VisualSEEk: a fully automated content-based image query system, in: ACM Multimedia, 1996, pp. 87-98. Available from: .
4. S.F. Chang, W. Chen, H.J. Meng, H. Sundaram, D. Zhong, VideoQ: an automated content based video search system using visual cues, in: ACM Multimedia, 1997, pp. 313-324. Available from: .
5. Saykol E., Güdükbay U., and Ulusoy O. A histogram-based approach for object-based query-by-shape-and-color in image and video databases. Image and Vision Computing 23 13 (2005) 170-1180
6. J. Ashley, R. Barber, M. Flickner, J. Hafner, D. Lee, W. Niblack, D. Petkovic, Automatic and semiautomatic methods for image annotation and retrieval in query by image content (QBIC), in: Wayne Niblack, Ramesh C. Jain, (Eds.), Proceedings of the SPIE Storage and Retrieval for Image and Video Databases III, vol. 2420, March 1995, pp. 24-35.
7. Marques O., and Barman N. Semi-automatic semantic annotation of images using machine learning techniques. Lecture Notes in Computer Science vol. 2870 (2003), Springer-Verlag 550-565
8. Gu C., and Lee M.-C. Semiautomatic segmentation and tracking of semantic video objects. IEEE Transactions on Circuits and Systems for Video Technology 8 5 (1998) 572-584
9. Dönderler M.E., Şaykol E., Arslan U., Ulusoy O., and Güdükbay U. Bilvideo: design and implementation of a video database management system. Multimedia Tools and Applications 27 (2005) 79-104
10. Dönderler M.E., Ulusoy O., and Güdükbay U. A rule-based video database system architecture. Information Sciences 143 (2002) 13-45
11. J.S. Boreczky, L.A. Rowe, Comparison of video shot boundary detection techniques, in: Proceedings of the SPIE Storage and Retrieval for Image and Video Databases IV, vol. 2670, San Jose, California, USA, 1996, pp. 170-179.
12. Lienhart R. Reliable transition detection in videos: a survey and practitioner's guide. International Journal of Image and Graphics (IJIG) 1 3 (2001) 469-486
13. A.F. Smeaton, P. Over, W. Kraaij, Evaluation campaigns and TRECVid, in: MIR'06: Proceedings of the Eighth ACM International Workshop on Multimedia Information Retrieval, New York, NY, USA, 2006, pp. 321-330.
14. F. Porikli, Y. Wang, Automatic video object segmentation using volume growing and hierarchical clustering, Journal of Applied Signal Processing, special issue on Object-Based and Semantic Image and Video Analysis, July 2004. Available from: .
15. Deng Y., and Manjunath B. Unsupervised segmentation of color-texture regions in images and video. IEEE Transactions on Pattern Analysis and Machine Intelligence 23 8 (2001) 800-810. Available from:
16. Carson C., Belongie S., Greenspan H., and Malik J. Blobworld: image segmentation using expectation-maximization and its application to image querying. IEEE Transactions on Pattern Analysis and Machine Intelligence 24 8 (2002) 1026-1038
17. Ma W., and Manjunath B. Edgeflow: a technique for boundary detection and segmentation. IEEE Transactions on Image Processing 9 8 (2000) 1375-1388. Available from:
18. Shi J., and Malik J. Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 22 8 (2000) 888-905. Available from:
19. Felzenszwalb P.F., and Huttenlocher D. Efficient graph-based image segmentation. International Journal of Computer Vision 59 2 (2004) 167-181. Available from:
20. Zhang D., and Lu G. Segmentation of moving objects in image sequences: a review. Circuits, Systems, and Signal Processing 20 2 (2001) 143-183
21. Moscheni F., Bhattacharjee S., and Kunt M. Spatio-temporal segmentation based on region merging. IEEE Transactions on Pattern Analysis and Machine Intelligence 20 9 (1998) 897-915
22. L. Li, W. Huang, I.Y. Gu, Q. Tian, Foreground object detection from videos containing complex background, in: Proceedings of the Eleventh ACM International Conference on Multimedia, New York, NY, USA, 2003, pp. 2-10.
23. S.M. Desa, Q.A. Salih, Image subtraction for real time moving object extraction, in: International Conference on Computer Graphics, Imaging and Visualization, 2004, pp. 41-45.
24. K. Ryan, A. Amer, L. Gagnon, Video object segmentation based on object enhancement and region merging, in: Proceedings of the IEEE International Conference on Multimedia and Expo (ICME), 2006, pp. 273-276.
25. J. Hsieh, J. Lee, Video object segmentation using kernel-based models and spatiotemporal similarity, in: Proceedings of IEEE International Conference on Image Processing (ICIP), 2006, pp. 1821-1824.
26. G. Zhang, W. Zhu, Automatic video object segmentation by integrating object registration and background constructing technology, in: Proceedings of the International Conference on Communications, Circuits and Systems, June 2006, pp. 437-441.
27. Comaniciu D., Ramesh V., and Meer P. Kernel-based object tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence 25 5 (2003) 564-577
28. A. Cavallaro, O. Steiger, T. Ebrahimi, Multiple video object tracking in complex scenes, in: Proceedings of the Tenth ACM International Conference on Multimedia, New York, NY, USA, 2002, pp. 523-532.
29. Kim C., and Hwang J. Fast and automatic video object segmentation and tracking for content-based applications. IEEE Transactions on Circuits and Systems for Video Technology 12 2 (2002) 122-129
30. Xu L., Landabaso J., and Lei B. Segmentation and tracking of multiple moving objects for intelligent video analysis. BT Technology Journal 22 3 (2004) 140-150
31. J. Shao, R. Chellappa, F. Porikli, Shape-regulated particle filtering for tracking non-rigid objects, in: Proceedings of the International Conference on Image Processing (ICIP), August 2006, pp. 2813-2816. Available from: .
32. Itti L., Koch C., and Niebur E. A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence 20 11 (1998) 1254-1259
33. Y. Ma, H. Zhang, Contrast-based image attention analysis by using fuzzy growing, in: Proceedings of the Eleventh ACM International Conference on Multimedia, 2003, pp. 374-381.
34. S. Kwak, B. Ko, H. Byun, Automatic salient-object extraction using the contrast map and salient points, in: Advances in Multimedia Information Processing, Lecture Notes in Computer Science (LNCS), vol. 3332, 2004, pp. 138-145.
35. U. Rutishauser, D. Walther, C. Koch, P. Perona, Is bottom-up attention useful for object recognition?, in: Proceedings of IEEE Computer Society Conference on Computer and Vision Pattern Recognition (CVPR), vol. 2, July 2004, pp. 37-44.
36. F. Ge, S. Wang, T. Liu, Image-segmentation evaluation from the perspective of salient object extraction, in: Proceedings of IEEE Computer Society Conference on Computer and Vision Pattern Recognition (CVPR), vol. I, 2006, pp. 1146-1153.
37. Chalmond B., Francesconi B., and Herbin S. Using hidden scale for salient object detection. IEEE Transactions on Image Processing 15 9 (2006) 2644-2656
38. T. Liu, J. Sun, N.N. Zheng, X. Tang, H. Shum, Learning to detect a salient object, in: Proceedings of IEEE Computer Society Conference on Computer and Vision Pattern Recognition (CVPR), June 2007, pp. 1-8.
39. C. Tomasi, R. Manduchi, Bilateral filtering for gray and color images, in: Proceedings of IEEE International Conference on Computer Vision, Bombay, India, 1998, pp. 839-846.
40. Open Source Computer Vision Library. Available from:.
41. P.A. Viola, M.J. Jones, Rapid object detection using a boosted cascade of simple features, in: Proceedings of IEEE Computer Society Conference on Computer and Vision Pattern Recognition (CVPR), vol. 1, 2001, pp. I-511-I-518.
42. FFmpeg Library. Available from: .
43. Saliency Toolbox. Available from: .