Saliency model-based face segmentation and tracking in head-and-shoulder video sequences

Journal of Visual Communication and Image Representation

Volumn 19, Issue 5, 2008, Pages 320-333

  Li, Hongliang ^a   Ngan, King N ^a  

^a CHINESE UNIVERSITY OF HONG KONG   (Hong Kong)

Author keywords

Boundary saliency map; Face segmentation; Face tracking; Facial feature extraction; Facial saliency map; Head and shoulder video; Skin tone; Video segmentation; Video conferencing; Visual attention

Indexed keywords

FACE RECOGNITION; FEATURE EXTRACTION; IMAGE SEGMENTATION; VIDEO CONFERENCING;

BOUNDARY SALIENCY MAP; FACIAL SALIENCY MAP; VIDEO SEGMENTATION;

VIDEO STREAMING;

EID: 44449128885     PISSN: 10473203     EISSN: 10959076     Source Type: Journal    
DOI: 10.1016/j.jvcir.2008.04.001     Document Type: Article

References (52)

1. Meier T., and Ngan K.N. Video segmentation for content-based coding. IEEE Trans. Circ. Syst. Video Technol. 9 8 (1999) 1190-1203
2. Cavallaro A., Steiger O., and Ebrahimi T. Tracking video objects in cluttered background. IEEE Trans. Circ. Syst. Video Technol. 15 4 (2005) 575-584
3. Chai D., and Ngan K.N. Face segmentation using skin-color map in videophone application. IEEE Trans. Circ. Syst. Video Technol. 9 4 (1999) 551-564
4. Garcia C., and Tziritas G. Face detection using quantized skin color regions merging and wavelet packet analysis. IEEE Trans. Multimedia 1 3 (1999) 264-277
5. Hsu R.L., Mottaleb M.A., and Jain A.K. Face detection in color images. IEEE Trans. Pattern Anal. Mach. Intell. 24 5 (2002) 696-706
6. Lievin M., and Luthon F. Nonlinear color space and spatiotemporal MRF for hierarchical segmentation of face features in video. IEEE Trans. Image Process. 13 1 (2004) 63-71
7. Habili N., Lim C.C., and Moini A. Segmentation of the face and hands in sign language video sequences using color and motion cues. IEEE Trans. Circ. Syst. Video Technol. 14 8 (2004) 1086-1097
8. Phung S.L., Bouzerdoum A., and Chai D. Skin segmentation using color pixel classification: analysis and comparison. IEEE Trans. Pattern Anal. Mach. Intell. 27 1 (2005) 148-154
9. Jones M.J., and Rehg J.M. Statistical color model with application to skin detection. Int. J. Comput. Vis. 46 1 (2002) 81-96
10. Greespan H., Goldberger J., and Eshet I. Mixture model for face color modeling and segmentation. Pattern Recogn. Lett. 22 (2001) 1525-1536
11. Menser B., and Wien M. Segmentation and Tracking of Facial Regions in Color Image Sequences. SPIE Vis. Commun. Image Process. 4067 (2000) 731-740
12. Yang M.-H., and Ahuja N. Detecting human faces in color images. IEEE ICIP 98 1 (1998) 127-130
13. J.-C. Terrillon, M. David, S. Akamatsu, Detection of human faces in complex scene images by use of a skin color model and of invariant Fourier-Mellin moments, in: Proc. Fourteenth International Conference on Pattern Recognition, vol. 2, 1998, pp. 1350-1355.
14. Wang H., and Chang S.-F. A highly efficient system for automatic face region detection in MPEG video. IEEE Trans. Circ. Syst. Video Technol. 7 4 (1997) 615-628
15. Chen C., and Chiang S.-P. Detection of human faces in colour images. IEE Proc.: Vis. Image Signal Process. 144 6 (1997) 384-388
16. S. Kawato, J. Ohya, Automatic skin-color distribution extraction for face detection and tracking, in: Proc. WCCC-ICSP 2000. 5th International Conference on Signal Processing Proceedings, vol. 2, 2000, pp. 1415-1418.
17. J. Park, J. Seo, D. An, S. Chung, Detection of human faces using skin color and eyes, in: Proc. IEEE International Conference on Multimedia and Expo, vol. 1, 2000, pp. 133-136.
18. S.L. Phung, D. Chai, A. Bouzerdoum, Skin colour based face detection, in: Proc. Seventh Australian and New Zealand Intelligent Information Systems Conference, 2001, pp. 171-176.
19. T.S. Caetano, D.A.C. Barone, A probabilistic model for the human skin color, in: Proc. 11th International Conference on Image Analysis and Processing, 2001, pp. 279-283.
20. J. Fritsch, S. Lang, A. Kleinehagenbrock, G.A. Fink, G. Sagerer, Improving adaptive skin color segmentation by incorporating results from face detection, in: Proc. 11th IEEE International Workshop on Robot and Human Interactive Communication, 2002, pp. 337-343.
21. T.S. Caetano, S.D. Olabarriaga, D.A.C. Barone, Performance evaluation of single and multiple-Gaussian models for skin color modeling, in: Proc. XV Brazilian Symposium on Computer Graphics and Image Processing, 2002, pp. 275-282.
22. J. Yang, Z. Fu, T. Tan, W. Hu, Skin color detection using multiple cues, in: Proc. 17th International Conference on Pattern Recognition, vol. 1, 2004, pp. 632-635.
23. Verma R.C., Schmid C., and Mikolajczyk K. Face detection and tracking in a video by propagating detection probabilities. IEEE Trans. Pattern Anal. Mach. Intell. 25 10 (2003) 1215-1228
24. Liu C. A Bayesian discriminating features method for face detection. IEEE Trans. Pattern Anal. Mach. Intell. 25 6 (2003) 725-740
25. Li S.Z., Lu X., Hou X., Peng X., and Cheng Q. Learning multiview face subspaces and facial pose estimation using independent component analysis. IEEE Trans. Image Process. 14 6 (2005) 705-712
26. Rajagopalan A.N., Chellappa R., and Koterba N.T. Background learning for robust face recognition with PCA in the presence of clutter. IEEE Trans. Image Process. 14 6 (2005) 832-843
27. K. Sobottka, I. Pitas, A fully automatic approach to facial feature detection and tracking, in: Proc. of 1st Int. Conf. on Audio- and Video-based Biometric Person Authentication (AVBPA'97), Crans-Montana, Switzerland, 1997, pp. 77-84.
28. Lam K.-M., and Yan H. An analytic-to-holistic approach for face recognition based on a single frontal view. IEEE Trans. Pattern Anal. Mach. Intell. 20 7 (1998) 673-686
29. Yan S., He X., Hu Y., Zhang H., Li M., and Cheng Q. Bayesian shape localization for face recognition using global and local textures. IEEE Trans. Circ. Syst. Video Technol. 14 1 (2004) 102-113
30. Luo H., and Eleftheriadis A. Model-based segmentation and tracking of head-and-shoulder video objects for real time multimedia services. IEEE Trans. Multimedia 5 3 (2003) 379-389
31. Criminisi A., Cross G., Blake A., and Kolmogorov V. Bilayer segmentation of live video. Proc. CVPR (2006) 53C60
32. Levner I., and Zhang H. Classification-driven watershed segmentation. IEEE Trans. Image Process. 16 5 (2007) 1437-1445
33. Cooper M., Liu T., and Rieffel E. Video segmentation via temporal pattern classification. IEEE Trans. Multimedia 9 3 (2007) 610-618
34. Grady L. Random walks for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 28 11 (2007) 1768-1783
35. Yin P., Criminisi A., Winn J., and Essa I. Tree-based classifiers for bilayer video segmentation. Proc. CVPR (2007)
36. Y. Boykov, M.-P. Jolly, Interactive graph cuts for optimal boundary & region segmentation of objects in n-d images, in: Proceedings of ICCV, vol. 1, 2001, pp. 105-112.
37. Itti L., Koch C., and Niebur E. A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell. 20 11 (1998) 1254-1259
38. Ma Y.-F., and Zhang H.-J. A model of motion attention for video skimming. IEEE ICIP '02 1 22-25 (2002) I-129-I-132
39. Han J., Ngan K.N., Li M., and Zhang H.-J. Towards unsupervised attention object extraction by integrating visual attention and object growing. IEEE ICIP '04 2 24-27 (2004) 941-944
40. Lu Z., Lin W., Yang X., Ong E., and Yao S. Modeling visual attentions modulatory aftereffects on visual sensitivity and quality evaluation. IEEE Trans. Image Process. 14 11 (2005) 1928-1942
41. CVL FACE DATABASE. Available from: .
42. Hsia S.C., and Tsai P.S. Efficient light balancing techniques for text images in video presentation systems. IEEE Trans. Circ. Syst. Video Technol. 15 8 (2005) 1026-1031
43. Jackway P.T., and Deriche M. Scale-space properties of the multiscale morphological dilation-erosion. IEEE Trans. Pattern Anal. Mach. Intell. 18 1 (1996) 38-51
44. Canny J. A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 9 6 (1986) 679-698
45. B. Martinkauppi, M. Soriano, M. Pietikainen, Detection of skin color under changing illumination: a comparative study, in: Proceedings of the 12th IEEE International Conference on Image Analysis and Processing, 2003, pp. 652-657.
46. Verma R.C., Schmid C., and Mikolajczyk K. Face detection and tracking in a video by propagating detection probabilities. IEEE Trans. Pattern Anal. Mach. Intell. 25 10 (2003) 1215-1228
47. Froba B., and Kublbeck C. Face tracking by means of continuous detection. Proc. CVPR (2004) 65-71
48. Scott V.D., Yin L., Ko M.J., and Hung T. Multiple-view face tracking for modeling and analysis based on non-cooperative video imagery. Proc. CVPR (2007)
49. Li Y., Ai H., Takayoshi Y., Lao S., and Masato K. Tracking in low frame rate video: a cascade particle filter with discriminative observers of different lifespans. Proc. CVPR (2007) 1-8
50. D. Martin, C. Fowlkes, D. Tal, J. Malik, A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics, in: Proc. 8th Int'l Conf. Computer Vision, vol. 2, 2001, pp. 416-423.
51. Viola P., and Jones M.J. Robust real-time face detection. Int. J. Comput. Vis. 57 2 (2004) 137-154
52. C. Rother, V. Kolmogorov, A. Blake, GrabCut: interactive foreground extraction using iterated graph cuts, in: Proceedings of SIGGRAPH, 2004.