Leveraging multiple cues for recognizing family photos

Image and Vision Computing

Volumn 58, Issue , 2017, Pages 61-75

  Wang, Xiaolong ^a   Guo, Guodong ^b   Merler, Michele ^c   C F Codella, Noel ^c   MV, Rohith ^a   Smith, John R ^c   Kambhamettu, Chandra ^a  

^a UNIVERSITY OF DELAWARE   (United States)

^b WEST VIRGINIA UNIVERSITY   (United States)

^c IBM T J WATSON RESEARCH CENTER   (United States)

Author keywords

Family photo recognition; Group photo analysis; Semantics; Social media

Indexed keywords

CONTENT BASED RETRIEVAL; SEMANTICS; SOCIAL NETWORKING (ONLINE);

CLASSIFICATION ACCURACY; FACIAL APPEARANCE; FAMILY PHOTO RECOGNITION; INDIVIDUAL MODELING; PHOTO ANALYSIS; SEMANTIC FEATURES; SOCIAL MEDIA; SOCIAL RELATIONS;

IMAGE ANALYSIS;

EID: 84998885619     PISSN: 02628856     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.imavis.2016.07.006     Document Type: Article

References (70)

1. [1] Li, D., Chuah, M.C., EMOD: an efficient on-device mobile visual search system. Proceedings of the 6th ACM Multimedia Systems Conference, 2015.
2. [2] Wang, G., Gallagher, A., Luo, J., Forsyth, D., Seeing people in social context: recognizing people and social relationships. ECCV, 2010, 169–182.
3. [3] Chen, Y.-Y., Hsu, W.H., Liao, H.-Y.M., Discovering informative social subgraphs and predicting pairwise relationships from group photos. Proceedings of the 20th ACM International Conference on Multimedia, 2012, 669–678.
4. [4] Wang, X., Guo, G., Rohith, M., Kambhamettu, C., Leveraging geometry and appearance cues for recognizing family photos. 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), 2015, 1–8.
5. [5] Ng, H.-W., Winkler, S., A data-driven approach to cleaning large face datasets. 2014 IEEE International Conference on Image Processing (ICIP), 2014, 343–347.
6. [6] Gallagher, A.C., Chen, T., Understanding images of groups of people. CVPR, 2009.
7. [7] Singla, P., Kautz, H., Luo, J., Gallagher, A., Discovery of social relationships in consumer photo collections using Markov logic. CVPRW, 2008, 1–7.
8. [8] Poon, H., Domingos, P., Sound and efficient inference with probabilistic and deterministic dependencies. AAAI, 2006.
9. [9] Burges, C.J., A tutorial on support vector machines for pattern recognition. Data Min. Knowl. Disc. 2:2 (1998), 121–167.
10. [10] Lu, G., Yan, Y., Sebe, N., Kambhamettu, C., Knowing where I am: exploiting multi-task learning for multi-view indoor image-based localization. British Machine Vision Conference (BMVC), 2014.
11. [11] Bosch, A., Zisserman, A., Munoz, X., Representing shape with a spatial pyramid kernel. Proceedings of the 6th ACM international Conference on Image and Video Retrieval, 2007, ACM, 401–408.
12. [12] Lowe, D.G., Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 60:2 (2004), 91–110.
13. [13] Huttenlocher, D.P., Klanderman, G.A., Rucklidge, W.J., Comparing images using the Hausdorff distance. PAMI 15:9 (1993), 850–863.
14. [14] Krizhevsky, A., Sutskever, I., Hinton, G.E., Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems, 2012, 1097–1105.
15. [15] Chang, H., Zhou, Y., Spellman, P., Parvin, B., Stacked predictive sparse coding for classification of distinct regions in tumor histopathology. Proceedings of the IEEE International Conference on Computer Vision, 2013.
16. [16] Zhou, Y., Chang, H., Barner, K., Spellman, P., Parvin, B., Classification of histology sections via multispectral convolutional sparse coding. 2014 IEEE Conference on Computer Vision and Pattern Recognition, 2014.
17. [17] Ren, S., He, K., Girshick, R., Sun, J., Faster R-CNN: towards real-time object detection with region proposal networks. Advances in Neural Information Processing Systems, 2015, 91–99.
18. [18] Long, J., Shelhamer, E., Darrell, T., Fully convolutional networks for semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, 3431–3440.
19. [19] Zhou, Y., Chang, H., Barner, K.E., Parvin, B., Nuclei segmentation via sparsity constrained convolutional regression. 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI), 2015.
20. [20] Lu, G., Sebe, N., Xu, C., Kambhamettu, C., Memory efficient large-scale image-based localization, Multimedia Tools and Applications. 2015.
21. [21] Lu, G., Ly, V., Kambhamettu, C., Structure-from-motion reconstruction based on weighted hamming descriptors. 2014 International Joint Conference on Neural Networks (IJCNN), 2014.
22. [22] Viola, P., Jones, M., Rapid object detection using a boosted cascade of simple features. CVPR, 1, 2001 I–511.
23. [23] Graham, R.L., Frances Yao, F., Finding the convex hull of a simple polygon. J. Algorithms 4:4 (1983), 324–331.
24. [24] Horvath, T., Marton, G., Risztics, P., Szirmay-Kalos, L., Ray coherence between a sphere and a convex polyhedron. Computer Graphics Forum, 11, 1992, Wiley Online Library, 163–172.
25. [25] Meeran, S., Share, A., Optimum path planning using convex hull and local search heuristic algorithms. Mechatronics 7:8 (1997), 737–756.
26. [26] Corney, J., Rea, H., Clark, D., Pritchard, J., Breaks, M., MacLeod, R., Coarse filters for shape matching. IEEE Comput. Graph. Appl. 22:3 (2002), 65–74.
27. [27] Łukaszewski, A., Szczepkowicz, A., Computer simulation of FIM images—the convex hull model. Vacuum 54:1 (1999), 67–71.
28. [28] Hershberger, J., Snoeyink, J., Cartographic line simplification and polygon CSG formulae in O (n log n) time. Comput. Geom. 11:3 (1998), 175–185.
29. [29] Burrus, C., Parks, T.W., DFT/FFT and Convolution Algorithms: Theory and Implementation. 1991, John Wiley & Sons, Inc.
30. [30] Van Loan, C., Computational Frameworks for the Fast Fourier Transform. 10, 1992, Siam.
31. [31] Peng, X., Zhang, S., Yang, Y., Metaxas, D.N., Piefa: personalized incremental and ensemble face alignment. Proceedings of the IEEE International Conference on Computer Vision, 2015, 3880–3888.
32. [32] Rui, Y., She, A.C., Huang, T.S., Modified Fourier descriptors for shape representation—a practical approach. Proc of First International Workshop on Image Databases and Multi Media Search, 1996, Citeseer, 22–23.
33. [33] Zahn, C.T., Roskies, R.Z., Fourier descriptors for plane closed curves. IEEE Trans. Comput. 100:3 (1972), 269–281.
34. [34] Belongie, S., Malik, J., Puzicha, J., Shape matching and object recognition using shape contexts. IEEE Trans. Pattern Anal. Mach. Intell. 24:4 (2002), 509–522.
35. [35] Xu, C., Wang, Y., Tan, T., Quan, L., Automatic 3D face recognition combining global geometric features with local shape variation information. Proceedings. Sixth IEEE international Conference on Automatic Face and Gesture Recognition, 2004, 2004, 308–313.
36. [36] Vezzetti, E., Marcolin, F., 3D human face description: landmarks measures and geometrical features. Image Vis. Comput. 30:10 (2012), 698–712.
37. [37] Ahonen, T., Hadid, A., Pietikainen, M., Face description with local binary patterns: application to face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 28:12 (2006), 2037–2041.
38. [38] Taigman, Y., Yang, M., Ranzato, M., Wolf, L., Deepface: closing the gap to human-level performance in face verification. 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014, 1701–1708.
39. [39] Sun, Y., Chen, Y., Wang, X., Tang, X., Deep learning face representation by joint identification-verification. Advances in Neural Information Processing Systems, 2014, 1988–1996.
40. [40] Schroff, F., Kalenichenko, D., Philbin, J., Facenet: a unified embedding for face recognition and clustering, CVPR. 2015.
41. [41] Wang, X., Guo, R., Kambhamettu, C., Deeply-learned feature for age estimation. 2015 IEEE Winter Conference on Applications of Computer Vision, 2015.
42. [42] Le Cun, B.B., Denker, J.S., Henderson, D., Howard, R.E., Hubbard, W., Jackel, L.D., Handwritten digit recognition with a back-propagation network. Advances in Neural Information Processing Systems, 1990, Citeseer.
43. [43] Huang, G.B., Ramesh, M., Berg, T., Learned-Miller, E., Labeled faces in the wild: a database for studying face recognition in unconstrained environments. 2007, Technical Report 07-49, University of Massachusetts, Amherst.
44. [44] Wolf, L., Hassner, T., Maoz, I., Face recognition in unconstrained videos with matched background similarity. 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2011, 529–534.
45. [45] Guo, G., Mu, G., Ricanek, K., Cross-age face recognition on a very large database: the performance versus age intervals and improvement using soft biometric traits. 2010 20th International Conference on Pattern Recognition (ICPR), 2010, 3392–3395.
46. [46] Li, Z., Park, U., Jain, A.K., A discriminative model for age invariant face recognition. IEEE Trans. Inf. Forensics Secur. 6:3 (2011), 1028–1037.
47. [47] Guo, R., Liu, L., Wang, W., Taalimi, A., Zhang, C., Qi, H., Deep tree-structured face: a unified representation for multi-task facial biometrics. 2016 IEEE Winter Conference on Applications of Computer Vision (WACV), 2016.
48. [48] Xia, S., Shao, M., Luo, J., Fu, Y., Understanding kin relationships in a photo. IEEE Trans. Multimedia 14:4 (2012), 1046–1056.
49. [49] Milborrow, S., Nicolls, F., Locating facial features with an extended active shape model. ECCV, 2008, 504–513.
50. [50] Nair, V., Hinton, G.E., Rectified linear units improve restricted Boltzmann machines. Proceedings of the 27th International Conference on Machine Learning (ICML-10), 2010, 807–814.
51. [51] Scherer, D., Müller, A., Behnke, S., Evaluation of pooling operations in convolutional architectures for object recognition. ICANN 2010, 2010, Springer, 92–101.
52. [52] Serre, T., Oliva, A., Poggio, T., A feedforward architecture accounts for rapid categorization. Proc. Natl. Acad. Sci. 104:15 (2007), 6424–6429.
53. [53] Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., Darrell, T., Caffe: convolutional architecture for fast feature embedding. Proceedings of the ACM International Conference on Multimedia, 2014, ACM, 675–678.
54. [54] Guo, G., Mu, G., Fu, Y., Huang, T.S., Human age estimation using bio-inspired features. CVPR, 2009, 112–119.
55. [55] Guo, G., Wang, X., A study on human age estimation under facial expression changes. CVPR, 2012.
56. [56] Wang, X., Ly, V., Lu, G., Kambhamettu, C., Can we minimize the influence due to gender and race in age estimation?. 2013 12th International Conference on Machine Learning and Applications (ICMLA), 2, 2013, 309–314.
57. [57] Alsabti, K., Ranka, S., Singh, V., An efficient parallel algorithm for high dimensional similarity join. Proceedings of the First Merged International and Symposium on Parallel and Distributed Processing 1998, Parallel Processing Symposium, 1998. IPPS/SPDP 1998., 1998, 556–560.
58. [58] Smith, J., Cao, L., Codella, N., Hill, M., Merler, M., Nguyen, Q., Pring, E., Uceda-Sosa, R., Massive-scale learning of image and video semantic concepts. IBM Journal of Research and Development 59:2/3 (2015), 7:1–7:13.
59. [59] Merler, M., Huang, B., Xie, L., Hua, G., Natsev, A., Semantic model vectors for complex video event recognition. Multimedia, IEEE Transactions on 14:1 (2012), 88–101.
60. [60] Vapnik, V., The Nature of Statistical Learning Theory. 1999.
61. [61] Chang, C.-C., Lin, C.-J., LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. (TIST), 2(3), 2011, 27.
62. [62] Jain, A., Ross, A., Nandakumar, K., Introduction to Biometrics. 2011, Springer.
63. [63] Fierrez-Aguilar, J., Ortega-Garcia, J., Gonzalez-Rodriguez, J., Fusion strategies in multimodal biometric verification. ICME, 3, 2003 III–5.
64. [64] Ayache, S., Quénot, G., Gensel, J., Classifier fusion for SVM-based multimedia semantic indexing. Advances in Information Retrieval, 2007, Springer, 494–504.
65. [65] Fang, R., Tang, K.D., Snavely, N., Chen, T., Towards computational models of kinship verification. ICIP, 2010, 1577–1580.
66. [66] Guo, G., Wang, X., Kinship measurement on salient facial features. IEEE Trans. Instrum. Meas. 61:8 (2012), 2322–2325.
67. [67] Lu, J., Hu, J., Zhou, X., Shang, Y., Tan, Y.-P., Wang, G., Neighborhood repulsed metric learning for kinship verification. CVPR, 2012, 2594–2601.
68. [68] Wang, X., Kambhamettu, C., Leveraging appearance and geometry for kinship verification. 2014 IEEE International Conference on Image Processing (ICIP), 2014.
69. [69] Fang, R., Gallagher, A.C., Chen, T., Loui, A., Kinship classification by modeling facial feature heredity. ICIP, 2013.
70. [70] Peng, X., Huang, J., Hu, Q., Zhang, S., Elgammal, A., Metaxas, D., From circle to 3-sphere: head pose estimation by instance parameterization. Comput. Vis. Image Underst., 2015, 92–102.