Discovering visual concept structure with sparse and incomplete tags

Artificial Intelligence

Volumn 250, Issue , 2017, Pages 16-36

  Wang, Jingya ^a   Zhu, Xiatian ^a   Gong, Shaogang ^a  

^a QUEEN MARY UNIVERSITY OF LONDON   (United Kingdom)

Author keywords

Data clustering; Incomplete tags; Missing tag completion; Random forest; Sparse tags; Tag correlation; Tag hierarchy; Visual semantic structure

Indexed keywords

ARTIFICIAL INTELLIGENCE; BENCHMARKING; CLUSTER ANALYSIS; CLUSTERING ALGORITHMS; DECISION TREES; RANDOM FORESTS;

DATA CLUSTERING; INCOMPLETE TAGS; SPARSE TAGS; TAG HIERARCHY; VISUAL SEMANTICS;

SEMANTICS;

EID: 85020674450     PISSN: 00043702     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.artint.2017.05.002     Document Type: Article

References (73)

1. Beyer, K., Goldstein, J., Ramakrishnan, R., Shaft, U., When is “nearest neighbor” meaningful?. Database Theory—ICDT'99, 1999, 217–235.
2. Jain, A.K., Data clustering: 50 years beyond k-means. Pattern Recognit. Lett. 31:8 (2010), 651–666.
3. Zhou, G.-T., Lan, T., Vahdat, A., Mori, G., Latent margin clustering. Advances in Neural Information Processing Systems, Lake Tahoe, Nevada, USA, 2013, 28–36.
4. Vahdat, A., Zhou, G.-T., Mori, G., Discovering video clusters from visual features and noisy tags. European Conference on Computer Vision, Zurich, Switzerland, 2014, 526–539.
5. Wu, L., Jin, R., Jain, A.K., Tag completion for image retrieval. IEEE Trans. Pattern Anal. Mach. Intell. 35:3 (2013), 716–727.
6. Feng, Z., Feng, S., Jin, R., Jain, A.K., Image tag completion by noisy matrix recovery. European Conference on Computer Vision, Zurich, Switzerland, 2014, 424–438.
7. Lin, Z., Ding, G., Hu, M., Wang, J., Ye, X., Image tag completion via image-specific and tag-specific linear sparse reconstructions. IEEE Conference on Computer Vision and Pattern Recognition, Portland, Oregon, United States, 2013, 1618–1625.
8. Truong, B.T., Venkatesh, S., Video abstraction: a systematic review and classification. ACM Trans. Multimed. Comput. Commun. Appl., 3(1), 2007, 3.
9. Duin, R., Loog, M., Linear dimensionality reduction via a heteroscedastic extension of LDA: the Chernoff criterion. IEEE Trans. Pattern Anal. Mach. Intell. 26:6 (2004), 732–739.
10. Chua, T.-S., Tang, J., Hong, R., Li, H., Luo, Z., Zheng, Y., NUS-WIDE: a real-world web image database from National University of Singapore. ACM International Conference on Image and Video Retrieval, Santorini, Greece, 2009, 48.
11. Vahdat, A., Mori, G., Handling uncertain tags in visual recognition. IEEE International Conference on Computer Vision, Sydney, Australia, 2013, 737–744.
12. Natarajan, P., Wu, S., Vitaladevuni, S., Zhuang, X., Tsakalidis, S., Park, U., Prasad, R., Natarajan, P., Multimodal feature fusion for robust event detection in web videos. IEEE Conference on Computer Vision and Pattern Recognition, Providence, Rhode Island, 2012, 1298–1305.
13. Makadia, A., Pavlovic, V., Kumar, S., A new baseline for image annotation. European Conference on Computer Vision, Berlin, Heidelberg, 2008, 316–329.
14. Huang, H.-C., Chuang, Y.-Y., Chen, C.-S., Affinity aggregation for spectral clustering. IEEE Conference on Computer Vision and Pattern Recognition, Marseille, France, 2012, 773–780.
15. Ng, A.Y., Jordan, M.I., Weiss, Y., et al. On spectral clustering: analysis and an algorithm. Advances in Neural Information Processing Systems, vol. 2, Vancouver, British Columbia, 2002, 849–856.
16. Quadrianto, N., Lampert, C.H., Learning multi-view neighborhood preserving projections. International Conference on Machine Learning, Bellevue, Washington, United States, 2011, 425–432.
17. Srivastava, N., Salakhutdinov, R.R., Multimodal learning with deep Boltzmann machines. Advances in Neural Information Processing Systems, 2012, Lake Tahoe, Nevada, United States, 2222–2230.
18. Ngiam, J., Khosla, A., Kim, M., Nam, J., Lee, H., Ng, A.Y., Multimodal deep learning. International Conference on Machine Learning, 2011, 689–696.
19. Frome, A., Corrado, G.S., Shlens, J., Bengio, S., Dean, J., Mikolov, T., et al. Devise: a deep visual-semantic embedding model. Advances in Neural Information Processing Systems, 2013, 2121–2129.
20. Hardoon, D.R., Szedmak, S., Shawe-Taylor, J., Canonical correlation analysis: an overview with application to learning methods. Neural Comput. 16:12 (2004), 2639–2664.
21. Hwang, S.J., Grauman, K., Learning the relative importance of objects from tagged images for retrieval and cross-modal search. Int. J. Comput. Vis. 100:2 (2012), 134–153.
22. Gong, Y., Ke, Q., Isard, M., Lazebnik, S., A multi-view embedding space for modeling internet images, tags, and their semantics. Int. J. Comput. Vis. 106:2 (2014), 210–233.
23. Rai, P., Daume, H., Multi-label prediction via sparse infinite CCA. Advances in Neural Information Processing Systems, Vancouver, British Columbia, Canada, 2009, 1518–1526.
24. Sharma, A., Kumar, A., Daume, H. III, Jacobs, D.W., Generalized multiview analysis: a discriminative latent space. IEEE Conference on Computer Vision and Pattern Recognition, Providence, Rhode Island, United States, 2012, 2160–2167.
25. Andrew, G., Arora, R., Bilmes, J.A., Livescu, K., Deep canonical correlation analysis. International Conference on Machine Learning, 2013, 1247–1255.
26. Wang, W., Arora, R., Livescu, K., Bilmes, J., On deep multi-view representation learning. International Conference on Machine Learning, 2015, 1083–1092.
27. Natarajan, N., Dhillon, I.S., Ravikumar, P.K., Tewari, A., Learning with noisy labels. Advances in Neural Information Processing Systems, 2013, 1196–1204.
28. S. Sukhbaatar, J. Bruna, M. Paluri, L. Bourdev, R. Fergus, Training convolutional networks with noisy labels, in: Workshop of International Conference in Learning Representations.
29. Frénay, B., Verleysen, M., Classification in the presence of label noise: a survey. IEEE Trans. Neural Netw. Learn. Syst. 25:5 (2014), 845–869.
30. Cabral, R.S., Torre, F., Costeira, J.P., Bernardino, A., Matrix completion for multi-label image classification. Advances in Neural Information Processing Systems, Granada, Spain, 2011, 190–198.
31. Mu, Y., Dong, J., Yuan, X., Yan, S., Accelerated low-rank visual recovery by random projection. IEEE Conference on Computer Vision and Pattern Recognition, Colorado Springs, USA, 2011, 2609–2616.
32. Liu, X., Yan, S., Chua, T.-S., Jin, H., Image label completion by pursuing contextual decomposability. ACM Trans. Multimed. Comput. Commun. Appl., 8(2), 2012, 21.
33. Candès, E.J., Recht, B., Exact matrix completion via convex optimization. Found. Comput. Math. 9:6 (2009), 717–772.
34. Fellbaum, C., WordNet. 1998, Wiley Online Library.
35. Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L., ImageNet: a large-scale hierarchical image database. IEEE Conference on Computer Vision and Pattern Recognition, Miami, Florida, United States, 2009, 248–255.
36. Personalized recommendation in social tagging systems using hierarchical clustering. Proceedings of the 2008 ACM Conference on Recommender Systems, New York, NY, USA, 2008, 259–266.
37. Zheng, S., Cheng, M.-M., Warrell, J., Sturgess, P., Vineet, V., Rother, C., Torr, P.H., Dense semantic image segmentation with objects and attributes. IEEE Conference on Computer Vision and Pattern Recognition, Columbus, Ohio, 2014, 3214–3221.
38. Deng, J., Ding, N., Jia, Y., Frome, A., Murphy, K., Bengio, S., Li, Y., Neven, H., Adam, H., Large-scale object classification using label relation graphs. European Conference on Computer Vision, Zurich, Switzerland, 2014, 48–64.
39. Griffiths, T., Ghahramani, Z., Infinite latent feature models and the Indian buffet process. Advances in Neural Information Processing Systems, 2005, 475–482.
40. Chen, X., Mu, Y., Yan, S., Chua, T.-S., Efficient large-scale image annotation by probabilistic collaborative multi-label propagation. ACM International Conference on Multimedia, New York, NY, USA, 2010, 35–44.
41. Choi, M.J., Lim, J.J., Torralba, A., Willsky, A.S., Exploiting hierarchical context on a large database of object categories. IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, California, USA, 2010, 129–136.
42. Desai, C., Ramanan, D., Fowlkes, C.C., Discriminative models for multi-class object layout. Int. J. Comput. Vis. 95:1 (2011), 1–12.
43. Chen, X., Yuan, X.-T., Chen, Q., Yan, S., Chua, T.-S., Multi-label visual classification with label exclusive context. IEEE International Conference on Computer Vision, Barcelona, Spain, 2011, 834–841.
44. Zhou, Y., Jin, R., Hoi, S., Exclusive lasso for multi-task feature selection. International Conference on Artificial Intelligence and Statistics, 2010, Chia Laguna Resort, Sardinia, Italy, 988–995.
45. ® Comp. Graph. Vis. 7:2–3 (2012), 81–227.
46. Montillo, A., Shotton, J., Winn, J., Iglesias, J.E., Metaxas, D., Criminisi, A., Entangled decision forests and their application for semantic segmentation of CT images. Information Processing in Medical Imaging, Kloster Irsee, Germany, 2011, 184–196.
47. Zhao, X., Kim, T.-K., Luo, W., Unified face analysis by iterative multi-output random forests. IEEE Conference on Computer Vision and Pattern Recognition, Columbus, Ohio, 2014, 1765–1772.
48. Zhu, X., Loy, C.C., Gong, S., Video synopsis by heterogeneous multi-source correlation. IEEE International Conference on Computer Vision, Sydney, Australia, 2013, 81–88.
49. Breiman, L., Random forests. Mach. Learn. 45:1 (2001), 5–32.
50. Shi, T., Horvath, S., Unsupervised learning with random forest predictors. J. Comput. Graph. Stat. 15:1 (2006), 118–138.
51. Breiman, L., Friedman, J., Stone, C., Olshen, R., Classification and Regression Trees. 1984, Chapman & Hall/CRC.
52. Liu, B., Xia, Y., Yu, P.S., Clustering through decision tree construction. Ninth International Conference on Information and Knowledge Management, McLean, VA, USA, 2000, 20–29.
53. Berger, A., Caruana, R., Cohn, D., Freitag, D., Mittal, V., Bridging the lexical chasm: statistical approaches to answer-finding. ACM SIGIR Conference on Research and Development in Information Retrieval, Athens, Greece, 2000.
54. Sivic, J., Zisserman, A., Video Google: a text retrieval approach to object matching in videos. IEEE International Conference on Computer Vision, Nice, France, 2003, 1470–1477.
55. Cover, T.M., Hart, P.E., Nearest neighbor pattern classification. IEEE Trans. Inf. Theory 13:1 (1967), 21–27.
56. Weinberger, K.Q., Saul, L.K., Distance metric learning for large margin nearest neighbor classification. J. Mach. Learn. Res. 10 (2009), 207–244.
57. P. Over, G.M. Awad, J. Fiscus, B. Antonishek, M. Michel, A.F. Smeaton, W. Kraaij, G. Quénot, TRECVID 2010 – an overview of the goals, tasks, data, evaluation mechanisms, and metrics, 2011, pp. 1–52.
58. Zhao, B., Wang, F., Zhang, C., Efficient multiclass maximum margin clustering. International Conference on Machine Learning, 2008, ACM, 1248–1255.
59. Wei, X., Croft, W.B., LDA-based document models for ad-hoc retrieval. ACM SIGIR Conference on Research and Development in Information Retrieval, 2006, ACM, 178–185.
60. Johnson, J., Ballan, L., Fei-Fei, L., Love thy neighbors: image annotation by exploiting image metadata. IEEE International Conference on Computer Vision, 2015, 4624–4632.
61. Hu, H., Zhou, G.-T., Deng, Z., Liao, Z., Mori, G., Learning structured inference neural networks with label relations. IEEE Conference on Computer Vision and Pattern Recognition, 2016, 2960–2968.
62. Klaser, A., Marszałek, M., Schmid, C., A spatio-temporal descriptor based on 3d-gradients. British Machine Vision Conference, Leeds, UK, 2008, 275-1.
63. Vedaldi, A., Zisserman, A., Efficient additive kernels via explicit feature maps. IEEE Trans. Pattern Anal. Mach. Intell. 34:3 (2012), 480–492.
64. Simonyan, K., Zisserman, A., Very deep convolutional networks for large-scale image recognition. International Conference on Learning Representation, 2015.
65. Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. 115:3 (2015), 211–252.
66. Frey, B.J., Dueck, D., Clustering by passing messages between data points. Science 315:5814 (2007), 972–976.
67. Xu, L., Neufeld, J., Larson, B., Schuurmans, D., Maximum margin clustering. Advances in Neural Information Processing Systems, 2004, 1537–1544.
68. Vinh, N.X., Epps, J., Bailey, J., Information theoretic measures for clusterings comparison: is a correction for chance necessary?. International Conference on Machine Learning, Montreal, Canada, 2009, 1073–1080.
69. Rand, W.M., Objective criteria for the evaluation of clustering methods. J. Am. Stat. Assoc. 66:336 (1971), 846–850.
70. Steinley, D., Properties of the Hubert–Arable adjusted Rand index. Psychol. Methods, 9(3), 2004, 386.
71. Jardine, N., van Rijsbergen, C.J., The use of hierarchic clustering in information retrieval. Inf. Storage Retr., 1971, 217–240.
72. Vinh, N.X., Epps, J., Bailey, J., Information theoretic measures for clusterings comparison: variants, properties, normalization and correction for chance. J. Mach. Learn. Res. 11 (2010), 2837–2854.
73. Shotton, J., Johnson, M., Cipolla, R., Semantic texton forests for image categorization and segmentation. IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, Alaska, United States, 2008, 1–8.