Learning consistent feature representation for cross-modal multimedia retrieval

IEEE Transactions on Multimedia

Volumn 17, Issue 3, 2015, Pages 370-381

  Kang, Cuicui ^a   Xiang, Shiming ^a   Liao, Shengcai ^a   Xu, Changsheng ^a   Pan, Chunhong ^a  

^a INSTITUTE OF AUTOMATION   (China)

Author keywords

Cross modal matching; Documents and images; Multimedia; Retrieval

Indexed keywords

ALGORITHMS; CONTENT BASED RETRIEVAL; IMAGE RETRIEVAL; INFORMATION RETRIEVAL; SEMANTICS;

CLASS LABEL INFORMATIONS; CROSS-MODAL; DOCUMENTS AND IMAGES; FEATURE REPRESENTATION; LOW-LEVEL IMAGE FEATURES; MULTIMEDIA; RETRIEVAL; STATE-OF-THE-ART ALGORITHMS;

IMAGE MATCHING;

EID: 84923362102     PISSN: 15209210     EISSN: None     Source Type: Journal    
DOI: 10.1109/TMM.2015.2390499     Document Type: Article

References (48)

1. A. Beck and M. Teboulle, "A fast iterative shrinkage-thresholding algorithm for linear inverse problems," SIAM J. Imaging Sci., vol. 2, no. 1, pp. 183-202, 2009.
2. T.-S. Chua, J. Tang, R. Hong, H. Li, Z. Luo, and Y.-T. Zheng, "NUSWIDE: A real-world web image database from National University of Singapore," in Proc. ACM Conf. Image Video Retrieval, 2009, pp. 1-9.
3. J. Costa Pereira, E. Coviello, G. Doyle, N. Rasiwasia, G. Lanckriet, R. Levy, and N. Vasconcelos, "On the role of correlation and abstraction in cross-modal multimedia retrieval," IEEE Trans. Pattern Anal.Mach. Intell., vol. 36, no. 3, pp. 521-535, Mar. 2014.
4. N. Dalal and B. Triggs, "Histograms of oriented gradients for human detection," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2005, vol. 1, pp. 886-893.
5. L. Duan, D. Xu, and I. W. Tsang, "Learning with augmented features for heterogeneous domain adaptation," in Proc. Int. Conf. Mach. Learn., 2012, pp. 711-718.
6. M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman, The PASCAL Visual Object Classes Challenge 2007(VOC2007) Results., 2007 [Online]. Available: http://www. pascal-network.org/challenges/VOC/voc2007/workshop/index.html
7. Y. Gong, Q. Ke, M. Isard, and S. Lazebnik, "A multi-view embedding space for modeling internet images, tags, and their semantics," Int. J. Comput. Vis., vol. 106, no. 2, pp. 210-233, 2014.
8. M. Guillaumin, J. J. Verbeek, and C. Schmid, "Multimodal semi-supervised learning for image classification," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2010, pp. 902-909.
9. D. R. Hardoon, S. Szedm?k, and J. Shawe-Taylor, "Canonical correlation analysis: An overview with application to learning methods," Neural Comput., vol. 16, no. 12, pp. 2639-2664, 2004.
10. S. J. Hwang and K. Grauman, "Accounting for the relative importance of objects in image retrieval," in Proc. BMVC, , 2010, pp. 1-12.
11. S. J. Hwang and K. Grauman, "Reading between the lines: Object localization using implicit cues from image tags," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2010, pp. 2971-2978.
12. Y. Jia, M. Salzmann, and T. Darrell, "Learning cross-modality similarity for multinomial data," in Proc. Int. Conf. Comput. Vis., Nov. 2011, pp. 2407-2414.
13. M. Kan, S. Shan, H. Zhang, S. Lao, and X. Chen, "Multi-view discriminant analysis," in Proc. ECCV (1), 2012, vol. 7572, pp. 808-821.
14. C. H. Lampert and O. Kr?mer, "Weakly-paired maximum covariance analysis for multimodal dimensionality reduction and transfer learning," in Proc. ECCV (2), 2010, vol. 6312, pp. 566-579.
15. L. D. Lee and H. S. Seung, "Learning the parts of objects with nonnegative matrix factorization," in Nature, 1999, vol. 401, pp. 788-791.
16. Z. Lei, S. Liao, A. K. Jain, and S. Z. Li, "Coupled discriminant analysis for heterogeneous face recognition," IEEE Trans. Inf. Forensics and Security, vol. 7, no. 6, pp. 1707-1716, Dec. 2012.
17. A. Li, S. Shan, X. Chen, and W. Gao, "Maximizing intra-individual correlations for face recognition across pose differences," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2009, pp. 605-611.
18. S. Liao, Z. Lei, and S. Li, "Nonnegative matrix factorization with Gibbs random field modeling," in Proc. 12th IEEE Int. Conf. Comput. Vis. Workshops, Sep.-Oct. 2009, pp. 79-86.
19. S. Liao, D. Yi, Z. Lei, R. Qin, and S. Z. Li, "Heterogeneous face recognition from local structures of normalized appearance," in Proc. ICB, , vol. 5558, pp. 209-218.
20. J. Liu, S. Ji, and J. Ye, "Multi-task feature learning via efficient -norm minimization," in Proc. Uncertainty AI, 2009, pp. 339-348.
21. W. Liu, J. Wang, S. Kumar, and S.-F. Chang, "Hashing with graphs," in Proc. ICML, 2011, pp. 1-8.
22. J. Masci, M. Bronstein, A. Bronstein, and J. Schmidhuber, "Multimodal similarity-preserving hashing," IEEE Trans. Pattern Anal. Mach. Intell., vol. 36, no. 4, pp. 824-830, Apr. 2014.
23. T. Mei, Y. Rui, S. Li, and Q. Tian, "Multimedia search reranking: A literature survey," ACM Comput. Surveys, vol. 46, no. 3, pp. 1-38, 2014.
24. F. Monay and D. Gatica-Perez, "Modeling semantic aspects for crossmedia image indexing," IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 10, pp. 1802-1817, Oct. 2007.
25. Y. Nesterov, Introductory Lectures on Convex Optimization: A Basic Course. . Dordrecht, The Netherlands : Kluwer Academic Publishers, 2003.
26. J. Ngiam, A. Khosla, M. Kim, J. Nam, H. Lee, and A. Y. Ng, "Multimodal deep learning," in Proc. CML, 2011, pp. 689-696.
27. A. Oliva and A. Torralba, "Modeling the shape of the scene: A holistic representation of the spatial envelope," Int. J. Comput. Vis., vol. 42, no. 3, pp. 145-175, 2001.
28. Y. Pan, T. Yao, T. Mei, H. Li, C.-W. Ngo, and Y. Rui, "Click-throughbased cross-view learning for image search," in Proc. ACM Conf. Res. Develop. Inf. Retrieval (SIGIR), 2014, pp. 717-726.
29. N. Rasiwasia, P. J. Moreno, and N. Vasconcelos, "Bridging the gap: Query by semantic example," IEEE Trans. Multimedia, vol. 9, no. 5, pp. 923-938, Aug. 2007.
30. N. Rasiwasia, J. C. Pereira, E. Coviello, G. Doyle, G. R. G. Lanckriet, R. Levy, and N. Vasconcelos, "A new approach to cross-modal multimedia retrieval," in Proc. ACM Multimedia, 2010, pp. 251-260.
31. R. Rosipal and N. Kr?mer, "Overview and recent advances in partial least squares," in Proc. SLSFS, 2006, pp. 34-51.
32. A. Sharma and D. W. Jacobs, "Bypassing synthesis: PLS for face recognition with pose, low-resolution and sketch," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2011, pp. 593-600.
33. A. Sharma, A. Kumar, H. D. , III, and D. W. Jacobs, "Generalized multiview analysis: A discriminative latent space," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2012, pp. 2160-2167.
34. R. Socher, M. Ganjoo, C. D. Manning, and A. Y. Ng, "Zero-shot learning through cross-modal transfer," in Proc. NIPS, 2013, pp. 935-943.
35. R. Socher and F.-F. Li, "Connecting modalities: Semi-supervised segmentation and annotation of images using unaligned text corpora," in Proc. IEEE Conf. Comput. Vis. Pattern Recog.., Jun. 2010, pp. 966-973.
36. N. Srivastava and R. Salakhutdinov, "Multimodal learning with deep boltzmann machines," in Proc. Neural Inf. Process. Syst., 2012, pp. 2231-2239.
37. L. Sun, S. Ji, and J. Ye, "A least squares formulation for canonical correlation analysis," in Proc. 25th Int. Conf. Mach. Learn., 2008, vol. 307, pp. 1024-1031.
38. J. B. Tenenbaum and W. T. Freeman, "Separating style and content with bilinear models," Neural Comput., vol. 12, no. 6, pp. 1247-1283, 2000.
39. R. Tibshirani, "Regression shrinkage and selection via the lasso," J. Royal Statist. Soc., Ser. B, vol. 58, no. 1, pp. 267-288, 1996.
40. P. Tseng, "On accelerated proximal gradient methods for convex-concave optimization," in SIAM J. Optimization, 2008, to be published.
41. R. Udupa and M. M. Khapra, "Improving the multilingual user experience of wikipedia using cross-language name search," in Proc. HLT-NAACL, 2010, pp. 492-500.
42. D. B. C. Wang and L. Fei-Fei, "Simultaneous image classification and annotation," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2009, pp. 1903-1910.
43. J. Wang, S. Kumar, and S.-F. Chang, "Semi-supervised hashing for large-scale search," IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 12, pp. 2393-2406, Dec. 2012.
44. K. Wang, R. He, W. Wang, L. Wang, and T. Tan, "Learning coupled feature spaces for cross-modal matching," in Proc. Int. Conf. Comput. Vis., 2013, pp. 2088-2095.
45. T. Xia, D. Tao, T. Mei, and Y. Zhang, "Multiview spectral embedding," IEEE Trans. Syst., Man, Cybern. B: Cybern., vol. 40, no. 6, pp. 1438-1446, Dec. 2010.
46. S. Xiang, F. Nie, G. Meng, C. Pan, and C. Zhang, "Discriminative least squares regression for multiclass classification and feature selection," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 11, pp. 1738-1754, Nov. 2012.
47. J. Zhuang and S. C. H. Hoi, "A two-view learning approach for image tag ranking," in Proc. WSDM, 2011, pp. 625-634.
48. Y. Zhuang, Y. F. Wang, F. Wu, Y. Zhang, and W. Lu, "Supervised coupled dictionary learning with group structures for multi-modal retrieval," in Proc. 27th AAAI Conf. AI, 2013, pp. 1070-1076.