Cross media analysis and retrieval on the web: Theory and algorithm

Jisuanji Fuzhu Sheji Yu Tuxingxue Xuebao/Journal of Computer-Aided Design and Computer Graphics

Volumn 22, Issue 1, 2010, Pages 1-9

  Wu, Fei ^a   Zhuang, Yueting ^a  

^a ZHEJIANG UNIVERSITY   (China)

Author keywords

Cross media; Large scale information; Search engine; Semantic understanding and annotation

Indexed keywords

INFORMATION RETRIEVAL; SEMANTICS;

CROSS CORRELATIONS; CROSS-MEDIA; CROSS-MEDIA RETRIEVAL; HIGH-ACCURACY; HUMAN COGNITION; LARGE SCALE INFORMATION; MULTIMEDIA ANNOTATIONS; SEMANTIC UNDERSTANDING;

SEARCH ENGINES;

EID: 77349089293     PISSN: 10039775     EISSN: None     Source Type: Journal    
DOI: 10.3724/SP.J.1089.2010.10439     Document Type: Article

References (72)

1. Yeh T, Lee J J, Darrell T. Photo-based question answering[C]//Proceedings of the ACM International Conference on Multimedia, Vancouver, 2008: 389-398.
2. Katz B, Felshin S, Yuret D, et al. Omnibase: uniform access to heterogeneous data for question answering[C]//Proceedings of the 7th International Workshop on Applications of Natural Language to Information Systems, Stockholm, 2002: 230-234.
3. Snavely N, Seitz S M, Szeliski R. Photo tourism: exploring photo collections in 3D[J]. ACM Transactions on Graphics, 2006, 25(3): 835-846.
4. Rother C, Bordeaux L, Hamadi Y, et al. AutoCollage[J]. ACM Transactions on Graphics, 2006, 25(3): 847-852.
5. Etzioni O. Machine reading of web text[C]//Proceedings of the 4th International Conference on Knowledge Capture. Whistler, 2007: 1-4.
6. Ng T D, Christel M G, Hauptmann A G, et al. Collages as dynamic summaries of mined video content for intelligent multimedia knowledge management[C]//Proceedings of AAAI Spring Symposium on Intelligent Multimedia Knowledge Management, Palo Alto, 2003: 11-20.
7. McGurk H, MacDonald J. Hearing lips and seeing voices[J]. Nature, 1976, 264(5588): 746-748.
8. Calvert G A. Crossmodal processing in the human brain: insights from functional neuroimaging Studies[J]. Cerebral Cortex, 2001, 11(12): 1110-1123.
9. Bushara K O, Hanakawa T, Immisch I, et al. Neural correlates of cross-modal binding[J]. Nature Neuroscience, 2002, 6(2): 190-195.
10. Ma Q, Nadamoto A, Tanaka K. Complementary information retrieval for cross-media news content[C]//Proceedings of the 2nd ACM International Workshop on Multimedia Databases, Washington D C, 2004: 45-54.
11. Zhu X J, Goldberg A B, Eldawy M, et al. A text-to-picture synthesis system for augmenting communication[C]//Proceedings of the 22nd AAAI Conference on Artificial Intelligence, Vancouver, 2007: 681-687.
12. Smoliar S W, Zhang H J. Content-based video indexing and retrieval[J]. IEEE Multimedia, 1994, 1(2): 62-72.
13. Rui Y, Huang T S, Ortega M, et al. Relevance feedback: a power tool for interactive content-based image retrieval[J]. IEEE Transactions on Circuits and Systems for Video Technology, 1998, 8(5): 644-655.
14. Datta R, Joshi D, Li J, et al. Image retrieval: ideas, influences, and trends of the new age[J]. ACM Computing Surveys, 2008, 40(2): 5-60.
15. Wu Y, Tian Q, Huang T S. Discriminant-EM algorithm with application to image retrieval[C]//Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Hilton Head, 2000: 1222-1227.
16. Tong S, Chang E. Support vector machine active learning for image retrieval[C]//Proceedings of the ACM International Conference on Multimedia, Ottawa, 2001: 107-118.
17. He X F, Ma W Y, Zhang H J. Learning an image manifold for retrieval[C]//Proceedings of the ACM International Conference on Multimedia, New York, 2004: 17-23.
18. Saul L K, Weinberger K Q, Sha F, et al. Spectral methods for dimensionality reduction[M]//Chapelle O, Scholkopf B, Zien A. Semisupervised Learning, Cambridge: MIT Press, 2006: 293-308.
19. Belkin M, Niyogi P, Sindhwani V. Manifold regularization: a geometric framework for learning from labeled and unlabeled examples[J]. Journal of Machine Learning Research, 2006, 7: 2399-2434.
20. Wang J Z, Geman D, Luo J B, et al. Real-world image annotation and retrieval: an introduction to the special section[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008, 30(11): 1873-1876.
21. Wang X J, Zhang L, Li X R, et al. Annotating images by mining image search results[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008, 30(11): 1919-1932.
22. Li J, Wang J Z. Real-time computerized annotation of pictures[C]//Proceedings of the ACM International Conference on Multimedia, Santa Barbara, 2006: 911-920.
23. Satoh S, Nakamura Y, Kanade T. Name-It: naming and detecting faces in news videos[J]. IEEE Multimedia, 1999, 6(1): 22-35.
24. Yang J, Hauptmann A G. Annotating news video with locations[C]//Proceedings of the ACM International Conference on Image and Video Retrieval, Tempe, 2006: 153-162.
25. Rabiner L R. A tutorial on hidden Markov models and selected applications in speech recognition[J]. Proceedings of the IEEE, 1989, 77(2): 257-286.
26. Burges C J C. A tutorial on support vector machines for pattern recognition[J]. Data Mining and Knowledge Discovery, 1998, 2(2): 121-167.
27. Zhou Z H, Yu Y. AdaBoost[M]//Wu X, Kumar V. The top ten algorithms in data mining, Boca Raton: Chapman and Hall, 2009: 1-25.
28. Wang G, Chua T S, Zhao M. Exploring knowledge of sub-domain in a multi-resolution bootstrapping framework for concept detection in news video[C]//Proceedings of the ACM International Conference on Multimedia, Vancouver, 2008: 249-258.
29. Hauptmann A G, Christel M G. Successful approaches in the TREC video retrieval evaluations[C]//Proceedings of the ACM International Conference on Multimedia, New York, 2004: 668-675.
30. Shum H. Human intention modeling and interactive computer vision[C]//Proceedings of International Conference on Intelligent Robots and Systems, Beijing, 2006: 32-32.
31. Cai D, He X F, Wen J R, et al. Block-level link analysis[C]//Proceedings of the ACM SIGIR Conference on Information Retrieval, Sheffield, 2004: 440-447.
32. Chakrabarti D, Kumar R, Punera K. A graph-theoretic approach to webpage segmentation[C]//Proceedings of the ACM International Conference on World Wide Web, Beijing, 2008: 377-386.
33. Wu L, Hua X S, Yu N H, et al. Flickr distance[C]//Proceedings of the ACM International Conference on Multimedia, Vancouver, 2008: 31-40.
34. Cilibrasi R L, Vitányi P M B. The Google similarity distance[J]. IEEE Transactions on Knowledge and Data Engineering, 2007, 19(3): 370-383.
35. Lowe D G. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 2004, 60(2): 91-110.
36. Quelhas P, Monay F, Odobez J M, et al. A thousand words in a scene[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2007, 29(9): 1575-1589.
37. Zhu L, Chen Y H, Yuille A. Unsupervised learning of probabilistic grammar-Markov models for object categories[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2009, 31(1): 114-128.
38. Sutton C, McCallum A, Rohanimanesh K. Dynamic conditional random fields: factorized probabilistic models for labeling and segmenting sequence data[J]. Journal of Machine Learning Research, 2007, 8: 693-723.
39. Yang J, Jiang Y G, Hauptmann A G, et al. Evaluating bag-of-visual-words representations in scene classification[C]//Proceedings of the ACM Multimedia Information Retrieval Workshop, Augsburg, 2007: 197-205.
40. Yu H C, Bennamoun M. 1D-PCA, 2D-PCA to nD-PCA[C]//Proceedings of the 18th International Conference on Pattern Recognition, Hong Kong, 2006: 181-184.
41. Tao D C, Li X L, Wu X D, et al. Supervised tensor learning[J]. Knowledge and Information Systems, 2007, 13(1): 1-42.
42. Yan S C, Xu D, Lin S, et al. Element rearrangement for tensor-based subspace learning[C]//Proceedings of Computer Vision and Pattern Recognition, Minneapolis, 2007: 1-8.
43. Naphade M R, Kristjansson T, Frey B, et al. Probabilistic multimedia objects (Multijects): a novel approach to video indexing and retrieval in multimedia systems[C]//Proceedings of International Conference of Image Process, Chicago, 1998: 536-540.
44. Snoek C G M, Worring M. Multimedia event-based video indexing using time intervals[J]. IEEE Transactions on Multimedia, 2005, 7(4): 638-647.
45. Wu Fei, Liu Yanan, Zhuang Yueting. Transductive multi-modality video semantic concept detection with tensor representation[J]. Journal of Software, 2008, 19(11): 2853-2868 (in Chinese).
46. Yang Y, Zhuang Y T, Wu F, et al. Harmonizing hierarchical manifolds for multimedia document semantics understanding and cross-media retrieval[J]. IEEE Transactions on Multimedia, 2008, 10(3): 437-446.
47. Candès E J, Romberg J, Tao T. Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information[J]. IEEE Transactions on Information Theory, 2006, 52(2): 489-509.
48. Wright J, Yang A Y, Ganesh A, et al. Robust face recognition via sparse representation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2009, 31(2): 1-18.
49. Liu J, Wang B, Li M J, et al. Dual cross-media relevance model for image annotation[C]//Proceedings of the ACM International Conference on Multimedia, Augsburg, 2007: 605-614.
50. Sankar K, Jawahar C V. Probabilistic reverse annotation for large scale image retrieval[C]//Proceedings of Computer Vision and Pattern Recognition, Minneapolis, 2007: 1-6.
51. Zhang H, Zhuang Y T, Wu F. Cross-modal correlation learning for clustering on image-audio dataset[C]//Proceedings of the ACM International Conference on Multimedia, Augsburg, 2007: 273-276.
52. Zhang Hong, Wu Fei, Zhuang Yueting, et al. Cross-media retrieval method based on content correlations[J]. Chinese Journal of Computers, 2008, 31(5): 820-826 (in Chinese)
53. Jiang T, Tan A H. Discovering image-text associations for cross-media web information fusion[C]//Proceedings of Knowledge Discovery in Databases, Berlin, 2006: 561-568.
54. Hardoon D R, Szedmak S, Shawe-Taylor J. Canonical correlation analysis: an overview with application to learning methods[J]. Neural Computation, 2004, 16: 2639-2664.
55. Zhuang Y T, Yang Y, Wu F. Mining semantic correlation of heterogeneous multimedia data for cross-media retrieval[J]. IEEE Transactions on Multimedia, 2008, 10(2): 221-229.
56. Brin S, Page L. The anatomy of a large-scale hypertextual web search engine[J]. Computer Networks and ISDN Systems, 1998, 30(1): 107-117.
57. Kleinberg J M. Authoritative sources in a hyperlinked environment[J]. Journal of the ACM, 1999, 46(5): 604-632.
58. Jing Y S, Baluja S. VisualRank: applying PageRank to large-scale image search[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008, 30(11): 1877-1890.
59. Zhou K, Xue G R, Zha H Y, et al. Learning to rank with ties[C]//Proceedings of the Annual International ACM SIGIR Conference, Singapore, 2008: 275-282.
60. Zhou D Y, Weston J, Gretton A, et al. Ranking on data manifolds[C]//Proceedings of Annual Conference on Neural Information Processing Systems, Vancouver, 2003: 123-130.
61. Zeng H J, He Q C, Chen Z, et al. Learning to cluster web search results[C]//Proceedings of the Annual International ACM SIGIR Conference, Sheffield, 2004: 210-217.
62. Mahoney M W, Lim L H, Carlsson G E. Algorithmic and statistical challenges in modern large-scale data analysis are the focus of MMDS 2008[J]. SIGKDD Explorations, 2008, 10(2): 57-60.
63. Talwalkar A, Kumar S, Rowley H. Large-scale manifold learning[C]//Proceedings of Computer Vision and Pattern Recognition, Anchorage, 2008: 1-8.
64. Weber R, Schek H T, Blott S. A quantitative analysis and performance study for similarity-search methods in high-dimensional spaces[C]//Proceedings of the 24th International Conference on Very Large Data Bases, New York, 1998: 194-205.
65. Andoni A, Indyk P. Near-optimal hashing algorithms for approximate nearest neighbor in high dimensions[J]. Communications of the ACM, 2008, 51(1): 117-122.
66. Huang T. What are the 7 millennium problems in multimediainformation retrieval?[C]//Proceedings of the ACM International Conference on Multimedia Information Retrieval, Vancouver, 2008: 1.
67. Rao R, Olshausen B, Lewicki M. Probabilistic models of the brain: perception and neural function[M]. Cambridge: MIT Press, 2002.
68. Huang T S, Dagli C K, Rajaram S, et al. Active learning for interactive multimedia retrieval[J]. Proceedings of the IEEE, 2008, 96(4): 648-667.
69. Castro R, Kalish C, Nowak R, et al. Human active learning[C]//Proceedings of Annual Conference on Neural Information Processing Systems, Vancouver, 2008: 241-248.
70. Venkatesh S, Adams B, Phung D, et al. "You Tube and I find"-personalizing multimedia content access[J]. Proceedings of the IEEE, 2008, 96(4): 697-711.
71. Weld D S, Wu F, Adar E, et al. Intelligence in Wikipedia[C]//Proceedings of Association for the Advancement of Artificial Intelligence, Chicago, 2008: 1609-1614.
72. Torralba A, Fergus R, Freeman W T. 80 million tiny images: a large data set for nonparametric object and scene recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008, 30(11): 1958-1970.