Recognition as translating images into text

Proceedings of SPIE - The International Society for Optical Engineering

Volumn 5018, Issue , 2003, Pages 168-178

  Barnard, Kobus ^a   Duygulu, Pinar ^b   Forsyth, David ^c  

^a University of Arizona   (United States)

^b MIDDLE EAST TECHNICAL UNIVERSITY   (Turkey)

^c UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Aspect model; Hierarchical clustering; Learning image semantics; Machine translation; Object recognition

Indexed keywords

COMPUTER VISION; IMAGE RETRIEVAL; OBJECT RECOGNITION; SEMANTICS;

HIERARCHICAL CLUSTERING;

IMAGE PROCESSING;

EID: 0038057932     PISSN: 0277786X     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1117/12.478427     Document Type: Conference Paper

References (42)

1. Marr Vision Freeman D. 1982.
2. S. Santini, Semantic Modalities in Content-Based Retrieval, IEEE International Conference on Multimedia and Expo (2000).
3. M. La Cascia, S. Sethi, and S. Sclaroff, Combining Textual and Visual Cues for Content-based Image Retrieval on the World Wide Web, IEEE Workshop on Content-Based Access of Image and Video Libraries (1998).
4. R. K. Srihari and D. T. Burhans, Visual Semantics: Extracting Visual Information from Text Accompanying Pictures, AAAI '94 (1994).
5. R. K. Srihari, R. Chopra, D. Burhans, M. Venkataraman, and V. Govindaraju, Use of Collateral Text in Image Interpretation, ARPA Image Understanding Workshop (1994).
6. C. Carson, S. Belongie, H. Greenspan, and J. Malik, Blobworld: Image segmentation using Expectation-Maximization and its application to image querying, submitted for publication to IEEE PAMI. Available in the interim from http://HTTP.CS.Berkeley.EDU/̃carson/papers/pami.html.
7. F. Chen, U. Gargi, L. Niles, and H. Schütze, Multi-modal browsing of images in web documents, SPIE Document Recognition and Retrieval (1999).
8. K. Barnard, P. Duygulu, and D. Forsyth, Modeling the Statistics of Image Features and Associated Text, Document Recognition and Retrieval IX - Electronic Imaging (2002).
9. K. Barnard, P. Duygulu, and D. Forsyth, Clustering Art, IEEE Conference on Computer Vision and Pattern Recognition, II:434-441 (2001).
10. N. C. Rowe, Marie-4: A high-recall self-improving web crawler that finds images using captions, IEEE Intelligent Systems, July/August, 8-14 (2002).
11. M. Markkula and E. Sormunen, End-user searching challenges indexing practices in the digital newspaper photo archive, Information retrieval, 1, 259-285 (2000).
12. P. G. B. Enser, Progress in documentation pictorial information retrieval, Journal of Documentation, 51, 126-170 (1995).
13. P. G. B. Enser, Query analysis in a visual information retrieval context, Journal of Document and Text Management, 1, 25-39 (1993).
14. L. H. Armitage and P. G. B. Enser, Analysis of user need in image archives, Journal of Information Science, 23, 287-299 (1997).
15. K. Barnard and D. Forsyth, Learning the Semantics of Words and Pictures, International Conference on Computer Vision, II:408-415 (2001).
16. C. O. Frost, B. Taylor, A. Noakes, S. Markel, D. Torres, and K. M. Drabenstott, Browse and search patterns in a digital image database, Information retrieval, 1, 287-313 (2000).
17. D. A. Forsyth and J. Ponce, Computer Vision - A Modern Approach, in press.
18. D. A. Forsyth, Computer Vision Tools for Finding Images and Video Sequences, Library Trends, 48, 326-355 (1999).
19. H. Schneiderman and T. Kanade, A Statistical Approach to 3D Object Recognition Applied to Faces and Cars, IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 100 (2000).
20. M. M. Fleck, D. A. Forsyth, and C. Bregler, Finding Naked People, 4th European Conference on Computer Vision, Springer, II:591-602 (1996).
21. M. Oren, C. Papageorgiou, P. Sinha, and E. Osuna, Pedestrian detection using wavelet templates, Computer vision and pattern recognition, 193-9 (1997).
22. J. P. Eakins, Towards intelligent image retrieval, Pattern Recognition, 35, 3-14 (2002).
23. S. Santini, A. Gupta, and R. Jain, Emergent semantics through interaction in Image Databases, IEEE Transactions on Knowledge and Data Engineering, (2001).
24. S. Ornager, View a picture. Theoretical image analysis and empirical user studies on indexing and retrieval, Swedis Library Research, 2, 31-41 (1996).
25. L. H. Keister, User types and queries: impact on image access systems, in Challenges in indexing electronic text and images. Learned Information (1994).
26. M. J. Swain, C. Frankel, and V. Athitsos, WebSeer: An Image Search Engine for the World Wide Web, Computer Science Department, University of Chicago TR-96-14, available from (1996).
27. J.-Y. Chen, C. A. Bouman, and J. C. Dalton, Hierarchical Browsing and Search of Large Image Databases, IEEE Transactions on Image Processing, 9, 442-455 (2000).
28. R. Srihari, Extracting Visual Information from Text: Using Captions to Label Human Faces in Newspaper Photographs, SUNY at Buffalo, Ph.D. thesis, (1991).
29. O. Maron, Learning from Ambiguity, Massachusetts Institute of Technology, Ph.D., (1998).
30. O. Maron and A. L. Ratan, Multiple-Instance Learning for Natural Scene Classification, The Fifteenth International Conference on Machine Learning (1998).
31. Y. Mori, H. Takahashi, and R. Oka, Image-to-word transformation based on dividing and vector quantizing images with words, First International Workshop on Multimedia Intelligent Storage and Retrieval Management (in conjunction with ACM Multimedia Conference 1999) (1999).
32. D. Melamed, Empirical methods for exploiting parallel texts, MIT Press, Cambridge, Massachusetts, 2001.
33. D. Jurafsky and J. H. Martin, Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics and Speech Recognition, Prentice-Hall, 2000.
34. C. Manning and H. Schütze, Foundations of Statistical Natural Language Processing, MIT Press. Cambridge, MA, 1999.
35. J. Shi and J. Malik., Normalized Cuts and Image Segmentation, IEEE Trans. Patt. Anal. Mach. Intell., 22, 888-905 (2000).
36. K. Barnard, P. Duygulu, N. D. Freitas, D. Forsyth, D. Blei, and M. I. Jordan, Matching Words and Pictures, Journal of Machine Learning Research (in press).
37. P. F. Brown, S. A. D. Pietra, V. J. D. Pietra, and R. L. Mercer, The mathematics of machine translation: parameter estimation, Computational Linguistics, 19, 263-311 (1993).
38. A. P. Dempster, N. M. Laird, and D. B. Rubin, Maximum likelihood from incomplete data via the EM algorithm, Journal of the Royal Statistical Society. Series B (Methodological), 39, 1-38 (1977).
39. P. Duygulu, K. Barnard, J. F. G. D. Freitas, and D. A. Forsyth, Object recognition as machine translation: Learning a lexicon for a fixed image vocabulary, The Seventh European Conference on Computer Vision, IV:97-112 (2002).
40. T. Hofmann, Learning and representing topic. A hierarchical mixture model for word occurrence in document databases, Workshop on learning from text and the web (1998).
41. T. Hofmann and J. Puzicha, Statistical models for co-occurrence data, Massachusetts Institute of Technology, A.I. Memo 1635, available from (1998).
42. R. Jonker and A. Volgenant, A Shortest Augmenting Path Algorithm for Dense and Sparse Linear Assignment Problems, Computing, 38, 325-340 (1987).