Supervised learning of semantic classes for image annotation and retrieval

IEEE Transactions on Pattern Analysis and Machine Intelligence

Volumn 29, Issue 3, 2007, Pages 394-410

  Carneiro, Gustavo ^a   Chan, Antoni B ^b   Moreno, Pedro J ^c   Vasconcelos, Nuno ^b  

^a SIEMENS CORPORATE RESEARCH   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

^c GOOGLE INC   (United States)

Author keywords

Content based image retrieval; Expectation maximization; Gaussian mixtures; Image segmentation; Multiple instance learning; Object recognition; Semantic image annotation and retrieval; Weakly supervised learning

Indexed keywords

ALGORITHMS; COMPUTER SIMULATION; IMAGE RETRIEVAL; IMAGE SEGMENTATION; LEARNING SYSTEMS; OBJECT RECOGNITION; OPTIMIZATION; PROBABILITY;

EXPECTATION-MAXIMIZATION; GAUSSIAN MIXTURES; IMAGE ANNOTATION; MULTIPLE INSTANCE LEARNING; WEAKLY SUPERVISED LEARNING;

SEMANTICS;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; AUTOMATED PATTERN RECOGNITION; COMPUTER ASSISTED DIAGNOSIS; DATA BASE; DOCUMENTATION; FACTUAL DATABASE; IMAGE ENHANCEMENT; INFORMATION RETRIEVAL; METHODOLOGY; NATURAL LANGUAGE PROCESSING; SEMANTICS; SENSITIVITY AND SPECIFICITY;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; DATABASE MANAGEMENT SYSTEMS; DATABASES, FACTUAL; DOCUMENTATION; IMAGE ENHANCEMENT; IMAGE INTERPRETATION, COMPUTER-ASSISTED; INFORMATION STORAGE AND RETRIEVAL; NATURAL LANGUAGE PROCESSING; PATTERN RECOGNITION, AUTOMATED; SEMANTICS; SENSITIVITY AND SPECIFICITY;

EID: 33847419773     PISSN: 01628828     EISSN: None     Source Type: Journal    
DOI: 10.1109/TPAMI.2007.61     Document Type: Article

References (46)

1. R. Attar and A. Fraenkel, "Local Feedback in Full-Text Retrieval Systems," J. ACM, vol. 24, no. 3, pp. 397-417, July 1977.
2. P. Auer, "On Learning from Multi-Instance Examples: Empirical Evaluation of a Theoretical Approach," Proc. 14th Int'l Conf. Machine Learning, pp. 21-29, 1997.
3. K. Barnard and D. Forsyth, "Learning the Semantics of Words and Pictures," Proc. Int'l Conf. Computer Vision, vol. 2, pp. 408-415, 2001.
4. D. Blei and M. Jordan, "Modeling Annotated Data," Proc. ACM SIGIR Conf. Research and Development in Information Retrieval, 2003.
5. G. Carneiro and N. Vasconcelos, "Formulating Semantic Image Annotation as a Supervised Learning Problem," Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2005.
6. G. Carneiro and N. Vasconcelos, "A Database Centric View of Semantic Image Annotation and Retrieval," Proc. ACM SIGIR Conf. Research and Development in Information Retrieval, 2005.
7. J. De Bonet and P. Viola, "Structure Driven Image Database Retrieval," Proc. Conf. Advances in Neural Information Processing Systems, vol. 10, 1997.
8. P. Carbonetto, N. de Freitas, and K. Barnard, "A Statistical Model for General Contextual Object Recognition," Proc. European Conf. Computer Vision, 2004.
9. A. Dempster, N. Laird, and D. Rubin, "Maximum-Likelihood from Incomplete Data via the EM Algorithm," J. Royal Statistical Soc., vol. B-39, 1977.
10. T. Dietterich, R. Lathrop, and T. Lozano-Perez, "Solving the Multiple Instance Problem with Axis-Parallel Rectangles," Artificial Intelligence, vol. 89, nos. 1-2, pp. 31-71, 1997.
11. R. Duda, P. Hart, and D. Stork, Pattern Classification. John Wiley and Sons, 2001.
12. P. Duygulu, K. Barnard, and D.F.N. Freitas, "Object Recognition as Machine Translation: Learning a Lexicon for a Fixed Image Vocabulary," Proc. European Conf. Computer Vision, 2002.
13. S. Feng, R. Manmatha, and V. Lavrenko, "Multiple Bernoulli Relevance Models for Image and Video Annotation," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, 2004.
14. D. Forsyth and M. Fleck, "Body Plans," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 678-683, 1997.
15. P. Carbonetto, H. Kueck, and N. Freitas, "A Constrained Semi-Supervised Learning Approach to Data Association," Proc. European Conf. Computer Vision, 2004.
16. N. Haering, Z. Myles, and N. Lobo, "Locating Dedicuous Trees," Proc. Workshop in Content-Based Access to Image and Video Libraries, pp. 18-25, 1997.
17. J. Hafner, H. Sawhney, W. Equitz, M. Flickner, and W. Niblack, "Efficient Color Histogram Indexing for Quadratic Form Distance Functions," IEEE Trans. Pattern. Analysis and Machine Intelligence, vol. 17, no. 7, pp. 729-736, July 1995.
18. J. Huang, S. Kumar, M. Mitra, W. Zhu, and R. Zabih, "Spatial Color Indexing and Applications," Int'l J. Computer Vision, vol. 35, no. 3, pp. 245-268, Dec. 1999.
19. A. Jain and A. Vailaya, "Image Retrieval Using Color and Shape," Pattern Recognition J., vol. 29, pp. 1233-1244, Aug. 1996.
20. A. Kalai and A. Blum, "A Note on Learning from Multiple Instance Examples," Artificial Intelligence, vol. 30, no. 1, pp. 23-30, 1998.
21. V. Lavrenko, R. Manmatha, and J. Jeon, "A Model for Learning the Semantics of Pictures," Proc. Conf. Advances in Neural Information Processing Systems, 2003.
22. Y. Li and L. Shapiro, "Consistent Line Clusters for Building Recognition in CBIR," Proc. Int'l Conf. Pattern Recognition, vol. 3, pp. 952-956, 2002.
23. J. Li and J. Wang, "Automatic Linguistic Indexing of Pictures by a Statistical Modeling Approach," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 9, pp. 1075-1088, Sept. 2003.
24. F. Liu and R. Picard, "Periodicity, Directionality, and Randomness: Wold Features for Image Modeling and Retrieval," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, no. 3, pp. 722-733, July 1996.
25. B. Manjunath and W. Ma, "Texture Features for Browsing and Retrieval of Image Data," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, no. 8, pp. 837-842, Aug. 1996.
26. R. Manmatha and S. Ravela, "A Syntactic Characterization of Appearance and Its Application to Image Retrieval," Proc. SPIE Conf. Human Vision and Electronic Imaging II, vol. 3016, 1997.
27. O. Maron and T. Lozano-Perez, "A Framework for Multiple-Instance Learning," Proc. Conf. Advances in Neural Information Processing Systems, vol. 10, 1997.
28. O. Maron and A. Ratan, "Multiple-Instance Learning for Natural Scene Classification," Proc. 15th Int'l Conf. Machine Learning, pp. 341-349, 1998.
29. Y. Mori, H. Takahashi, and R. Oka, "Image-to-Word Transformation Based on Dividing and Vector Quantizing Images with Words," Proc. First Int'l Workshop Multimedia Intelligent Storage and Retrieval Management, 1999.
30. W. Niblack, R. Barber, W. Equitz, M. Flickner, E. Glasman, D. Pektovic, P. Yanker, C. Faloutsos, and G. Taubin, "The QBIC Project: Querying Images by Content Using Color, Texture, and Shape," Proc. SPIE Storage and Retrieval for Image and Video Databases, pp. 173-181, 1993.
31. J. Pan, H. Yang, C. Faloutsos, and P. Duygulu, "GCap: Graph-Based Automatic Image Captioning," Proc. Fourth Workshop Multimedia Data and Document Eng. (MDDE '04), 2004.
32. A. Pentland, R. Picard, and S. Sclaroff, "Photobook: Content-Based Manipulation of Image Databases," Int'l J. Computer Vision, vol. 18, no. 3, pp. 233-254, June 1996.
33. R. Picard, "Digital Libraries: Meeting Place for High-Level and Low-Level Vision," Proc. Asian Conf. Computer Vision, Dec. 1995.
34. R. Picard, "Light-Years from Lena: Video and Image Libraries of the Future," Proc. Int'l Conf. Image Processing, Oct. 1995.
35. Y. Rubner, C. Tomasi, and L. Guibas, "A Metric for Distributions with Applications to Image Databases," Proc. Int'l Conf. Computer Vision, 1998.
36. A. Smeulders, M. Worring, S. Santini, A. Gupta, and R. Jain, "Content-Based Image Retrieval: The End of the Early Years," IEEE Trans. Pattern. Analysis and Machine Intelligence, vol. 22, no. 12, pp. 1349-1380, Dec. 2000.
37. J. Smith and S. Chang, "VisualSEEk: A Fully Automated Content-Based Image Query System," ACM Multimedia, pp. 87-98, 1996.
38. J. Smith and S. Chang, "Visually Searching the Web for Content," Multimedia, vol. 4, no. 3, pp. 12-20, July-Sept. 1997.
39. M. Stricker and M. Orengo, "Similarity of Color Images," Proc. SPIE Storage and Retrieval for Image and Video Databases, 1995.
40. M. Szummer and R. Picard, "Indoor-Outdoor Image Classification," Proc. Workshop Content-Based Access to Image and Video Databases, 1998.
41. A. Vailaya, A. Jain, and H. Zhang, "On Image Classification: City vs. Landscape," Pattern Recognition, vol. 31, pp. 1921-1936, Dec. 1998.
42. N. Vasconcelos, "Minimum Probability of Error Image Retrieval," IEEE Trans. Signal Processing, vol. 52, no. 8, Aug. 2004.
43. N. Vasconcelos, "Image Indexing with Mixture Hierarchies," Proc. IEEE Computer Vision and Pattern Recognition Conf., 2001.
44. N. Vasconcelos and M. Kunt, "Content-Based Retrieval from Image Databases: Current Solutions and Future Directions," Proc. Int'l Conf. Image Processing, 2001.
45. N. Vasconcelos and A. Lippman, "Library-Based Coding: A Representation for Efficient Video Compression and Retrieval," Proc. Data Compression Conf., 1997.
46. N. Vasconcelos and M. Vasconcelos, "Scalable Discriminant Feature Selection for Image Retrieval and Recognition," Proc. IEEE Computer Vision and Pattern Recognition Conf., 2004.