Image annotation within the context of personal photo collections using hierarchical event and scene models

IEEE Transactions on Multimedia

Volumn 11, Issue 2, 2009, Pages 208-219

  Cao, Liangliang ^a   Luo, Jiebo ^b   Kautz, Henry ^b   Huang, Thomas S ^a  

^a University of Illinois at Urbana Champaign   (United States)

^b EASTMAN KODAK COMPANY   (United States)

Author keywords

Consumer photo collections; CRF; GPS; Scene and event annotation

Indexed keywords

GLOBAL POSITIONING SYSTEM; IMAGE ANALYSIS; INFORMATION RETRIEVAL SYSTEMS; METADATA; ROOM AND PILLAR MINING;

CONDITIONAL RANDOM FIELDS; CONSTRAINED CLUSTERING; CONSUMER PHOTO COLLECTIONS; CONTEXTUAL INFORMATIONS; CRF; EVENT-BASED; GPS; GROUND TRUTHS; HIERARCHICAL DESCRIPTIONS; IMAGE ANNOTATIONS; PERSONAL PHOTO COLLECTIONS; PRACTICAL ISSUES; SCENE AND EVENT ANNOTATION; SCENE MODELS; SUB-COLLECTIONS;

SATELLITE NAVIGATION AIDS;

EID: 59049093475     PISSN: 15209210     EISSN: None     Source Type: Journal    
DOI: 10.1109/TMM.2008.2009693     Document Type: Article

References (45)

1. M. Agarwal and K. Konolige, "Real-time localization in outdoor environments using stereo vision and inexpensive GPS," in Int. Conf. Pattern Recognition, 2006.
2. A. Amir, M. Berg, S.-F. Chang, W. Hsu, G. Iyengar, and C. Lin, "Ibm research trecvid-2003 video retrieval system," NIST TRECVID, 2003.
3. J. Assfalg, M. Bertini, C. Colombo, and A. Bimbo, "Semantic annotation of sports videos," IEEE Trans Multimedia, 2002.
4. Y. Aytar, O. Orhan, and M. Shah. "Improving semantic concept detection and retrieval using contextual estimates," in Proc. ICME, 2007.
5. K. Barnard, P. Duygulu, D. Forsyth, N. de Freitas, D. M. Blei, and M. I. Jordan, "Matching words and pictures," J. Much. Learn. Rese., vol. 3, pp. 1107-1135, 2003.
6. M. Boutell, X. Shen, J. Luo, and C. Brown, "Learning multi-label semantic scene classification," Pattern Recognition, vol. 37, no. 9, pp. 1757-1771, 2004.
7. D. Comaniciu and P. Meer, "Mean shift: A robust approach toward feature space analysis," IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 5, pp. 603-619, 2002.
8. M. Cooper, "Collective media annotation using undirected random field models," in Int. Conf. Semantic Computing, 2007, pp. 337-343.
9. M. Cooper, J. Foote, A. Girgensohn, and L. Wilcox, "Temporal event clustering for digital photo collections," ACM Trans. Multimedia Comput. Commun. Applicat., vol. 1, no. 3, pp. 269-288, 2005.
10. L. Fei-Fei and P. Perona, "A bayesian hierarchical model for learning natural scene categories," in IEEE Conf. Computer Vision and Pattern Recognition, 2005, pp. 524-531.
11. A. Jaffe, M. Naaman, T. Tassa, and M. Davis, "Generating summaries and visualization for large collections of geo-referenced photographs," in ACM Int. Workshop on Multimedia Information Retrieval, 2006, pp. 89-98.
12. W. Jiang, S.-F. Chang, and A. Loui, "Kernel sharing with joint boosting for multi-class concept detection," in Proc. CVPR (Workshop on Semantic Learning in Multimedia), 2007.
13. T. Joachims, "Making large scale SVM learning practical," in Advances in Kernel Methods-Support Vector Learning. Cambridge, MA: MIT Press, 1999.
14. M. Johnson and R. Cipolla, "Improved image annotation and labeling through multi-label boosting." in Brit. Machine Vision Conf., 2005.
15. D. Joshi and J. Luo, "Inferring generic places based on visual content and bag of geotags," in ACM Conf. Image and Video Retrieval, 2008.
16. L. Kennedy, M. Naaman, S. Ahern, R. Nair, and T. Rattenbury, "How flickr helps us make sense of the world: Context and content in communitv-contributed media collections." in ACM Conf. Multimedia, 2007. pp. 631-640.
17. S. Kumar and M. Hebert, "Discriminative random fields," Int. J. Comput. Vis., vol. 68, no. 2, pp. 179-201, 2006.
18. J. Lafferty, A. McCallum, and F. Pereira, "Conditional random fields: Probabilistic models for segmenting and labeling sequence data," in Int. Conf. Machine Learning, 2001, pp. 282-289.
19. S. Lazebnik, C. Schmid, and J. Ponce, "Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories." in IEEE Conf. Computer Vision and Pattern Recognition, 2005.
20. L.-J. Li and L. Fei-Fei, "What, where and who? classifying event by scene and object recognition," in IEEE Int. Conf. Computer Vision, 2007.
21. L. Liao, D. Fox, and H. Kautz, "Location-based activity recognition," in Neural Inform. Process. Syst., 2005.
22. J.-H. Lim, Q. Tian, and P. Mulhem, "Home photo content modeling for personalized event-based retrieval," IEEE Multimedia, vol. 10, no. 4, pp. 28-37, Oct.-Dec. 2008.
23. A. Loui and A. Savakis, "Automated event clustering and quality screening of consumer pictures for digital albuming," IEEE Trans. Multimedia, vol. 5, no. 3, pp. 390-402, Jun. 2003.
24. J. Luo, M. Boutell, and C. Brown. "Pictures are not taken in a vacuum - An overview of exploiting context for semantic scene content understanding," IEEE Signal Process., vol. 23, pp. 101-114, 2006.
25. W. Ma and H. J. Zhang, "Benchmarking of image features for contentbased retrieval," in Proc. Signals, Systems and Computers, 1998.
26. M. Naaman, "Leveraging Geo-Referenced Digital Photographs," Ph.D. dissertation, Stanford Univ., Stanford, CA, 2005.
27. M. Naaman, Y. Song, A. Paepcke, and H. Garcia-Molina, "Automatic organization for digital photographs with geographic coordinates," in Int. Conf. Digital Libraries, 2004, vol. 7, pp. 53-62.
28. C. Pantofaru and M. Hebert, A Comparison of Image Segmentation Algorithms Carnegie Mellon Univ., Pittsburgh, PA, Tech. Rep. CMU-RI-TR-05-40, 2005, pp. 383-394.
29. P. Quelhas, F. Monay, J. Odobez, D. Gatica-Perez, T. Tuytelaars, and L. Van, "Gool. Modeling scenes with local descriptors and latent aspects," in IEEE Int. Conf. Computer Vision, 2005, vol. 1.
30. Y. Rui, T. Huang, and S.-F. Chang, "Image retrieval: Current techniques, promising directions, and open issues." J. Vis. Commun. Image Represent., vol. 10, pp. 39-62, 1999.
31. B. Russell, A. Torralba, K. Murphy, and W. Freeman, "Labelme: A database and web-based tool for image annotation," Int. J. Comput. Vis., 2007.
32. F. Sha and F. Pereira. "Shallow parsing with conditional random fields," in Proc. HLT-NAACL, 2003, vol. 1, pp. 134-141.
33. I. Simon, N. Snavely, and S. Seitz, "Scene summarization for online image collections," in IEEE Int. Conf. Computer Vision, 2007, pp. 1-8.
34. J. Sivic, B. Russell, A. Efros, A. Zisserman, and W. Freeman, "Discovering object categories in image collections," in IEEE Int. Conf. Computer Vision, 2005, vol. 1, no. 65.
35. C. A. Sutton, K. Rohanimanesh, and A. McCallum, "Dynamic conditional random fields: Factorized probabilistic models for labeling and segmenting sequence data," in Int. Conf Machine Learning, 2004.
36. D. Tal and J. Malik, "A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics." in IEEE Proc. Int. Conf. Computer Vision. 2001.
37. K. Tieu and P. Viola, "Boosting image retrieval," Int. J. Comput. Vis., 2004.
38. J. Wang, J. Li, and G. Wiederhold, "SIMPLIcity: Semantics-sensitive integrated matching for picture libraries," IEEE Trans. Pattern Anal. Mack Intell., vol. 23, no. 9, pp. 947-963, 1999.
39. Y. Weiss, "Comparing the Mean Field Method and Belief Propagation for Approximate Inference in MRFs," in Advanced Mean Field Methods, D. Saad and M. Opper, Eds. Cambridge, MA: MIT Press. 2001.
40. Y. Wu, J.-Y. Bouguet, A. Nefian, and I. Kozintsev, "Learning concept templates from web images to query personal image databases," in IEEE Int. Conf. Multimedia and Expo, 2007, pp. 1986-1989.
41. Y. Wu, I. Kozintsev, J.-Y. Bouguet, and C. Dulong, "Sampling strategies for active learning in personal photo retrieval," in IEEE Int. Conf. Multimedia and Expo, 2006, pp. 529-532.
42. R. Yan, M. Chen, and A. Hauptmann, "Mining relationship between video concepts using probabilistic graphical model," in IEEE Int. Conf. Multimedia & Expo, 2006.
43. A. Yavlinsky and D. Heesch, "An online system for gathering imagesimilarity judgments," in ACM Pmc. Multimedia, 2007.
44. J. Yuan, J. Luo, and Y. Wu, "Mining compositional features for boosting," in IEEE Conf. Computer Vision and Pattern Recognition, 2008.
45. L. Zelnik-Manor and M. Irani, "Event-based analysis of video," in IEEE Conf. Computer Vision and Pattern Recognition, 2001.