Integrating concept ontology and multitask learning to achieve more effective classifier training for multilevel image annotation

IEEE Transactions on Image Processing

Volumn 17, Issue 3, 2008, Pages 407-426

  Fan, Jianping ^a   Gao, Yuli ^a,b   Luo, Hangzai ^a,c  

^a University of North Carolina at Charlotte   (United States)

^b HEWLETT PACKARD LABORATORIES   (United States)

^c EAST CHINA NORMAL UNIVERSITY   (China)

Author keywords

Concept ontology; Hierarchical boosting; Interactive hypotheses assessment; Interconcept visual similarity; Intraconcept visual diversity; Multiple kernel learning; Multitask learning

Indexed keywords

ALGORITHMS; FEATURE EXTRACTION; IMAGING SYSTEMS; ONTOLOGY;

HIERARCHICAL BOOSTING; INTERCONCEPT VISUAL SIMILARITY; INTRACONCEPT VISUAL DIVERSITY; MULTIPLE KERNEL LEARNING; MULTITASK LEARNING;

IMAGE RETRIEVAL;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; AUTOMATED PATTERN RECOGNITION; COMPUTER ASSISTED DIAGNOSIS; DATA BASE; DOCUMENTATION; FACTUAL DATABASE; IMAGE ENHANCEMENT; INFORMATION RETRIEVAL; METHODOLOGY; SYSTEM ANALYSIS;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; DATABASE MANAGEMENT SYSTEMS; DATABASES, FACTUAL; DOCUMENTATION; IMAGE ENHANCEMENT; IMAGE INTERPRETATION, COMPUTER-ASSISTED; INFORMATION STORAGE AND RETRIEVAL; PATTERN RECOGNITION, AUTOMATED; SYSTEMS INTEGRATION;

EID: 40849134401     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2008.916999     Document Type: Article

References (77)

1. Y. Rui, T. S. Huang, and S.-F. Chang, "Image retrieval: Current techniques, promising directions and open issues," J. Vis. Commun. Image Represent., vol. 10, pp. 39-62, 1999.
2. A. W. M. Smeulders, M. Worring, S. Santini, A. Gupta, and R. Jain, "Content-based image retrieval at the end of the early years," IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 12, pp. 1349-1380, Dec. 2000.
3. A. Vailaya, M. Figueiredo, A. K. Jain, and H. J. Zhang, "A Bayesian framework for semantic classification of outdoor vacation images," Proc. SPIE, vol. 3656, 1998.
4. R. Lienhart and A. Hartmann, "Classifying images on the web automatically," J. Electron. Imag., vol. 11, no. 4, pp. 445-454, 2002.
5. J. R. Smith, M. Naphade, and A. Natsev, "Multimedia semantic indexing using model vectors," presented at the IEEE ICME, 2003.
6. C. Breen, L. Khan, and A. Ponnusany, "Image classification using neural networks and ontologies," in Proc. DEXA Workshop, 2002, pp. 98-102.
7. M. Naphade, J. R. Smith, J. Tesic, S.-F. Chang, W. Hsu, L. Kennedy, A. Hauptmann, and J. Curtis, "Large-scale concept ontology for multimedia," IEEE Trans. Multimedia, vol. 13, no. 3, pp. 86-91, Jul.-Sep. 2006.
8. M. Yelizaveta, T.-S. Chua, and R. Jain, "Semi-supervised annotation of brush work in paitings domain using serial combinations of multiple experts," ACM Multimedia, 2006.
9. Y. Wu, B. L. Tseng, and J. R. Smith, "Ontology-based multi-classification learning for video concept detection," in Proc. IEEE ICME, 2004, pp. 1003-1006.
10. M. Naphade and T. S. Huang, "A probabilistic framework for semantic video indexing, filterig and retrieval," IEEE Trans. Multimedia, vol. 3, no. 1, pp. 141-151, Mar. 2001.
11. C. Carson, S. Belongie, H. Greenspan, and J. Malik, "Blobworld: Image segmentation using expectation-maximization and its application to image querying," IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 8, pp. 1026-1038, Aug. 2002.
12. J. Z. Wang, J. Li, and G. Wiederhold, "Simplicity: Semantic-sensitive integrated matching for picture libraries," IEEE Trans. Pattern Anal. Mach. Intell., vol. 23, no. 9, pp. 947-963, Sep. 2001.
13. R. Zhang, Z. Zhang, M. Li, W.-Y. Ma, and H.-J. Zhang, "A probabilistic semantic model for image annotation and multi-modal image retrieval," presented at the IEEE ICCV, 2005.
14. Y. Li, L. G. Shapiro, and J. A. Bilmes, "A generative/discriminative learning algorithm for image classification," presented at the IEEE ICCV, 2005.
15. J. Jeon and R. Manmatha, "Using maximum entropy for automatic image annotation," CIVR, pp. 24-32, 2004.
16. J. Luo, A. E. Savakis, and A. Singhal, "A Bayesian network-based framework for semantic image understanding," Pattern Recognit., vol. 38, no. 6, pp. 919-934, 2005.
17. A. B. Torralba and A. Oliva, "Semantic organization of scenes using discriminant structural templates," presented at the Proc. ICCV, 1999.
18. J. Fan, Y. Gao, and H. Luo, "Multi-level annotation of natural scenes using dominant image compounds and semantic concepts," ACM Multimedia, 2004.
19. W.-Y. Ma and B. S. Manjunath, "Texture features and learning similarity," in Proc. IEEE CVPR, 1996, pp. 425-430.
20. E. Chang, K. Goh, G. Sychay, and G. Wu, "CBSA: Content-based annotation for multimodal image retrieval using Bayes point machines," IEEE Trans. Circuits Syst. Video Technol., vol. 13, no. 1, pp. 26-38, Jan. 2003.
21. A. Oliva and A. Torralba, "Building the gist of a scene: The role of global image features in recognition," Progr. Brain Res.: Vis. Percept., no. 155, pp. 23-36, 2005.
22. D. Lowe, "Distinctive image features from scale invariant keypoints," Int. J. Comput. Vis., vol. 60, pp. 91-110, 2004.
23. P. Quelhas, F. Monay, J.-M. Odobez, D. Gatica-Perez, T. Tuytelaars, and L. J. V. Gool, "Modeling scenes with local descriptors and latent aspects," in Proc. IEEE ICCV, 2005, pp. 883-890.
24. J. Huang, S. R. Kumar, and R. Zabih, "An automatic hierarchical image classification scheme," ACM Multimedia, 1998.
25. N. Vasconcelos, "Image indexing with mixture hierarchies," in Proc. IEEE Trans. CVPR, 2001, pp. I-3-I-10.
26. J. Li and J. Z. Wang, "Automatic linguistic indexing of pictures by a statistical modeling approach," IEEE Trans. Pattern Anal. Mach. Intell., vol. 25, no. 9, pp. 1075-1088, Sep. 2003.
27. J. Fan, H. Luo, Y. Gao, and M.-S. Hacid, "Mining image databases on semantics via statistical learning," ACM SIGKDD, 2005.
28. L. Fei-Fei and P. Perona, "A Bayesian hierarchical model for learning natural scene categories," in Proc. IEEE CVPR, 2005, pp. 524-531.
29. K. Barnard and D. Forsyth, "Learning the semantics of words and pictures," in Proc. IEEE ICCV, 2001, pp. 408-415.
30. R. E. Schapire and Y. Singer, "Boostexter: A boosting-based system for text categorization," Mach. Learn., vol. 39, pp. 135-168, 2000.
31. J. Friedman, T. Hastie, and R. Tibshirani, "Additive logistic regression: A statistical view of boosting," Ann. Statist., vol. 28, no. 2, pp. 337-374, 2000.
32. A. Torralba, K. P. Murphy, and W. T. Freeman, "Sharing features: Efficient boosting procedures for multiclass object detection," in Proc. IEEE CVPR, 2004, pp. II-762-II-769.
33. A. Esuli, T. Fagni, and F. Sebastiani, "TreeBoost.MH: A boosting algorithm for multi-label hierarchical text categorization," presented at the SPIRE, 2006.
34. K. Yu, A. Schwaighofor, V. Tresp, W.-Y. Ma, and H. J. Zhang, "Collaborative ensemble learning: Combining content-based information filtering via hierarchical Bayes," presented at the Int. Conf. Uncertainty in Artificial Intelligence (UAI), 2003.
35. T. Evgeniou, C. A. Micchelli, and M. Pontil, "Learning multiple tasks with kernel methods," J. Mach. Learn. Res., vol. 6, pp. 615-637, 2005.
36. R. K. Ando and T. Zhang, "A framework for learning predictive structures from multiple tasks and unlabeled data," J. Mach. Learn. Res., vol. 6, pp. 1817-1853, 2005.
37. S. Thrun and L. Pratt, Learning to Learn. Norwell, MA: Kluwer, 1997.
38. S. Kumar and M. Hebert, "Discriminative random fields," Int. J. Comput. Vis., vol. 68, no. 2, pp. 179-201, 2006.
39. J. Yang, Y. Liu, E. X. Ping, and A. G. Hauptmann, "Harmonium models for semantic video representation and classification," presented at the SIAM Conf. Data Mining, 2007.
40. M.-Y. Chen and A. G. Hauptmann, "Discriminative fields for modeling semantic concepts in video," presented at the RIAO Large-Scale Semantic Access to Content, May 2007.
41. R. Shi, T.-S. Chua, C.-H. Lee, and S. Gao, "Bayesian learning of hierarchical multinomial mixture models of concepts for automatic image annotation," presented at the CIVR, 2006.
42. W. Jiang, S.-F. Chang, and A. Loui, "Context-based concept fusion with boosted conditional random fields," presented at the IEEE ICASSP, 2007.
43. J. Lafferty, A. McCallum, and F. Pereira, "Conditional random field: Probabilistic models for segmenting and labeling sequence data," presented at the ICML, 2001.
44. S. Tong and E. Y. Chang, "Support vector machine active learning for image retrieval," ACM Multimedia, pp. 107-118, 2001.
45. D. Tao, X. Tang, X. Li, and X. Wu, "Asymmetric bagging and random subspace for support vector machines-based relevance feedback in image retrieval," IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 7, pp. 1088-1099, Jul. 2006.
46. J. Li, N. Allinson, D. Tao, and X. Li, "Multi-training support vector machine for image retrieval," IEEE Trans. Image Process., vol. 15, no. 11, pp. 3597-3601, Nov. 2006.
47. Y. Rubner, C. Tomasi, and L. J. Guibas, "A metric for distributions with applications to image databases," in Proc. IEEE CVPR, 1998, pp. 59-66.
48. V. Vapnik, The Nature of Statistical Learning Theory. New York: Springer-Verlag, 1995.
49. J. C. Platt, "Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods," in Adavances in Large Margin Classifiers. Cambridge, MA: MIT Press, 1999.
50. S. Hoi, M. Lyu, and E. Y. Chang, "Learning the unified kernel machines for classification," ACM SIGKDD, 2006.
51. S. Sonnenburg, G. Ratsch, C. Schafer, and B. Scholkopf, "Large scale multiple kernel learning," J. Mach. Learn. Res., vol. 7, pp. 1531-1565, 2006.
52. T. Evgeniou and M. Pontil, "Regularized multi-task learning," ACM SIGKDD, 2004.
53. T. Lai, J. Tait, and S. McDonald, "Image browsing and navigation using hierarchical classification," CIVR, 1999.
54. S. Santini, A. Gupta, and R. Jain, "Emergent semantics through interaction in image databases," IEEE Trans. Knowl. Data Eng., vol. 13, no. 3, pp. 337-351, Mar. 2001.
55. Y. Rubner, C. Tomasi, and L. Guibas, "A metric for distributions with applications to image databases," in Proc. IEEE ICCV, 1998, pp. 59-66.
56. D. Stan and I. Sethi, "EID: A system for exploration of image databases," Inf. Process. Management, vol. 39, pp. 335-361, 2003.
57. B. Moghaddam, Q. Tian, N. Lesh, C. Shen, and T. S. Huang, "Visualization and user-modeling for browsing personal photo libraries," Int. J. Comput. Vis., vol. 56, pp. 109-130, 2004.
58. J. Lamping and R. Rao, "The hyperbolic browser: A focus + content technique for visualizing large hierarchies," J. Vis. Lang. Comput. vol. 7, pp. 33-55, 1996.
59. J. Fan, Y. Gao, and H. Luo, "Hierarchical classification for automatic image annotation," ACM SIGIR, 2007.
60. A.D. Maedche, Ontology Learning for the Semantic Web. New York: Springer Verlag, 2002.
61. P. Buitelaar, P. Cimiano, and B. Magnini, Ontology Learning from Text: Methods, Evaluation, and Applications. Amsterdam, The Netherlands: ISO Press, 2005.
62. A. B. Benitez, J. R. Smith, and S.-F. Chang, "Medianet: A multimedia information network for knowledge representation," Proc. SPIE, vol. 4210, 2000.
63. A. Jaimes and J. R. Smith, "Semi-automatic, data-driven construction of multimedia ontologies," presented at the IEEE ICME, 2003.
64. M. Swain and D. Ballard, "Color indexing," Int. J. Comput. Vis., 1991.
65. Y. Deng, B. S. Manjunath, C. Kenney, M. S. Moore, and H. Shin, "An efficient color representation for image retrieval," IEEE Trans. Image Process., vol. 10, no. 1, pp. 140-147, Jan. 2001.
66. T. Chang and C. Kou, "Texture analysis and classification with tree-structured wavelet transform," IEEE Trans. Image Process., vol. 2, no. 4, pp. 429-441, Apr. 1993.
67. D. J. Crandall and J. Luo, "Robust color object detection using spatial-color joint probability functions," in Proc. IEEE CVPR, 2004, pp. 379-385.
68. R. Fergus, P. Perona, and A. Zisserman, "Object class recognition by unsupervised scale-invariant learning," Proc. IEEE CVPR, pp. II-264-II-271, 2003.
69. G.-J. Qi, X.-S. Hua, Y. Rui, J. Tang, T. Mei, and H.-J. Zhang, "Correlative multi-label video annotation," ACM Multimedia, pp. 17-26, 2007.
70. D. E. Knuth, "The art of computer programming," in Sorting and Searching. Reading, MA: Addison-Wesley, 1978, vol. 3.
71. M. Girolami, "Mercer kernel-based clustering in feature space," IEEE Trans. Neural Netw., vol. 13, no. 3, pp. 780-784, Jun. 2002.
72. A. Ben-Hur, D. Horn, H. T. Siegelmann, and V. Vapnik, "Support vector clustering," J. Mach. Learn. Res., vol. 2, pp. 125-137, 2001.
73. B. Scholkopf, A. J. Smola, and K.-R. Muller, "Kernel principal component analysis," Neural Comput., vol. 10, no. 5, pp. 1299-1319, 1998.
74. G. Lanchriet, N. Cristianini, P. Bartlett, L. E. Ghaow, and M. I. Jordan, "Learning the kernel matrix with semidefinite programming," J. Med. Learn. Res., vol. 5, pp. 27-72, 2004.
75. R. Schapire, Y. Freund, P. Bartlett, and W. Lee, "Boosting the margin: A new explanation for the effectiveness of voting methods," Ann. Statist., vol. 26, no. 5, pp. 1651-1686, 1998.
76. C. Fellbaum, WordNet: An Electronic Lexical Database. Boston, MA: MIT Press, 1998.
77. J. Fan, Y. Gao, H. Luo, and R. Jain, "Mining multilevel image semantics via hierarchical classification," IEEE Trans. Multimedia, vol. 10, no. 2, pp. 167-187, Feb. 2008.