Discovering thematic objects in image collections and videos

IEEE Transactions on Image Processing

Volumn 21, Issue 4, 2012, Pages 2207-2219

  Yuan, Junsong ^a   Zhao, Gangqiang ^a   Fu, Yun ^b   Li, Zhu ^c   Katsaggelos, Aggelos K ^d   Wu, Ying ^d  

^a NANYANG TECHNOLOGICAL UNIVERSITY   (Singapore)

^b State University of New York   (United States)

^c FUTUREWEI TECHNOLOGIES   (United States)

^d Northwestern University   (United States)

Author keywords

Image data mining; thematic object discovery

Indexed keywords

BOTTOM UP APPROACH; CANDIDATE PRUNING; GENERATIVE MODEL; IMAGE COLLECTIONS; IMAGE DATA; KEY OBJECT; LIGHTING CONDITION CHANGE; OBJECT OF INTERESTS; PRIORI KNOWLEDGE; THEMATIC OBJECT DISCOVERY; TOPDOWN; VIDEO DATA; VIDEO SEQUENCES; VIDEO SUMMARIZATION; VISUAL CONTENT; VISUAL PATTERN;

VIDEO RECORDING;

DATA MINING;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; AUTOMATED PATTERN RECOGNITION; BIOLOGICAL MODEL; COMPUTER ASSISTED DIAGNOSIS; COMPUTER SIMULATION; IMAGE ENHANCEMENT; IMAGE SUBTRACTION; METHODOLOGY; PHOTOGRAPHY; REPRODUCIBILITY; SENSITIVITY AND SPECIFICITY; STATISTICAL MODEL; VIDEORECORDING;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; IMAGE ENHANCEMENT; IMAGE INTERPRETATION, COMPUTER-ASSISTED; MODELS, BIOLOGICAL; MODELS, STATISTICAL; PATTERN RECOGNITION, AUTOMATED; PHOTOGRAPHY; REPRODUCIBILITY OF RESULTS; SENSITIVITY AND SPECIFICITY; SUBTRACTION TECHNIQUE; VIDEO RECORDING;

EID: 84859024517     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2011.2181952     Document Type: Article

References (84)

1. D. Liu and T. Chen, "Discov: A framework for discovering objects in video," IEEE Trans. Multimedia, vol. 10, no. 2, pp. 200-208, Feb. 2008.
2. D. Liu, G. Hua, and T. Chen, "A hierarchical visual model for video object summarization," IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 12, pp. 2178-2190, Dec. 2010.
3. J. Sivic and A. Zisserman, "Video data mining using configurations of viewpoint invariant regions," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2004, pp. I-488-I-495.
4. Y. J. Lee and K. Grauman, "Object-graphs for context-aware category discovery," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2010, pp. 1-8.
5. H.-T. Chen, "Preattentive co-saliency detection," in Proc. IEEE Conf. Image Process., 2010, pp. 1117-1120.
6. M. Cho, Y. M. Shin, and K. M. Lee, "Co-recognition of image pairs by data-driven Monte Carlo image exploration," in Proc. Eur. Conf. Comput. Vis., 2008, pp. 144-157.
7. H. Li and K. N. Ngan, "A co-saliency model of image pairs," IEEE Trans. Image Process., vol. 20, no. 12, pp. 3365-3375, Dec. 2011.
8. J. Sivic, B. C. Russell, A. A. Efros, A. Zisserman, and W. T. Freeman, "Discovering objects and their location in images," in Proc. IEEE Intl. Conf. Comput. Vis., 2005, pp. 370-377. (Pubitemid 44054982)
9. B. C. Russell, A. A. Efros, J. Sivic,W. T. Freeman, and A. Zisserman, "Using multiple segmentation to discover objects and their extent in image collections," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2006, pp. 1605-1614.
10. J. Yuan, Y.Wu, and M. Yang, "From frequent itemsets to semantically meaningful visual patterns," in Proc. Int. Conf. Know. Discov. Data Mining, 2007, pp. 864-873. (Pubitemid 350229271)
11. L. Cao and F.-F. Li, "Spatially coherent latent topic model for concurrent segmentation and classification of objects and scenes," in Proc. IEEE Int. Conf. Comput. Vis., 2007, pp. 1-8.
12. T. Tuytelaars, C. H. Lampert, M. B. Blaschko, and W. Buntine, "Unsupervised object discovery: A comparison," Int. J. Comput. Vis., vol. 88, no. 2, pp. 284-302, Jun. 2010.
13. J. Philbin, O. Chum, M. Isard, J. Sivic, and A. Zisserman, "Lost in quantization: Improving particular object retrieval in large scale image databases," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2008, pp. 1-8.
14. H.-K. Tan and C.-W. Ngo, "Localized matching using earth mover' s distance towards discovery of common patterns from small image samples," Image Vision Comput., vol. 27, no. 10, pp. 1470-1483, Sep. 2009.
15. P.Hong and T. S.Huang, "Spatial pattern discovery by learning a probabilistic parametric model from multiple attributed relational graphs," J. Discrete Appl. Math., vol. 139, no. 1-3, pp. 113-135, Apr. 2004.
16. S. Todorovic and N. Ahuja, "Unsupervised category modeling, recognition, and segmentation in images," IEEE Trans. Pattern Anal. Mach. Intell., vol. 30, no. 12, pp. 2158-2174, Dec. 2008.
17. J. Gao, Y. Hu, J. Liu, and R. Yang, "Unsupervised learning of high-order structural semantics from images," in Proc. IEEE Int. Conf. Comput. Vis., 2009, pp. 2122-2129.
18. H. Liu and S. Yan, "Common visual pattern discovery via spatially coherent correspondences," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2010, pp. 1609-1616.
19. J.Yuan,Z. Li,Y. Fu, Y.Wu, and T. S. Huang, "Common spatial pattern discovery by efficient candidate pruning," in Proc. IEEE Conf. Image Process., 2007, pp. I-165-I-168.
20. G. Zhao and J. Yuan, "Mining and cropping common objects from images," in Proc. ACM Multimedia, 2010, pp. 975-978.
21. J. Yuan and Y. Wu, "Spatial random partition for common visual pattern discovery," in Proc. IEEE Int. Conf. Comput. Vis., 2007, pp. 1-8.
22. O. Chum and J. Matas, "Large scale discovery of spatially related images," IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 2, pp. 371-377, Feb. 2010.
23. K. Heath, N. Gelfand, M. Ovsjanikov,M. Aanjaneya, and L. J. Guibas, "Imagewebs: Computing and exploiting connectivity in image collections," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2010, pp. 3432-3439.
24. S. Bagon, O. Brostovski, M. Galun, and M. Irani, "Detecting and sketching the common," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2010, pp. 33-40.
25. R. Fergus, P. Perona, and A. Zisserman, "Object class recognition by unsupervised scale-invariant learning," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2003, pp. II-264-II-271.
26. M. Weber, M. Welling, and P. Perona, "Unsupervised learning of models for recognition," in Proc. Eur. Conf. Comput. Vis., 2000, pp. 18-32.
27. T. Hofmann, "Unsupervised learning by probabilistic latent semantic analysis," Mach. Learn., vol. 42, no. 1/2, pp. 177-196, Jan./Feb. 2001. (Pubitemid 32872403)
28. D. Blei, A. Ng, and M. Jordan, "Latent dirichlet allocation," J. Mach. Learn. Res., vol. 3, pp. 993-1022, Mar. 2003.
29. T. Quack, V. Ferrari, B. Leibe, and L. Van Gool, "Efficient mining of frequent and distinctive feature configurations," in Proc. IEEE Int. Conf. Comput. Vis., 2007, pp. 1-8.
30. S. Nowozin, K. Tsuda, T. Uno, T. Kudo, and G. Bakir, "Weighted substructure mining for image analysis," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2007, pp. 1-8.
31. K. Grauman and T. Darrell, "Unsupervised learning of categories from sets of partially matching image features," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Nov. 2006, pp. 19-25. (Pubitemid 44931308)
32. Y. N.Wu, Z. Si, H. Gong, and S.-C. Zhu, "Learning active basis model for object detection and recognition," Int. J. Comput. Vis., vol. 90, no. 2, pp. 198-235, Nov. 2010.
33. S. Drouin, P. Hebert, and M. Parizeau, "Incremental discovery of object parts in video sequences," Comput. Vis. Image Understand., vol. 110, no. 1, pp. 60-74, Apr. 2008. (Pubitemid 351375437)
34. D. Lowe, "Distinctive image features from scale-invariant keypoints," Int. J. Comput. Vis., vol. 60, no. 2, pp. 91-110, Nov. 2004.
35. J. Matas, O. Chum, M. Urban, and T. Pajdla, "Robust wide baseline stereo from maximally stable extremal regions," in Proc. Brit. Mach. Vis. Conf., 2002, pp. 384-393.
36. T.H.Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein, Introduction to Algorithms. Cambridge, MA: MIT Press, 2001.
37. C. H. Lampert, M. B. Blaschko, and T. Hofmann, "Beyond sliding windows: Object localization by efficient subwindow search," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2008, pp. 1-8.
38. G.Zhao, J. Yuan, J.Xu, andY.Wu, "Discovering the thematic object in commercial videos," IEEE MultiMedia vol. 18, no. 3, pp. 56-65, Mar. 2011 [Online]. Available: http://dx.doi.org/10.1109/MMUL.2011.40
39. M. Datar, N. Immorlica, P. Indyk, and V. S. Mirrokni, "Locality-sensitive hashing scheme based on p-stable distributions," in Proc. 20th Annu. Symp. Comput. Geometry, 2004, pp. 253-262.
40. S. Todorovic and N. Ahuja, "Extracting subimages of an unknown category from a set of images," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2006, pp. 927-934. (Pubitemid 44931422)
41. P.-E. Forssën and D. Lowe, "Shape descriptors for maximally stable extremal regions," in Proc. IEEE Int. Conf. Comput. Vis., 2007, pp. 1-8.
42. J. Shi and J. Malik, "Normalized cuts and image segmentation," IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 8, pp. 888-905, Aug. 2000.
43. D. Liu and T. Chen, "Discov: A framework for discovering objects in video," IEEE Trans. Multimedia, vol. 10, no. 2, pp. 200-208, Feb. 2008.
44. D. Liu, G. Hua, and T. Chen, "A hierarchical visual model for video object summarization," IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 12, pp. 2178-2190, Dec. 2010.
45. J. Sivic and A. Zisserman, "Video data mining using configurations of viewpoint invariant regions," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2004, pp. I-488-I-495.
46. Y. J. Lee and K. Grauman, "Object-graphs for context-aware category discovery," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2010, pp. 1-8.
47. H.-T. Chen, "Preattentive co-saliency detection," in Proc. IEEE Conf. Image Process., 2010, pp. 1117-1120.
48. M. Cho, Y. M. Shin, and K. M. Lee, "Co-recognition of image pairs by data-driven Monte Carlo image exploration," in Proc. Eur. Conf. Comput. Vis., 2008, pp. 144-157.
49. H. Li and K. N. Ngan, "A co-saliency model of image pairs," IEEE Trans. Image Process., vol. 20, no. 12, pp. 3365-3375, Dec. 2011.
50. J. Sivic, B. C. Russell, A. A. Efros, A. Zisserman, and W. T. Freeman, "Discovering objects and their location in images," in Proc. IEEE Intl. Conf. Comput. Vis., 2005, pp. 370-377. (Pubitemid 44054982)
51. B. C. Russell, A. A. Efros, J. Sivic,W. T. Freeman, and A. Zisserman, "Using multiple segmentation to discover objects and their extent in image collections," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2006, pp. 1605-1614.
52. J. Yuan, Y.Wu, and M. Yang, "From frequent itemsets to semantically meaningful visual patterns," in Proc. Int. Conf. Know. Discov. Data Mining, 2007, pp. 864-873. (Pubitemid 350229271)
53. L. Cao and F.-F. Li, "Spatially coherent latent topic model for concurrent segmentation and classification of objects and scenes," in Proc. IEEE Int. Conf. Comput. Vis., 2007, pp. 1-8.
54. T. Tuytelaars, C. H. Lampert, M. B. Blaschko, and W. Buntine, "Unsupervised object discovery: A comparison," Int. J. Comput. Vis., vol. 88, no. 2, pp. 284-302, Jun. 2010.
55. J. Philbin, O. Chum, M. Isard, J. Sivic, and A. Zisserman, "Lost in quantization: Improving particular object retrieval in large scale image databases," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2008, pp. 1-8.
56. H.-K. Tan and C.-W. Ngo, "Localized matching using earth mover' s distance towards discovery of common patterns from small image samples," Image Vision Comput., vol. 27, no. 10, pp. 1470-1483, Sep. 2009.
57. P.Hong and T. S.Huang, "Spatial pattern discovery by learning a probabilistic parametric model from multiple attributed relational graphs," J. Discrete Appl. Math., vol. 139, no. 1-3, pp. 113-135, Apr. 2004.
58. S. Todorovic and N. Ahuja, "Unsupervised category modeling, recognition, and segmentation in images," IEEE Trans. Pattern Anal. Mach. Intell., vol. 30, no. 12, pp. 2158-2174, Dec. 2008.
59. J. Gao, Y. Hu, J. Liu, and R. Yang, "Unsupervised learning of high-order structural semantics from images," in Proc. IEEE Int. Conf. Comput. Vis., 2009, pp. 2122-2129.
60. H. Liu and S. Yan, "Common visual pattern discovery via spatially coherent correspondences," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2010, pp. 1609-1616.
61. J.Yuan,Z. Li,Y. Fu, Y.Wu, and T. S. Huang, "Common spatial pattern discovery by efficient candidate pruning," in Proc. IEEE Conf. Image Process., 2007, pp. I-165-I-168.
62. G. Zhao and J. Yuan, "Mining and cropping common objects from images," in Proc. ACM Multimedia, 2010, pp. 975-978.
63. J. Yuan and Y. Wu, "Spatial random partition for common visual pattern discovery," in Proc. IEEE Int. Conf. Comput. Vis., 2007, pp. 1-8.
64. O. Chum and J. Matas, "Large scale discovery of spatially related images," IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 2, pp. 371-377, Feb. 2010.
65. K. Heath, N. Gelfand, M. Ovsjanikov,M. Aanjaneya, and L. J. Guibas, "Imagewebs: Computing and exploiting connectivity in image collections," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2010, pp. 3432-3439.
66. S. Bagon, O. Brostovski, M. Galun, and M. Irani, "Detecting and sketching the common," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2010, pp. 33-40.
67. R. Fergus, P. Perona, and A. Zisserman, "Object class recognition by unsupervised scale-invariant learning," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2003, pp. II-264-II-271.
68. M. Weber, M. Welling, and P. Perona, "Unsupervised learning of models for recognition," in Proc. Eur. Conf. Comput. Vis., 2000, pp. 18-32.
69. T. Hofmann, "Unsupervised learning by probabilistic latent semantic analysis," Mach. Learn., vol. 42, no. 1/2, pp. 177-196, Jan./Feb. 2001. (Pubitemid 32872403)
70. D. Blei, A. Ng, and M. Jordan, "Latent dirichlet allocation," J. Mach. Learn. Res., vol. 3, pp. 993-1022, Mar. 2003.
71. T. Quack, V. Ferrari, B. Leibe, and L. Van Gool, "Efficient mining of frequent and distinctive feature configurations," in Proc. IEEE Int. Conf. Comput. Vis., 2007, pp. 1-8.
72. S. Nowozin, K. Tsuda, T. Uno, T. Kudo, and G. Bakir, "Weighted substructure mining for image analysis," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2007, pp. 1-8.
73. K. Grauman and T. Darrell, "Unsupervised learning of categories from sets of partially matching image features," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Nov. 2006, pp. 19-25. (Pubitemid 44931308)
74. Y. N.Wu, Z. Si, H. Gong, and S.-C. Zhu, "Learning active basis model for object detection and recognition," Int. J. Comput. Vis., vol. 90, no. 2, pp. 198-235, Nov. 2010.
75. S. Drouin, P. Hebert, and M. Parizeau, "Incremental discovery of object parts in video sequences," Comput. Vis. Image Understand., vol. 110, no. 1, pp. 60-74, Apr. 2008. (Pubitemid 351375437)
76. D. Lowe, "Distinctive image features from scale-invariant keypoints," Int. J. Comput. Vis., vol. 60, no. 2, pp. 91-110, Nov. 2004.
77. J. Matas, O. Chum, M. Urban, and T. Pajdla, "Robust wide baseline stereo from maximally stable extremal regions," in Proc. Brit. Mach. Vis. Conf., 2002, pp. 384-393.
78. T.H.Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein, Introduction to Algorithms. Cambridge, MA: MIT Press, 2001.
79. C. H. Lampert, M. B. Blaschko, and T. Hofmann, "Beyond sliding windows: Object localization by efficient subwindow search," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2008, pp. 1-8.
80. G.Zhao, J. Yuan, J.Xu, andY.Wu, "Discovering the thematic object in commercial videos," IEEE MultiMedia vol. 18, no. 3, pp. 56-65, Mar. 2011 [Online]. Available: http://dx.doi.org/10.1109/MMUL.2011.40
81. M. Datar, N. Immorlica, P. Indyk, and V. S. Mirrokni, "Locality-sensitive hashing scheme based on p-stable distributions," in Proc. 20th Annu. Symp. Comput. Geometry, 2004, pp. 253-262.
82. S. Todorovic and N. Ahuja, "Extracting subimages of an unknown category from a set of images," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2006, pp. 927-934. (Pubitemid 44931422)
83. P.-E. Forssën and D. Lowe, "Shape descriptors for maximally stable extremal regions," in Proc. IEEE Int. Conf. Comput. Vis., 2007, pp. 1-8.
84. J. Shi and J. Malik, "Normalized cuts and image segmentation," IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 8, pp. 888-905, Aug. 2000.