Action Recognition in Still Images with Minimum Annotation Efforts

IEEE Transactions on Image Processing

Volumn 25, Issue 11, 2016, Pages 5479-5490

  Yu, Zhang ^a,b   Li, Cheng ^b   Wu, Jianxin ^c   Cai, Jianfei ^d   Do, Minh N ^e   Lu, Jiangbo ^a  

^a ADVANCED DIGITAL SCIENCES CENTER   (Singapore)

^b BIOINFORMATICS INSTITUTE   (Singapore)

^c NANJING UNIVERSITY   (China)

^d NANYANG TECHNOLOGICAL UNIVERSITY   (Singapore)

^e UNIVERSITY OF ILLINOIS AT URBANA CHAMPAIGN   (United States)

Author keywords

Action recognition; still image; without annotation

Indexed keywords

IMAGE PROCESSING; MATHEMATICAL MODELS;

ACTION RECOGNITION; ACTION RECOGNITION IN STILL IMAGES; HUMAN-ACTION RECOGNITION; HUMAN-OBJECT INTERACTION; RECOGNITION ACCURACY; STATE-OF-THE-ART METHODS; STILL IMAGES; WITHOUT ANNOTATION;

OPTICAL CHARACTER RECOGNITION;

EID: 85027034502     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2016.2605305     Document Type: Article

References (52)

1. R. Poppe, "A survey on vision-based human action recognition, " Image Vis. Comput., vol. 28, no. 6, pp. 976-990, 2010.
2. G. Cheng, Y. Wan, A. N. Saudagar, K. Namuduri, and B. P. Buckles. (2015). "Advances in human action recognition: A survey. " [Online]. Available: https://arxiv. org/abs/1501. 05964
3. M. Everingham, L. V. Gool, C. Williams, J. Winn, and A. Zisserman. The PASCAL Visual Object Classes Challenge 2012 (VOC2012) Results, accessed on Dec. 10, 2015. [Online]. Available: http://www. pascal-network. org/challenges/VOC/voc2012/workshop/index. html
4. J. Wu, Y. Zhang, and W. Lin, "Towards good practices for action video encoding, " in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2014, pp. 2577-2584.
5. T. Zhang, Y. Zhang, J. Cai, and A. C. Kot, "Efficient object feature selection for action recognition, " in Proc. Int. Conf. Acoust., Speech Signal Process., Mar. 2016, pp. 2707-2711.
6. G. Guo and A. Lai, "A survey on still image based human action recognition, " Pattern Recognit., vol. 47, no. 10, pp. 3343-3361, 2014.
7. S. Maji, L. Bourdev, and J. Malik, "Action recognition from a distributed representation of pose and appearance, " in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2011, pp. 3177-3184.
8. M. Hoai, "Regularized max pooling for image categorization, " in Proc. Brit. Mach. Vis. Conf., 2014, p. 1.
9. M. Oquab, L. Bottou, I. Laptev, and J. Sivic, "Learning and transferring mid-level image representations using convolutional neural networks, " in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2014, pp. 1717-1724.
10. S. Gupta and J. Malik. (2015). "Visual semantic role labeling. " [Online]. Available: http://arxiv. org/abs/1505. 04474
11. G. Gkioxari, R. Girshick, and J. Malik, "Contextual action recognition with RCNN, " in Proc. IEEE Int. Conf. Comput. Vis., Dec. 2015, pp. 1080-1088.
12. G. Sharma, F. Jurie, and C. Schmid. "Expanded parts model for semantic description of humans in still images. " [Online]. Available: http://arxiv. org/abs/1509. 04186
13. H. Yang, J. T. Zhou, Y. Zhang, B.-B. Gao, J. Wu, and J. Cai, "Exploit bounding box annotations for multi-label object recognition, " in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2016, pp. 280-288.
14. G. Gkioxari, R. Girshick, and J. Malik, "Actions and attributes from wholes and parts, " in Proc. IEEE Int. Conf. Comput. Vis., Dec. 2015, pp. 2470-2478.
15. A. Prest, C. Schmid, and V. Ferrari, "Weakly supervised learning of interactions between humans and objects, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 3, pp. 601-614, Mar. 2012.
16. K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition, " in Proc. Int. Conf. Learn. Represent., 2015, pp. 1-14.
17. J. Tompson, R. Goroshin, A. Jain, Y. LeCun, and C. Bregler, "Efficient object localization using convolutional networks, " in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2015, pp. 648-656.
18. V. Delaitre, J. Sivic, and I. Laptev, "Learning person-object interactions for action recognition in still images, " in Proc. Adv. Neural Inf. Process. Syst., 2011, pp. 1503-1511.
19. B. Yao and L. Fei-Fei, "Recognizing human-object interactions in still images by modeling the mutual context of objects and human poses, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 9, pp. 1691-1703, Sep. 2012.
20. P. F. Felzenszwalb, R. B. Girshick, D. McAllester, and D. Ramanan, "Object detection with discriminatively trained part-based models, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 9, pp. 1627-1645, Sep. 2010.
21. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "ImageNet classification with deep convolutional neural networks, " in Proc. Adv. Neural Inf. Process. Syst., 2012, pp. 1097-1105.
22. R. Girshick, J. Donahue, T. Darrell, and J. Malik, "Region-based convolutional networks for accurate object detection and segmentation, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 38, no. 1, pp. 142-158, Jan. 2015.
23. Y. Chai, E. Rahtu, V. Lempitsky, L. Van Gool, and A. Zisserman, "TriCoS: A tri-level class-discriminative co-segmentation method for image classification, " in Proc. Eur. Conf. Comput. Vis., 2012, pp. 794-807.
24. C. Rother, V. Kolmogorov, and A. Blake, "'GrabCut': Interactive foreground extraction using iterated graph cuts, " in Proc. SIGGRAPH, 2004, pp. 309-314.
25. M.-M. Cheng, N. J. Mitra, X. Huang, P. H. S. Torr, and S.-M. Hu, "Global contrast based salient region detection, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 37, no. 3, pp. 569-582, Mar. 2015.
26. G. Kim and E. P. Xing, "On multiple foreground cosegmentation, " in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2012, pp. 837-844.
27. T. Ma and L. J. Latecki, "Graph transduction learning with connectivity constraints with application to multiple foreground cosegmentation, " in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2013, pp. 1955-1962.
28. X. Liu, M. Song, D. Tao, J. Bu, and C. Chen, "Random geometric prior forest for multiclass object segmentation, " IEEE Trans. Image Process., vol. 24, no. 10, pp. 3060-3070, Oct. 2015.
29. X. Liu, D. Tao, M. Song, Y. Ruan, C. Chen, and J. Bu, "Weakly supervised multiclass video segmentation, " in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2014, pp. 57-64.
30. L. Zhang, M. Song, Z. Liu, X. Liu, J. Bu, and C. Chen, "Probabilistic graphlet cut: Exploiting spatial structure cue for weakly supervised image segmentation, " in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2013, pp. 1908-1915.
31. W. Ren, K. Huang, D. Tao, and T. Tan, "Weakly supervised large scale object localization with multiple instance learning and bag splitting, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 38, no. 2, pp. 405-416, Feb. 2016.
32. G. Papandreou, L.-C. Chen, K. P. Murphy, and A. L. Yuille, "Weaklyand semi-supervised learning of a deep convolutional network for semantic image segmentation, " in Proc. IEEE Int. Conf. Comput. Vis., Dec. 2015, pp. 1742-1750.
33. M. Oquab, L. Bottou, I. Laptev, and J. Sivic, "Is object localization for free?-Weakly-supervised learning with convolutional neural networks, " in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2015, pp. 685-694.
34. B. Zhou, A. Khosla, A. Lapedriza, A. Oliva, and A. Torralba, "Learning deep features for discriminative localization, " in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2016.
35. A. J. Bency, H. Kwon, H. Lee, S. Karthikeyan, and B. S. Manjunath. (2016). "Weakly supervised localization using deep feature maps. " [Online]. Available: https://arxiv. org/abs/1603. 00489
36. Y. Zhang, J. Wu, and J. Cai, "Compact representation of highdimensional feature vectors for large-scale image recognition and retrieval, " IEEE Trans. Image Process., vol. 25, no. 5, pp. 2407-2419, May 2016.
37. Y. Zhang, J. Wu, J. Cai, and W. Lin, "Flexible image similarity computation using hyper-spatial matching, " IEEE Trans. Image Process., vol. 23, no. 9, pp. 4112-4125, Sep. 2014.
38. Y. Zhang, J. Wu, and J. Cai, "Compact representation for image classification: To choose or to compress?" in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2014, pp. 907-914.
39. L. Niu, J. Cai, and D. Xu, "Domain adaptive fisher vector for visual recognition, " in Proc. Eur. Conf. Comput. Vis., 2016.
40. N. Zhang, J. Donahue, R. Girshick, and T. Darrell, "Part-based R-CNNs for fine-grained category detection, " in Proc. Eur. Conf. Comput. Vis., 2014, pp. 834-849.
41. H. Jegou, M. Douze, and C. Schmid, "Product quantization for nearest neighbor search, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 1, pp. 117-128, Jan. 2011.
42. J. R. R. Uijlings, K. E. A. van de Sande, T. Gevers, and A. W. M. Smeulders, "Selective search for object recognition, " Int. J. Comput. Vis., vol. 104, no. 2, pp. 154-171, Apr. 2013.
43. Y. Zhang et al., "Weakly supervised fine-grained categorization with part-based image representation, " IEEE Trans. Image Process., vol. 25, no. 4, pp. 1713-1725, Apr. 2016.
44. J. Sánchez, F. Perronnin, T. Mensink, and J. Verbeek, "Image classification with the Fisher vector: Theory and practice, " Int. J. Comput. Vis., vol. 105, no. 3, pp. 222-245, 2013.
45. J. Liu, S. Ji, and J. Ye. (2009). SLEP: Sparse learning with efficient projections. Arizona State University. [Online]. Available: http://www. yelab. net/publications/2009-slep. pdf
46. R. Girshick, "Fast R-CNN, " in Proc. IEEE Int. Conf. Comput. Vis., Dec. 2015, pp. 1440-1448.
47. K. He, X. Zhang, S. Ren, and J. Sun, "Spatial pyramid pooling in deep convolutional networks for visual recognition, " in Proc. Eur. Conf. Comput. Vis., 2014, pp. 346-361.
48. R.-E. Fan, K.-W. Chang, C.-J. Hsieh, X.-R. Wang, and C.-J. Lin, "LIBLINEAR: A library for large linear classification, " J. Mach. Learn. Res., vol. 9, pp. 1871-1874, Jun. 2008.
49. B. Yao, X. Jiang, A. Khosla, A. L. Lin, L. Guibas, and L. Fei-Fei, "Human action recognition by learning bases of action attributes and parts, " in Proc. IEEE Int. Conf. Comput. Vis., Nov. 2011, pp. 1331-1338.
50. V. Delaitre, I. Laptev, and J. Sivic, "Recognizing human actions in still images: A study of bag-of-features and part-based representations, " in Proc. Brit. Mach. Vis. Conf., 2010, pp. 1-11. [Online]. Available: http://www. di. ens. fr/willow/research/stillactions/
51. L.-J. Li, H. Su, E. P. Xing, and L. Fei-Fei, "Object bank: A highlevel image representation for scene classification & semantic feature sparsification, " in Proc. Adv. Neural Inf. Process. Syst., 2011, pp. 1-9.
52. J. Wang, J. Yang, K. Yu, F. Lv, T. Huang, and Y. Gong, "Localityconstrained linear coding for image classification, " in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2010, pp. 3360-3367.