Exemplar-Based Recognition of Human-Object Interactions

IEEE Transactions on Circuits and Systems for Video Technology

Volumn 26, Issue 4, 2016, Pages 647-660

  Hu, Jian Fang ^a,b   Zheng, Wei Shi ^a   Lai, Jianhuang ^a   Gong, Shaogang ^c   Xiang, Tao ^c  

^a SUN YAT SEN UNIVERSITY   (China)

^b SYSU CMU Shunde International Joint Research Institute   (China)

^c QUEEN MARY UNIVERSITY OF LONDON   (United Kingdom)

Author keywords

Action recognition; exemplar modeling; human object interactions (HOIs)

Indexed keywords

GESTURE RECOGNITION; MOTION ESTIMATION;

ACTION RECOGNITION; EXEMPLAR BASED RECOGNITION; EXEMPLAR MODELS; HUMAN-ACTION RECOGNITION; HUMAN-OBJECT INTERACTION; PROBABILISTIC DENSITY FUNCTION; SPATIAL STRUCTURE INFORMATION; STATE-OF-THE-ART PERFORMANCE;

OBJECT DETECTION;

EID: 84963876679     PISSN: 10518215     EISSN: None     Source Type: Journal    
DOI: 10.1109/TCSVT.2015.2397200     Document Type: Article

References (55)

1. B. Yao and L. Fei-Fei, "Modeling mutual context of object and human pose in human-object interaction activities," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2010, pp. 17-24.
2. A. Prest, C. Schmid, and V. Ferrari, "Weakly supervised learning of interactions between humans and objects," IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 3, pp. 601-614, Mar. 2012.
3. C. Desai, D. Ramanan, and C. Fowlkes, "Discriminative models for static human-object interactions," in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. Workshops (CVPRW), Jun. 2010, pp. 9-16.
4. B. Yao and L. Fei-Fei, "Grouplet: A structured image representation for recognizing human and object interactions," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2010, pp. 9-16.
5. B. Yao and L. Fei-Fei, "Recognizing human-object interactions in still images by modeling the mutual context of objects and human poses," IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 9, pp. 1691-1703, Sep. 2012.
6. S. Maji, L. Bourdev, and J. Malik, "Action recognition from a distributed representation of pose and appearance," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2011, pp. 3177-3184.
7. M. A. Sadeghi and A. Farhadi, "Recognition using visual phrases," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2011, pp. 1745-1752.
8. V. Delaitre, J. Sivic, and I. Laptev, "Learning person-object interactions for action recognition in still images," in Proc. Annu. Conf. Neural Inf. Process. Syst., 2011, pp. 1503-1511.
9. A. Gupta, A. Kembhavi, and L. S. Davis, "Observing human-object interactions: Using spatial and functional compatibility for recognition," IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 10, pp. 1775-1789, Oct. 2009.
10. A. Gupta and L. S. Davis, "Objects in action: An approach for combining action understanding and object perception," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2007, pp. 1-8.
11. A. Prest, V. Ferrari, and C. Schmid, "Explicit modeling of human-object interactions in realistic videos," IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 4, pp. 835-848, Apr. 2013.
12. D. Weinland, R. Ronfard, and E. Boyer, "A survey of vision-based methods for action representation, segmentation and recognition," Comput. Vis. Image Understand., vol. 115, no. 2, pp. 224-241, Feb. 2011.
13. P. Turaga, R. Chellappa, V. S. Subrahmanian, and O. Udrea, "Machine recognition of human activities: A survey," IEEE Trans. Circuits Syst. Video Technol., vol. 18, no. 11, pp. 1473-1488, Nov. 2008.
14. C. Desai and D. Ramanan, "Detecting actions, poses, and objects with relational phraselets," in Proc. 12th Eur. Conf. Comput. Vis., 2012, pp. 158-172.
15. B. Yao and L. Fei-Fei, "Action recognition with exemplar based 2.5D graph matching," in Proc. 12th Eur. Conf. Comput. Vis., 2012, pp. 173-186.
16. W. Yang, Y. Wang, and G. Mori, "Recognizing human actions from still images with latent poses," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2010, pp. 2030-2037.
17. B. Yao, Z. Liu, B. Nie, and S.-C. Zhu, "Animated pose templates for modelling and detecting human actions," IEEE Trans. Pattern Anal. Mach. Intell., vol. 36, no. 3, pp. 436-452, 2014.
18. P. F. Felzenszwalb and D. P. Huttenlocher, "Pictorial structures for object recognition," Int. J. Comput. Vis., vol. 61, no. 1, pp. 55-79, Jan. 2005.
19. L. Bourdev and J. Malik, "Poselets: Body part detectors trained using 3D human pose annotations," in Proc. 12th Int. Conf. Comput. Vis., Sep./Oct. 2009, pp. 1365-1372.
20. V. Delaitre, I. Laptev, and J. Sivic, "Recognizing human actions in still images: A study of bag-of-features and part-based representations," in Proc. Brit. Mach. Vis. Conf., vol. 2. 2010, no. 5, p. 7, doi: 10.5244/C.24.97.
21. L.-J. Li and L. Fei-Fei, "What, where and who? Classifying events by scene and object recognition," in Proc. IEEE 11th Int. Conf. Comput. Vis., Oct. 2007, pp. 1-8.
22. G. Sharma, F. Jurie, and C. Schmid, "Discriminative spatial saliency for image classification," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2012, pp. 3506-3513.
23. G. Sharma, F. Jurie, and C. Schmid, "Expanded parts model for human attribute and action recognition in still images," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2013, pp. 652-659.
24. F. S. Khan, R. M. Anwer, J. van de Weijer, A. D. Bagdanov, A. M. Lopez, and M. Felsberg, "Coloring action recognition in still images," Int. J. Comput. Vis., vol. 105, no. 3, pp. 205-221, Dec. 2013.
25. N. Ikizler-Cinbis, R. G. Cinbis, and S. Sclaroff, "Learning actions from the web," in Proc. IEEE 12th Int. Conf. Comput. Vis., Sep./Oct. 2009, pp. 995-1002.
26. C. Thurau and V. Hlavác, "Pose primitive based human action recognition in videos or still images," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2008, pp. 1-8.
27. A. Gaidon, Z. Harchaoui, and C. Schmid, "Temporal localization of actions with actoms," IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 11, pp. 2782-2795, Nov. 2013.
28. R. Filipovych and E. Ribeiro, "Recognizing primitive interactions by exploring actor-object states," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2008, pp. 1-7.
29. H. Kjellström, J. Romero, and D. Kragić, "Visual object-action recognition: Inferring object affordances from human demonstration," Comput. Vis. Image Understand., vol. 115, no. 1, pp. 81-90, Jan. 2011.
30. A. Jain, A. Gupta, M. Rodriguez, and L. S. Davis, "Representing videos using mid-level discriminative patches," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2013, pp. 2571-2578.
31. M. Rohrbach, S. Amin, M. Andriluka, and B. Schiele, "A database for fine grained activity detection of cooking activities," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2012, pp. 1194-1201.
32. R. Messing, C. Pal, and H. Kautz, "Activity recognition using the velocity histories of tracked keypoints," in Proc. IEEE 12th Int. Conf. Comput. Vis., Sep./Oct. 2009, pp. 104-111.
33. H. Koppula and A. Saxena, "Learning spatio-temporal structure from RGB-D videos for human activity detection and anticipation," in Proc. 30th Int. Conf. Mach. Learn. (ICML), 2013, pp. 792-800.
34. P. Wei, Y. Zhao, N. Zheng, and S.-C. Zhu, "Modeling 4D human-object interactions for event and object recognition," in Proc. IEEE Int. Conf. Comput. Vis., Dec. 2013, pp. 3272-3279.
35. J. Wang, Z. Liu, and Y. Wu, "Learning actionlet ensemble for 3D human action recognition," in Human Action Recognition With Depth Cameras. New York, NY, USA: Springer-Verlag, 2014, pp. 11-40.
36. J. Xiao, J. Hays, K. A. Ehinger, A. Oliva, and A. Torralba, "SUN database: Large-scale scene recognition from abbey to zoo," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2010, pp. 3485-3492.
37. T. Malisiewicz, A. Gupta, and A. A. Efros, "Ensemble of exemplar-SVMs for object detection and beyond," in Proc. IEEE Int. Conf. Comput. Vis., Nov. 2011, pp. 89-96.
38. G. Mori and J. Malik, "Recovering 3D human body configurations using shape contexts," IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 7, pp. 1052-1062, Jul. 2006.
39. W. Brendel and S. Todorovic, "Activities as time series of human postures," in Proc. Eur. Conf. Comput. Vis., 2010, pp. 721-734.
40. B. Yao, J. Ma, and L. Fei-Fei, "Discovering object functionality," in Proc. IEEE Int. Conf. Comput. Vis., Dec. 2013, pp. 2512-2519.
41. J.-F. Hu, W.-S. Zheng, J. Lai, S. Gong, and T. Xiang, "Recognising human-object interaction via exemplar based modelling," in Proc. IEEE Int. Conf. Comput. Vis., Dec. 2013, pp. 3144-3151.
42. B. J. Frey and D. Dueck, "Clustering by passing messages between data points," Science, vol. 315, no. 5814, pp. 972-976, Feb. 2007.
43. J.-S. Wu, W.-S. Zheng, and J.-H. Lai, "Approximate kernel competitive learning," Neural Netw., vol. 63, no. 1, pp. 117-132, Mar. 2015.
44. A. Bosch, A. Zisserman, and X. Muoz, "Image classification using random forests and ferns," in Proc. IEEE 11th Int. Conf. Comput. Vis. (ICCV), Oct. 2007, pp. 1-8.
45. A. Vedaldi and A. Zisserman, "Efficient additive kernels via explicit feature maps," IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 3, pp. 480-492, Mar. 2012.
46. W.-S. Zheng, S. Gong, and T. Xiang, "Quantifying and transferring contextual information in object detection," IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 4, pp. 762-777, Apr. 2012.
47. M. Varma and B. R. Babu, "More generality in efficient multiple kernel learning," in Proc. 26th Annu. Int. Conf. Mach. Learn., 2009, pp. 1065-1072.
48. S. Lazebnik, C. Schmid, and J. Ponce, "Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories," in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., vol. 2. Jun. 2006, pp. 2169-2178.
49. J. Wang, J. Yang, K. Yu, F. Lv, T. Huang, and Y. Gong, "Localityconstrained linear coding for image classification," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2010, pp. 3360-3367.
50. M. Juneja, A. Vedaldi, C. V. Jawahar, and A. Zisserman, "Blocks that shout: Distinctive parts for scene classification," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., vol. 1. Jun. 2013, pp. 923-930.
51. P. F. Felzenszwalb, R. B. Girshick, D. McAllester, and D. Ramanan, "Object detection with discriminatively trained part-based models," IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 9, pp. 1627-1645, Sep. 2010.
52. M. Everingham, L. Van Gool, C. Williams, J. Winn, and A. Zisserman. The PASCAL Visual Object Classes Challenge 2012 (VOC2012) Results. [Online]. Available: http://www.pascal-network.org/challenges/VOC/voc2012/workshop/index.html, accessed 2012.
53. M. Bregonzio, S. Gong, and T. Xiang, "Recognising action as clouds of space-time interest points," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2009, pp. 1948-1955.
54. T. Lan, Y. Wang, W. Yang, S. N. Robinovitch, and G. Mori, "Discriminative latent models for recognizing contextual group activities," IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 8, pp. 1549-1562, Aug. 2012.
55. A. Kläser, M. Marszałek, C. Schmid, and A. Zisserman, "Human focused action localization in video," in Trends and Topics in Computer Vision. New York, NY, USA: Springer-Verlag, 2012, pp. 219-233.