Interactive phrases: Semantic descriptions for human interaction recognition

IEEE Transactions on Pattern Analysis and Machine Intelligence

Volumn 36, Issue 9, 2014, Pages 1775-1788

  Kong, Yu ^a   Jia, Yunde ^b   Fu, Yun ^a  

^a Northeastern University   (United States)

^b BEIJING INSTITUTE OF TECHNOLOGY   (China)

Author keywords

action recognition; Human interaction; latent structural SVM

Indexed keywords

ARTIFICIAL INTELLIGENCE; COMPUTER VISION;

ACTION RECOGNITION; DISCRIMINATIVE MODELS; HIGH LEVEL DESCRIPTION; HUMAN INTERACTION RECOGNITION; HUMAN INTERACTIONS; INFORMATION-THEORETIC APPROACH; LATENT STRUCTURAL SVM; SEMANTIC DESCRIPTIONS;

SEMANTICS;

ALGORITHM; AUTOMATED PATTERN RECOGNITION; COMPUTER ASSISTED DIAGNOSIS; HUMAN; HUMAN RELATION; PERCEPTION; PHOTOGRAPHY; PROCEDURES; REPRODUCIBILITY; SEMANTICS; SENSITIVITY AND SPECIFICITY; SOCIAL BEHAVIOR; VIDEORECORDING; WHOLE BODY IMAGING;

ALGORITHMS; HUMANS; IMAGE INTERPRETATION, COMPUTER-ASSISTED; INTERPERSONAL RELATIONS; PATTERN RECOGNITION, AUTOMATED; PHOTOGRAPHY; REPRODUCIBILITY OF RESULTS; SEMANTICS; SENSITIVITY AND SPECIFICITY; SOCIAL BEHAVIOR; SOCIAL PERCEPTION; VIDEO RECORDING; WHOLE BODY IMAGING;

EID: 84905589630     PISSN: 01628828     EISSN: None     Source Type: Journal    
DOI: 10.1109/TPAMI.2014.2303090     Document Type: Article

References (51)

1. J. Liu, B. Kuipers, and S. Savarese, "Recognizing Human Actions by Attributes," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2011
2. J.C. Niebles, C.-W. Chen, and L. Fei-Fei, "Modeling Temporal Structure of Decomposable Motion Segments for Activity Classification," Proc. European Conf. Computer Vision, vol. 6312, 2010
3. Y. Wang and G. Mori, "Hidden Part Models for Human Action Recognition: Probabilistic versus Max-Margin," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 33, no. 7, pp. 1310-8828, July 2010
4. W. Choi and S. Savarese, "A Unified Framework for Multi-Target Tracking and Collective Activity Recognition," Proc. European Conf. Computer Vision, pp. 215-230, 2012
5. T. Lan, Y. Wang, W. Yang, S.N. Robinovitch, and G. Mori, "Discriminative Latent Models for Recognizing Contextual Group Activities," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 34, no. 8, pp. 1549-1562, Aug. 2012
6. A. Patron-Perez, M. Marszalek, I. Reid, and A. Zissermann, "Structured Learning of Human Interaction in TV Shows," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 34, no. 12, pp. 2441-2453, Dec. 2012
7. M. Ryoo and J. Aggarwal, "Stochastic Representation and Recognition of High-Level Group Activities," Intl J. Conputer Vision, vol. 93, pp. 183-200, 2011
8. W. Choi, K. Shahid, and S. Savarese, "Learning Context for Collective Activity Recognition," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2011
9. A. Gupta, A. Kembhavi, and L. Davis, "Observing Human-Object Interactions: Using Spatial and Functional Compatibility for Recognition," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 10, pp. 1775-1789, Oct. 2009
10. B. Yao and L. Fei-Fei, "Recognizing Human-Object Interactions still Images by Modeling the Mutual Context of Objects and Human Poses," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 34, no. 9, pp. 1691-1703, Sept. 2012
11. C. Desai, D. Ramanan, and C. Fowlkes, "Discriminative Models for Static Human-Object Interactions," Proc. IEEE Conf. Computer Vision and Pattern Recognition Workshops (CVPR), 2010
12. V. Ferrari and A. Zisserman, "Learning Visual Attributes," Proc. Advances in Neural Information Processing Systems, 2007
13. C.H. Lampert, H. Nickisch, and S. Harmeling, "Learning to Detect Unseen Object Classes by Between-Class Attribute Transfer," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009
14. Y. Wang and G. Mori, "A Discriminative Latent Model of Object Classes and Attributes," Proc. European Conf. Computer Vision, 2010
15. Y. Kong, Y. Jia, and Y. Fu, "Learning Human Interaction by Interactive Phrases," Proc. European Conf. Computer Vision, 2012
16. P. Felzenszwalb, D. McAllester, and D. Ramanan, "A Discriminatively Trained, Multiscale, Deformable Part Model," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008
17. P. Dollar, V. Rabaud, G. Cottrell, and S. Belongie, "Behavior Recognition via Sparse Spatio-Temporal Features," Proc. IEEE Second Joint Intl Workshop Visual Surveillance and Performance Evaluation of Tracking and Surveillance (VS-PETS), 2005
18. N. Slonim and N. Tishby, "Document Clustering Using Word Clusters via the Information Bottleneck Method," Proc. 23rd Ann. Intl ACM SIGIR Conf. Research and Development in Information Retrieval, 2000
19. E. Shechtman and M. Irani, "Space-Time Behavior Based Correlation," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 405-412, 2005
20. L. Gorelick, M. Blank, E. Shechtman, M. Irani, and R. Basri, "Actions as Space-Time Shapes," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 12, pp. 2247-2253, Dec. 2007 (Pubitemid 350124072)
21. A. Kovashka and K. Grauman, "Learning a Hierarchy of Discriminative Space-Time Neighborhood Features for Human Action Recognition," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 2046-2053, 2010
22. J. Aggarwal and M. Ryoo, "Human Activity Analysis: A Review," ACM Computing Surveys, vol. 43, no. 3, article 16, 2011
23. I. Laptev, M. Marszaek, C. Schmid, and B. Rozenfeld, "Learning Realistic Human Actions from Movies," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008
24. M. Ryoo and J. Aggarwal, "Spatio-Temporal Relationship Match: Video Structure Comparison for Recognition of Complex Human Activities," Proc. IEEE Intl Conf. Computer Vision, pp. 1593-1600, 2009
25. T.-H. Yu, T.-K. Kim, and R. Cipolla, "Real-Time Action Recognition by Spatiotemporal Semantic and Structural Forests," Proc. British Machine Vision Conf., 2010
26. M. Marszaek, I. Laptev, and C. Schmid, "Actions in Context," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009
27. J. Liu, J. Luo, and M. Shah, "Recognizing Realistic Actions from Videos in the Wild," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009
28. N.M. Oliver, B. Rosario, and A.P. Pentland, "A Bayesian Computer Vision System for Modeling Human Interactions," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 831-843, Aug. 2000
29. A. Vahdat, B. Gao, M. Ranjbar, and G. Mori, "A Discriminative Key Pose Sequence Model for Recognizing Human Interactions," Proc. IEEE Intl Conf. Computer Vision Workshops (ICCV Workshops), pp. 1729-1736, 2011
30. B. Ni, S. Yan, and A.A. Kassim, "Recognizing Human Group Activities with Localized Causalities," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1470-1477, 2009
31. R. Li, R. Chellappa, and S.K. Zhou, "Learning Multi-Modal Densities on Discriminative Temporal Interaction Manifold for Group Activity Recognition," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 2450-2457, 2009
32. R. Mehran, A. Oyama, and M. Shah, "Abnormal Crowd Behavior Detection Using Social Force Model," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009
33. B. Yao and L. Fei-Fei, "Modeling Mutual Context of Object and Human Pose in Human-Object Interaction Activities," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 17-24, 2010
34. S. Gong and T. Xiang, "Recognition of Group Activities Using Dynamic Probabilistic Networks," Proc. IEEE Intl Conf. Computer Vision, vol. 2, pp. 742-749, 2003
35. R. Filipovych and E. Ribeiro, "Recognizing Primitive Interactions by Exploring Actor-Object States," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-7, 2008
36. C. Desai, D. Ramanan, and C. Fowlkes, "Discriminative Models for Multi-Class Object Layout," Proc. IEEE Intl Conf. Computer Vision, 2009
37. A. Farhadi, I. Endres, D. Hoiem, and D. Forsyth, "Describing Objects by Their Attributes," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1778-1785, 2009
38. M.A. Sadeghi and A. Farhadi, "Recognition Using Visual Phrases," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2011
39. A. Gupta and L.S. Davis, "Beyond Nouns: Exploiting Prepositions and Comparative Adjectives for Learning Visual Classifiers," Proc. European Conf. Computer Vision, 2008
40. J. Lafferty, A. McCallum, and F. Pereira, "Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data," Proc. Intl Conf. Machine Learning, 2001
41. C. Chow and C. Liu, "Approximating Discrete Probability Distributions with Dependence Tree," IEEE Trans. Information Theory, vol. IT-14, no. 3, pp. 462-467, May 1968
42. B. Taskar, C. Guestrin, and D. Koller, "Max-Margin Markov Networks," Proc. Advances in Neural Information Processing Systems, 2003
43. I. Tsochantaridis, T. Joachims, T. Hofmann, and Y. Altun, "Large Margin Methods for Structured and Interdependent Output Variables," J. Machine Learning Research, vol. 6, pp. 1453-1484, 2005
44. Y. Wang and G. Mori, "Max-Margin Hidden Conditional Random Fields for Human Action Recognition," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 872-879, 2009
45. J. Bradley and C. Guestrin, "Learning Tree Conditional Random Fields," Proc. Intl Conf. Machine Learning, 2010
46. C.-N.J. Yu and T. Joachims, "Learning Structural SVMS with Latent Variables," Proc. Intl Conf. Machine Learning, 2009
47. T.-M.-T. Do and T. Artieres, "Large Margin Training for Hidden Markov Models with Partially Observed States," Proc. Intl Conf. Machine Learning, 2009
48. T. Lan, Y. Wang, G. Mori, and S.N. Robinovitch, "Retrieving Actions in Group Context," Proc. Intl Workshop Sign Gesture Activity, 2010
49. W. Choi, K. Shahid, and S. Savarese, "What Are They Doing?: Collective Activity Classification Using Spatio-Temporal Relationship among People," Proc. IEEE 12th Intl Conf. Computer Vision Workshops (ICCV Workshops), pp. 1282-1289, 2009
50. M.S. Ryoo, "Human Activity Prediction: Early Recognition of Ongoing Activities from Streaming Videos," Proc. IEEE Intl Conf. Computer Vision, 2011
51. M.S. Ryoo and J.K. Aggarwal, "UT-Interaction Dataset, ICPR Contest on Semantic Description of Human Activities (SDHA)," http://cvrc.ece. utexas.edu/SDHA2010/Human-Interaction. html, 2010.