Getting the gist of events: Recognition of two-participant actions from brief displays

Journal of Experimental Psychology: General

Volumn 142, Issue 3, 2013, Pages 880-905

  Hafri, Alon ^a   Papafragou, Anna ^b   Trueswell, John C ^a  

^a UNIVERSITY OF PENNSYLVANIA   (United States)

^b UNIVERSITY OF DELAWARE   (United States)

Author keywords

Action recognition; Event cognition; Event roles; Scene gist; Scene perception

Indexed keywords

ADULT; ARTICLE; ATTENTION; COGNITION; COMPREHENSION; FEMALE; HUMAN; MALE; REACTION TIME; RECOGNITION;

ADULT; ATTENTION; COGNITION; COMPREHENSION; FEMALE; HUMANS; MALE; REACTION TIME; RECOGNITION (PSYCHOLOGY);

EID: 84882738171     PISSN: 00963445     EISSN: None     Source Type: Journal    
DOI: 10.1037/a0030045     Document Type: Article

References (113)

1. Abrams, R. A.,&Christ, S. E. (2003). Motion onset captures attention. Psychological Science, 14, 427-432.
2. Aggarwal, J. K.,&Ryoo, M. S. (2011). Human activity analysis: A review. ACM Computer Surveys, 43(3), 16.
3. Armstrong, S. L., Gleitman, L. R.,&Gleitman, H. (1983). What some concepts might not be. Cognition, 13, 263-308.
4. Baillargeon, R., Li, J., Gertner, Y.,&Wu, D. (2011). How do infants reason about physical events? In U. Goswami (Ed.), The Wiley-Blackwell handbook of childhood cognitive development (2nd ed., pp. 11-48). Oxford, England: Blackwell.
5. Baayen, R. H., Davidson, D. J.,&Bates, D. M. (2008). Mixed-effects modeling with crossed random effects for subjects and items. Journal of Memory and Language, 59, 390-412.
6. Biederman, I., Blickle, T. W., Teitelbaum, R. C.,&Klatsky, G. J. (1988). Object search in nonscene displays. Journal of Experimental Psychology: Learning, Memory, and Cognition, 14, 456-467.
7. Biederman, I.,&Ju, G. (1988). Surface versus edge-based determinants of visual recognition. Cognitive Psychology, 20, 38-64.
8. Biederman, I., Mezzanotte, R. J.,&Rabinowitz, J. C. (1982). Scene perception: Detecting and judging objects undergoing relational violations. Cognitive Psychology, 14, 143-177.
9. Biederman, I., Rabinowitz, J. C., Glass, A. L.,&Stacy, E. W. (1974). On the information extracted from a glance at a scene. Journal of Experimental Psychology, 103, 597-600.
10. Blake, R.,&Shiffrar, M. (2007). Perception of human motion. Annual Review of Psychology, 58, 47-73.
11. Boyce, S. J., Pollatsek, A.,&Rayner, K. (1989). Effect of background information on object identification. Journal of Experimental Psychology: Human Perception and Performance, 15, 556-566.
12. Castelhano, M. S.,&Henderson, J. M. (2008). The influence of color on the perception of scene gist. Journal of Experimental Psychology: Human Perception and Performance, 34, 660-675.
13. Cicchino, J. B., Aslin, R. N.,&Rakison, D. H. (2011). Correspondences between what infants see and know about causal and self-propelled motion. Cognition, 118, 171-192.
14. Cohen, M. A., Alvarez, G. A.,&Nakayama, K. (2011). Natural-scene perception requires attention. Psychological Science, 22, 1165-1172.
15. Cutting, J. E.,&Kozlowski, L. T. (1977). Recognizing friends by their walk: Gait perception without familiarity cues. Bulletin of the Psychonomic Society, 9, 353-356.
16. Davenport, J. L. (2007). Consistency effects between objects in scenes. Memory&Cognition, 35, 393-401.
17. Davenport, J. L.,&Potter, M. C. (2004). Scene consistency in object and background perception. Psychological Science, 15, 559-564.
18. Delaitre, V., Laptev, I.,&Sivic, J. (2010). Recognizing human actions in still images: A study of bag-of-features and part-based representations. In F. Labrosse, R. Zwiggelaar, Y. Liu,&B. Tiddeman (Eds.), Proceedings of the British Machine Vision Conference (pp. 97.1.11-97). Malvern, England: BMVA Press.
19. Dobel, C., Glanemann, R., Kreysa, H., Zwitserlood, P.,&Eisenbeiss, S. (2010). Visual encoding of coherent and non-coherent scenes. In E. Pedersen&J. Bohnemeyer (Eds.), Event representation in language: Encoding events at the language cognition interface (pp. 189-215). Cambridge, England: Cambridge University Press.
20. Dobel, C., Gumnior, H., Bölte, J.,&Zwitserlood, P. (2007). Describing scenes hardly seen. Acta Psychologica, 125, 129-143.
21. Downing, P. E., Jiang, Y., Shuman, M.,&Kanwisher, N. (2001, September 28). A cortical area selective for visual processing of the human body. Science, 293, 2470-2473.
22. Downing, P. E.,&Peelen, M. V. (2011). The role of occipitotemporal body-selective regions in person perception, Cognitive Neuroscience, 2, 186-203.
23. Downing, P. E., Peelen, M. V., Wiggett, A. J.,&Tew, B. D. (2006). The role of the extrastriate body area in action perception. Social Neuroscience, 1, 52-62.
24. Dowty, D. (1991). Thematic proto-roles and argument selection. Language, 67, 547-619.
25. Fabre-Thorpe, M., Delorme, A., Marlot, C.,&Thorpe, S. (2001). A limit to the speed of processing in ultra-rapid visual categorization of novel natural scenes. Journal of Cognitive Neuroscience, 13, 171-180.
26. Fei-Fei, L., Iyer, A., Koch, C.,&Perona, P. (2007). What do we perceive in a glance of a real-world scene? Journal of Vision, 7 (1), 10.
27. Freyd, J. J. (1983). The mental representation of movement when static stimuli are viewed. Perception&Psychophysics, 33, 575-581.
28. Friedman, A. (1979). Framing pictures: The role of knowledge in automated encoding and memory for gist. Journal of Experimental Psychology: General, 108, 316-355.
29. Giese, M. A.,&Poggio, T. (2003). Neural mechanisms for the recognition of biological movements. Nature Reviews Neuroscience, 4, 179-192.
30. Gleitman, L. R., January, D., Nappa, R.,&Trueswell, J. C. (2007). On the give and take between event apprehension and utterance formulation. Journal of Memory and Language, 57, 544-569.
31. Göksun, T., Hirsh-Pasek, K.,&Golinkoff, R. M. (2010). Trading spaces: Carving up events for learning language. Perspectives on Psychological Science, 5, 33-42.
32. Goldin-Meadow, S. (1985). Language development under atypical learning conditions: Replication and implications of a study of deaf children of hearing parents. In K. Nelson (Ed.), Children's language (Vol. 5, pp. 197-245). Hillsdale, NJ: Erlbaum.
33. Golinkoff, R. M. (1975). Semantic development in infants: The concepts of agent and recipient. Merrill-Palmer Quarterly, 21, 181-193.
34. Golinkoff, R. M.,&Kerr, J. L. (1978). Infants' perception of semantically defined action role changes in filmed events. Merrill-Palmer Quarterly, 24, 53-61.
35. Gordon, P. (2003). The origin of argument structure in infant event representations. In A. Brugos, L. Micciulla,&C. E. Smith (Eds.), Proceedings of the 28th Annual Boston University Conference on Language Development (Vol. 1, pp. 189-198). Somerville, MA: Cascadilla Press.
36. Green, C.,&Hummel, J. E. (2006). Familiar interacting object pairs are perceptually grouped. Journal of Experimental Psychology: Human Perception and Performance, 32, 1107-1119.
37. Greene, M. R.,&Oliva, A. (2009a). The briefest of glances: The time course of natural scene understanding. Psychological Science, 20, 464-472.
38. Greene, M. R.,&Oliva, A. (2009b). Recognition of natural scenes from global properties: Seeing the forest without representing the trees. Cognitive Psychology, 58, 137-176.
39. Griffin, Z. M.,&Bock, K. (2000). What the eyes say about speaking. Psychological Science, 11, 274-279.
40. Grossman, E. D.,&Blake, R. (2002). Brain areas active during visual perception of biological motion. Neuron, 35, 1167-1175.
41. Grossman, E. D., Jardine, N. L.,&Pyles, J. A. (2010). fMRI-adaptation reveals invariant coding of biological motion on the human STS. Frontiers in Human Neuroscience, 4, 15.
42. Gupta, A.,&Davis, L. S. (2007). Objects in action: An approach for combining action understanding and object perception. Proceedings of the 2007 IEEE Conference on Computer Vision and Pattern Recognition (pp. 1-8).
43. Gupta, A., Kembhavi, A.,&Davis, L. S. (2009). Observing human-object interactions: Using spatial and functional compatibility for recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31, 1775-1789.
44. Henderson, J. M. (2005). Introduction to real-world scene perception. Visual Cognition, 12, 849-851.
45. Henderson, J. M.,&Ferreira, F. (2004). Scene perception for psycholinguists. In J. M. Henderson&F. Ferreira (Eds.), The interface of language, vision, and action: Eye movements and the visual world (pp. 1-51). New York, NY: Psychology Press.
46. Hillstrom, A. P.,&Yantis, S. (1994). Visual motion and attentional capture. Perception&Psychophysics, 55, 399-411.
47. Hollingworth, A. (2006). Scene and position specificity in visual memory for objects. Journal of Experimental Psychology: Learning, Memory, and Cognition, 32, 58-69.
48. Hollingworth, A.,&Henderson, J. M. (1998). Does consistent scene context facilitate object perception? Journal of Experimental Psychology: General, 127, 398-415.
49. Hollingworth, A.,&Henderson, J. M. (1999). Object identification is isolated from scene semantic constraint: Evidence from object type and token discrimination. Acta Psychologica, 102, 319-343.
50. Huttenlocher, J., Smiley, P.,&Charney, R. (1983). Emergence of action categories in the child: Evidence from verb meanings. Psychological Review, 90, 72-93.
51. Ikizler, N., Cinbis, R. G., Pehlivan, S.,&Duygulu, P. (2008). Recognizing actions from still images. Proceedings of the 19th International Conference on Pattern Recognition.
52. Ikizler-Cinbis, N., Cinbis, R. G.,&Sclaroff, S. (2009). Learning actions from the web. Proceedings of the 12th International Conference on Computer Vision (pp. 995-1002).
53. Intraub, H. (1981). Rapid conceptual identification of sequentially presented pictures. Journal of Experimental Psychology: Human Perception and Performance, 7, 604-610.
54. Intraub, H. (1984). Conceptual masking: The effects of subsequent visual events on memory for pictures. Journal of Experimental Psychology: Learning, Memory, and Cognition, 10, 115-125.
55. Johansson, G. (1973). Visual perception of biological motion and a model for its analysis. Perception&Psychophysics, 14, 201-211.
56. Kable, J. W.,&Chatterjee, A. (2006). Specificity of action representations in the lateral occipitotemporal cortex. Journal of Cognitive Neuroscience, 18, 1498-1517.
57. Kable, J. W., Kan, I. P., Wilson, A., Thompson-Schill, S. L.,&Chatterjee, A. (2005). Conceptual representations of action in the lateral temporal cortex. Journal of Cognitive Neuroscience, 17, 1855-1870.
58. Kable, J. W., Lease-Spellmeyer, J.,&Chatterjee, A. (2002). Neural substrates of action event knowledge. Journal of Cognitive Neuroscience, 14, 795-805.
59. Kersten, A. W.,&Billman, D. (1997). Event category learning. Journal of Experimental Psychology: Learning, Memory, and Cognition, 23, 638-658.
60. Kourtzi, Z.,&Kanwisher, N. (2000). Activation in human MT/MST by static images with implied motion. Journal of Cognitive Neuroscience, 12, 48-55.
61. Lan, T., Wang, Y., Yang, W.,&Mori, G. (2010). Beyond actions: Discriminative models for contextual group activities. In J. Lafferty, C. K. I. Williams, J. Shawe-Taylor, R. S. Zemel,&A. Culotta (Eds.), Advances in neural information processing systems 23. Retrieved from http://books.nips.cc/papers/files/nips23/NIPS2010_0115.pdf
62. Lange, J., Georg, K.,&Lappe, M. (2006). Visual perception of biological motion by form: A template-matching analysis. Journal of Vision, 6 (8), 6.
63. Lange, J.,&Lappe, M. (2006). A model of biological motion perception from configural form cues. Journal of Neuroscience, 26, 2894-2906.
64. Leslie, A. M.,&Keeble, S. (1987). Do six-month-old infants perceive causality? Cognition, 25, 265-288.
65. Macmillan, N. A.,&Creelman, C. D. (2005). Detection theory: A user's guide (2nd ed.). Mahwah, NJ: Erlbaum.
66. Malpass, D.,&Meyer, A. S. (2010). The time course of name retrieval during multiple-object naming: Evidence from extrafoveal-on-foveal effects. Journal of Experimental Psychology: Learning, Memory, and Cognition, 36, 523-537.
67. McRae, K., Ferretti, T.,&Amyote, L. (1997). Thematic roles as verbspecific concepts. Language and Cognitive Processes, 12, 137-176.
68. Meeren, H. K. M., van Heijnsbergen, C. C. R. J.,&de Gelder, B. (2005). Rapid perceptual integration of facial expression and emotional body language. Proceedings of the National Academy of Sciences of the United States of America, 102, 16518-16523.
69. Morgan, J. L.,&Meyer, A. S. (2005). Processing of extrafoveal objects during multiple-object naming. Journal of Experimental Psychology: Learning, Memory, and Cognition, 31, 428-442.
70. Morgan, J. L., van Elswijk, G.,&Meyer, A. S. (2008). Extrafoveal processing of objects in a naming task: Evidence from word probe experiments. Psychonomic Bulletin&Review, 15, 561-565.
71. Moro, V., Urgesi, C., Pernigo, S., Lanteri, P., Pazzaglia, M.,&Aglioti, S. M. (2008). The neural basis of body form and body action agnosia. Neuron, 60, 235-246.
72. Muentener, P.,&Carey, S. (2010). Infants' causal representations of state change events. Cognitive Psychology, 61, 63-86.
73. Newtson, D. (1973). Attribution and the unit of perception of ongoing behavior. Journal of Personality and Social Psychology, 28, 28-38.
74. Newtson, D. (1976). The perceptual organization of ongoing behavior. Journal of Experimental Social Psychology, 12, 436-450.
75. Oliva, A. (2005). Gist of the scene. In L. Itti, G. Rees,&J. K. Tsotsos (Eds.), Neurobiology of attention (pp. 251-257). San Diego, CA: Elsevier Academic Press.
76. Oliva, A.,&Schyns, P. G. (1997). Coarse blobs or fine edges? Evidence that information diagnosticity changes the perception of complex visual stimuli. Cognitive Psychology, 34, 72-107.
77. Oliva, A.,&Torralba, A. (2001). Modeling the shape of the scene: A holistic representation of the spatial envelope. International Journal in Computer Vision, 42, 145-175.
78. Oliva, A.,&Torralba, A. (2007). The role of context in object recognition. Trends in Cognitive Sciences, 11, 520-527.
79. Oram, M. W.,&Perrett, D. I. (1996). Integration of form and motion in the anterior superior temporal polysensory area (STPa) of the macaque monkey. Journal of Neurophysiology, 76, 109-129.
80. Papafragou, A., Hulbert, J.,&Trueswell, J. C. (2008). Does language guide event perception? Evidence from eye movements. Cognition, 108, 155-184.
81. Park, S.,&Aggarwal, J. K. (2000). Recognition of human interaction using multiple features in grayscale images. In Proceedings of the 15th International Conference on Pattern Recognition (Vol. 1, pp. 51-54).
82. Park, S.,&Aggarwal, J. K. (2004). A hierarchical Bayesian network for event recognition of human actions and interactions. Multimedia Systems, 10, 164-179.
83. Potter, M. C. (1975, March 14). Meaning in visual search. Science, 187, 965-966.
84. Potter, M. C. (1976). Short-term conceptual memory for pictures. Journal of Experimental Psychology: Human Learning and Memory, 2, 509-522.
85. Potter, M. C.,&Levy, E. I. (1969). Recognition memory for a rapid sequence of pictures. Journal of Experimental Psychology, 81, 10-15.
86. Pourtois, G., Peelen, M. V., Spinelli, L., Seeck, M.,&Vuilleumier, P. (2007). Direct intracranial recording of body-selective responses in human extrastriate visual cortex. Neuropsychologia, 45, 2621-2625.
87. Rakison, D. H. (2005). A secret agent? How infants learn about the identity of objects in a causal scene. Journal of Experimental Child Psychology, 91, 271-296.
88. Rosch, E. (1973). On the internal structure of perceptual and semantic categories. In T. E. Moore (Ed.), Cognitive development and the acquisition of language (pp. 111-144). New York, NY: Academic Press.
89. Ryoo, M. S.,&Aggarwal, J. K. (2006). Recognition of composite human activities through context-free grammar based representation. Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Vol. 2, pp. 1709-1718).
90. Schyns, P. G.,&Oliva, A. (1994). From blobs to boundary edges: Evidence for time- and spatial-scale-dependent scene recognition. Psychological Science, 5, 195-200.
91. Sekuler, R., Watamaniuk, S. N. J.,&Blake, R. (2002). Visual motion perception. In H. Pashler (Series Ed.)&S. Yantis (Vol. Ed.), Stevens' handbook of experimental psychology: Vol. 1. Sensation and perception (3rd ed., pp. 121-176). New York, NY: Wiley.
92. Shipley, T. F. (2008). An invitation to an event. In T. F. Shipley&J. M. Zacks (Eds.), Understanding events: From perception to action (pp. 3-30). New York, NY: Oxford University Press.
93. Shipley, T. F.,&Zacks, J. M. (Eds.). (2008). Understanding events: From perception to action. New York, NY: Oxford University Press.
94. Singer, J. M.,&Sheinberg, D. L. (2010). Temporal cortex neurons encode articulated actions as slow sequences of integrated poses. Journal of Neuroscience, 30, 3133-3145.
95. Spelke, E. S., Phillips, A. T.,&Woodward, A. L. (1995). Infants' knowledge of object motion and human action. In D. Sperber, D. Premack, & A. Premack (Eds.), Causal cognition: A multidisciplinary debate (pp. 44-77). New York, NY: Oxford University Press.
96. Steiger, J. H., Shapiro, A.,&Browne, M. W. (1985). On the multivariate asymptotic distribution of sequential chi-square statistics. Psychometrika, 50, 253-263.
97. Talmy, L. (2000). Toward a cognitive semantics: Vol. 2. Typology and process in concept structuring. Cambridge, MA: MIT Press.
98. Thierry, G., Pegna, A. J., Dodds, C., Roberts, M., Basan, S.,&Downing, P. (2006). An event-related potential component sensitive to images of the human body. NeuroImage, 32, 871-879.
99. Tranel, D., Kemmerer, D., Adolphs, R., Damasio, H.,&Damasio, A. R. (2003). Neural correlates of conceptual knowledge for actions. Cognitive Neuropsychology, 20, 409-432.
100. Troje, N. F. (2008). Retrieving information from human movement patterns. In T. F. Shipley&J. M. Zacks (Eds.), Understanding events: From perception to action (pp. 308-334). New York, NY: Oxford University Press.
101. Urgesi, C., Candidi, M., Ionta, S.,&Aglioti, S. M. (2007). Representation of body identity and body actions in extrastriate body area and ventral premotor cortex. Nature Neuroscience, 10, 30-31.
102. Vahdat, A., Gao, B., Ranjbar, M.,&Mori, G. (2011). A discriminative key pose sequence model for recognizing human interactions. interactions. Proceedings of the 2011 IEEE International Conference on Computer Vision Workshops (pp. 1729-1736).
103. van Boxtel, J. J. A.,&Lu, H. (2011). Visual search by action category. Journal of Vision, 11 (7), 19.
104. Vangeneugden, J., De Mazière, P. A., Van Hulle, M. M., Jaeggli, T., Van Gool, L.,&Vogels, R. (2011). Distinct mechanisms for coding of visual actions in macaque temporal cortex. Journal of Neuroscience, 31, 385-401.
105. Vangeneugden, J., Pollick, F.,&Vogels, R. (2009). Functional differentiation of macaque visual temporal cortical neurons using a parametric action space. Cerebral Cortex, 19, 593-611.
106. Verfaillie, K.,&Daems, A. (2002). Representing and anticipating human actions in vision. Visual Cognition, 9, 217-232.
107. Wang, Y., Jiang, H., Drew, M. S., Li, Z.-N.,&Mori, G. (2006). Unsupervised discovery of action classes. Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Vol. 2, pp. 1654-1661).
108. Wiggett, A. J.,&Downing, P. E. (2011). Representation of action in occipito-temporal cortex. Journal of Cognitive Neuroscience, 23, 1765-1780.
109. Yang, W., Wang, Y.,&Mori, G. (2010). Recognizing human actions from still images with latent poses. Proceedings of the 2010 IEEE Conference on Computer Vision and Pattern Recognition (pp. 2030-2037).
110. Yao, B.,&Fei-Fei, L. (2010). Modeling mutual context of object and human pose in human-object interaction activities. Proceedings of the 2010 IEEE Conference on Computer Vision and Pattern Recognition (pp.17-24).
111. Zacks, J. (2004). Using movement and intentions to understand simple events. Cognitive Science, 28, 979-1008.
112. Zacks, J. M., Speer, N. K., Swallow, K. M., Braver, T. S.,&Reynolds, J. R. (2007). Event perception: A mind-brain perspective. Psychological Bulletin, 133, 273-293.
113. Zacks, J. M., Tversky, B.,&Iyer, G. (2001). Perceiving, remembering, and communicating structure in events. Journal of Experimental Psychology: General, 130, 29-58.