Gestures over video streams to support remote collaboration on physical tasks

Human-Computer Interaction

Volumn 19, Issue 3, 2004, Pages 273-309

  Fussell, Susan R ^a   Setlock, Leslie D ^a   Yang, Jie ^a   Ou, Jiazhi ^a   Mauer, Elizabeth ^b   Kramer, Adam D I ^a  

^a Carnegie Mellon University   (United States)

^b Aptima Corporation   (Brazil)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER SUPPORTED COOPERATIVE WORK; GESTURE RECOGNITION; INTERACTIVE COMPUTER SYSTEMS; THREE DIMENSIONAL COMPUTER GRAPHICS; USER INTERFACES; VIDEO CONFERENCING;

COLLABORATIVE PHYSICAL TASK; REMOTE GESTURE; THREE DIMENSIONAL OBJECTS; VIDEO SYSTEMS;

HUMAN COMPUTER INTERACTION;

EID: 4944220717     PISSN: 07370024     EISSN: None     Source Type: Journal    
DOI: 10.1207/s15327051hci1903_3     Document Type: Article

References (52)

1. Alibali, M. W., Health, D. C., & Myers, H. J. (2001). Effects of visibility between speaker and listener on gesture production: Some gestures are meant to be seen. Journal of Memory and Language, 44, 169-188.
2. Bauer, M., Kortuem, G., & Segall, Z. (1999). "Where are you pointing at?" A study of remote collaboration in a wearable video conference system. Proceedings of the Third International Symposium on Wearable Computers (ISWC'99), 151-158. Piscataway, NJ: IEEE Press.
3. Bekker, M. M., Olson, J. S., & Olson, G. M. (1995). Analysis of gestures in face-to-face design teams provides guidance for how to use groupware in design. Proceedings of the DIS 1995 Conference on Designing Interactive Systems, 157-166. New York: ACM.
4. Bly, S. A., & Minneman, S. L. (1990). Commune: A shared drawing surface. Proceedings of the OIS 1990 Conference on Office Information Systems, 184-192. New York: ACM.
5. Bolt, R. (1980). "Put-that-there": Voice and gesture at the graphics interface. In Proceedings of the SIGGRAPH 1980 Conference on Computer Graphics and Interactive Techniques, 262-270. New York: ACM.
6. Brinck, T., & Gomez, L. M. (1992). A collaborative medium for the support of conversational props. Proceedings of the CSCW 1992 Conference on Computer Supported Cooperative Work, 171-178. New York: ACM.
7. Cassell, J. (1998). A framework for gesture generation and interpretation. In R. Cipolla & A. Pentland (Eds.), Computer vision in human-machine interaction (pp. 191-215). Cambridge, UK: Cambridge University Press.
8. Clark, H. H. (1996). Using language. Cambridge, UK: Cambridge University Press.
9. Clark, H. H., & Brennan, S. E. (1991). Grounding in communication. In L. B. Resnick, R. M. Levine, & S. D. Teasley (Eds.), Perspectives on socially shared cognition (pp. 127-149). Washington, DC: APA.
10. Clark, H. H., & Marshall, C. E. (1981). Definite reference and mutual knowledge. In A. K.Joshi, B. L. Webber, & I. A. Sag (Eds.), Elements of discourse understanding (pp. 10-63). Cambridge, UK: Cambridge University Press.
11. Clark, H., & Wilkes-Gibbs, D. (1986). Referring as a collaborative process. Cognition, 22, 1-39.
12. Efron, D. (1941). Gesture and environment. New York: King's Crown Press.
13. Ekman, P., & Friesen, W. (1969). The repertoire of nonverbal behavioral categories: Origins, usage, and coding. Semiotica, 1, 49-98.
14. Emmorey, K., & Casey, S. (2001). Gesture, thought and spatial language. Gesture, 13, 35-50.
15. Flor, N. V. (1998). Side-by-side collaboration: A case study. International Journal of Human-Computer Studies, 49, 201-222.
16. Ford, C. E. (1999). Collaborative construction of task activity: Coordinating multiple resources in a high school physics lab. Research on Language and Social Interaction, 32, 369-408.
17. Fussell, S. R., & Krauss, R. M. (1992). Coordination of knowledge in communication: Effects of speakers' assumptions about what others know. Journal of Personality and Social Psychology, 62, 378-391.
18. Fussell, S. R., Kraut, R. E., & Siegel, J. (2000). Coordination of communication: Effects of shared visual context on collaborative work. Proceedings of the CSCW 2000 Conference on Computer Supported Cooperative Work, 21-30. New York: ACM.
19. Fussell, S. R., Setlock, L. D., & Kraut, R. E. (2003). Effects of head-mounted and scene-oriented video systems on remote collaboration on physical tasks. Proceedings of the CHI 2003 Conference on Human Factors in Computing Systems, 513-520. New York: ACM.
20. Fussell, S. R., Setlock, L. S., & Parker, E. M. (2003). Where do helpers look? Gaze targets during collaborative physical tasks. Extended Abstracts of the CHI 2003 Conference on Human Factors in Computing Systems, 768-769. New York: ACM.
21. Gaver, W., Sellen, A., Heath, C., & Luff, P. (1993). One is not enough: Multiple views in a media space. Proceedings of the Interchi 1993 International Conference on Human Factors in Computing Systems, 335-341. New York: ACM.
22. Goodwin, C. (1996). Professional vision. American Anthropologist, 96, 606-633.
23. Greenberg, S., Gutwin, C., & Roseman, M. (1996). Semantic telepointers for groupware. Proceedings OzCHI '96 Australian Conference on Computer-Human Interaction, 54-61.
24. Greenberg, S., & Roseman, M. (1996). Group Web: A WWW browser as real time groupware. Companion Proceedings of the CHI 1996 Conference on Human Factors in Computing Systems, 271-272. New York: ACM.
25. Gullberg, M. (2003). Eye movements and gestures in human face-to-face interaction. In J. Hyona, R. Radach, & H. Deubel (Eds.), The mind's eyes: Cognitive and applied aspects of eye movements (pp. 685-703). Oxford, England: Elsevier Science.
26. Gullberg, M., & Holmqvist, K. (1999). Keeping an eye on gestures: Visual perception of gestures in face-to-face communication. Pragmatics and Cognition, 7, 35-63.
27. Gutwin, C. (2001). The effects of network delays on group work in real-time groupware. Proceedings of the ECSCW 2001 European Conference on Computer Supported Cooperative Work, 299-318. Bonn, Germany: Kluver.
28. Gutwin, C., & Penner, R. (2002). Improving interpretation of remote gestures with telepointer traces. Proceedings of the CSCW 2002 Conference on Computer Supported Cooperative Work, 49-57. New York: ACM.
29. Ishii, H., Kobayashi, M., & Grudin, J. (1993). Integration of interpersonal space and shared workspace: ClearBoard design and experiments. ACM Transactions on Information Systems, 11, 349-375.
30. Jefferson, G. (1972). Side sequences. In D. Sudnow (Ed.), Studies in social interaction (pp. 294-338). New York: Free Press.
31. Karsenty, L. (1999). Cooperative work and shared visual context: An empirical study of comprehension problems in side-by-side and remote help dialogues. Human-Computer Interaction, 14, 283-315.
32. Kendon, A. (1972). Some relationships between body motion and speech. In A. R. Siegman & B. Pope (Eds.), Studies in dyadic communication (pp. 177-210). Elmsford, NY: Pergamon.
33. Kraut, R. E., Fussell, S. R., & Siegel, J. (2003). Visual information as a conversational resource in collaborative physical tasks. Human-Computer Interaction, 18, 13-49.
34. Kraut, R. E., Gergle, D., & Fussell, S. R. (2002). The use of visual information in shared visual spaces: Informing the development of virtual co-presence. Proceedings of the CSCW 2002 Conference on Computer Supported Cooperative Work, 31-40. New York: ACM.
35. Kraut, R. E., Miller, M. D., & Siegel, J. (1996). Collaboration in performance of physical tasks: Effects on outcomes and communication. Proceedings of the CSCW 1996 Conference on Computer Supported Cooperative Work, 57-66. New York: ACM.
36. Kuzuoka, H., Kosuge, T., & Tanaka, K. (1994). GestureCam: A video communication system for sympathetic remote collaboration. Proceedings of the CSCW 1994 Conference on Computer Supported Cooperative Work, 35-43. New York: ACM.
37. Kuzuoka, H., Oyama, S., Yamazaki, K., Suzuki, K., & Mitsuishi, M. (2000). GestureMan: A mobile robot that embodies a remote instructor's actions. Proceedings of the CSCW 2000 Conference on Computer Supported Cooperative Work, 155-162. New York: ACM.
38. Kuzuoka, H., & Shoji, H. (1994). Results of observational studies of workspace collaboration. Electronics and Communication in Japan, 77, 58-68.
39. McNeill, D. (1992). Hand and mind: What gestures reveal about thought. Chicago: University of Chicago Press.
40. Minneman, S. L., & Bly, S. A. (1991). Managing a trios: A study of a multi-user drawing tool in distributed design work. Proceedings of the CHI 1991 Conference on Human Factors in Computing Systems, 217-224. New York: ACM.
41. Ou, J., Fussell, S. R., Chen, X., Setlock, L. D., & Yang, J. (2003). Gestural communication over video stream: Supporting multimodal interaction for remote collaborative physical tasks. Proceedings of ICMI 2003 5th International Conference on Multimodal Interfaces, 242-249. New York: ACM.
42. Pedersen, E. R., McCall, K., Moran, T. P., & Halasz, F. G. (1993). Tivoli: An electronic whiteboard for informal workgroup meetings. Proceedings of InterCHI 93, 391-398. New York: ACM.
43. Roussel, N. (2001). Exploring new uses of video with videoSpace. In R. Little & L. Nigay (Eds.), Proceedings of EHCI 01, the 8th IFIP International Conference on Engineering for Human-Computer Interaction, 73-90. Heildelberg: Springer.
44. Sacks, H., Schegloff, E. A., & Jefferson, G. (1974). A simplest systematics for the organization of turn-taking in conversation. Language, 50, 696-735.
45. Siegel, J., Kraut, R. E., John, B. E., & Carley, K. M. (1995). An empirical study of collaborative wearable computer systems. Conference Companion to the CHI 1995 Conference on Human Factors in Computing Systems, 312-313. New York: ACM.
46. Stefik, M., Foster, G., Bobrow, D., Kahn, K., Lanning, S., & Suchman, L. (1987). Beyond the chalkboard: Computer support for collaboration and problem solving in meetings. Communications of the ACM, 30, 32-47.
47. Streitz, N. A., Geissler, J., Haake, J. M., & Hol, J. (1994). DOLPHIN: Integrated meeting support across local and remote desktop environments and liveboards. Proceedings of the CSCW 1994 Conference on Computer Supported Cooperative Work, 345-358. New York: ACM.
48. Tang, J. C. (1991). Findings from observational studies of collaborative work. International Journal of Man-Machine Studies, 34, 143-160.
49. Tang, J. C., & Leifer, L. J. (1988). A framework for understanding the workspace activity of design teams. Proceedings of the CSCW 1988 Conference on Computer Supported Cooperative Work, 244-249. New York: ACM.
50. Tang, J. C., & Minneman, S. L. (1991). VideoDraw: A video interface for collaborative drawing. ACM Transactions on Information Systems, 9, 170-184.
51. Wolf, C. G., & Rhyne,J. R. (1993). Gesturing with shared drawing tools. Adjunct proceedings of the Conference on Human Factors in Computing Systems (Interact93), 137-138. New York: ACM.
52. Wolf, C. G., Rhyne, J. R., & Briggs, L. K. (1992). Communication and information retrieval with a pen-based meeting support tool. Proceedings of the CSCW 1992 Conference on Computer Supported Cooperative Work,