Ecological interfaces for improving mobile robot teleoperation

IEEE Transactions on Robotics

Volumn 23, Issue 5, 2007, Pages 927-941

  Nielsen, Curtis W ^a   Goodrich, Michael A ^b   Ricks, Robert W ^c  

^a IDAHO NATIONAL LABORATORY   (United States)

^b BRIGHAM YOUNG UNIVERSITY   (United States)

^c US Department of Defence   (United States)

Author keywords

3 D interface; Augmented virtuality; Human robot interaction; Information presentation; Teleoperation; USAR Sim; User study

Indexed keywords

HUMAN COMPUTER INTERACTION; INTERFACES (COMPUTER); MAN MACHINE SYSTEMS; NAVIGATION; REMOTE CONTROL; THREE DIMENSIONAL; VIRTUAL REALITY;

ECOLOGICAL INTERFACES; HUMAN-ROBOT INTERACTION;

MOBILE ROBOTS;

EID: 35348862480     PISSN: 15523098     EISSN: None     Source Type: Journal    
DOI: 10.1109/TRO.2007.907479     Document Type: Conference Paper

References (89)

1. T. B. Sheridan, "Musings on telepresence and virtual presence," Presence: Teleoper, Virtual Environ., vol. 1, no. 1, pp. 120-125, 1992.
2. T. B. Sheridan, Telerobotics, Automation, and Human Supervisory Control. Cambridge, MA: MIT Press, 1992.
3. A. A. Nofi, "Defining and measuring shared situation awareness," Center Naval Anal., Alexandria, VA, Tech. Rep. CRM D0002895, Nov. 2000.
4. M. R. Endsley, "Design and evaluation for situation awareness enhancement," in Proc. Hum. Factors Soc. 32nd Annu. Meet., Santa Monica, CA, 1988, pp. 97-101.
5. J. Casper and R. R. Murphy, "Human-robot interactions during the robot-assisted urban search and rescue response at the world trade center," IEEE Trans. Syst., Man, Cybern. B, vol. 33, no. 3, pp. 367-385, Jun. 2003.
6. J. L. Burke, R. R. Murphy, M. D. Coovert, and D. L. Riddle, "Moonlight in Miami: A field study of human-robot interaction in the context of an urban search and rescue disaster response training exercise," Hum.-Comput. Interact., vol. 19, pp. 85-116, 2004.
7. D. D. Woods, J. Tittle, M. Feil, and A. Roesler, "Envisioning human-robot coordination in future operations," IEEE Trans. Syst., Man, Cybern. C, vol. 34, no. 2, pp. 210-218, May 2004.
8. D. Woods and J. Watts, "How not to have to navigate through too many displays," in Handbook of Human-Computer Interaction, 2nd ed., M. Helander, T. Landauer, and P. Prabhu, Eds. Amsterdam, The Netherlands: Elsevier Science, 1997, pp. 1177-1201.
9. P. L. Alfano and G. E. Michel, "Restricting the field of view: Perceptual and performance effects," Percept. Mot. Skills, vol. 70, no. 1, pp. 35-45, 1990.
10. K. Arthur, "Effects of field of view on performance with head-mounted displays" Ph.D. dissertation, Dept. Comput. Sci., Univ. North Carolina, Chapel Hill, 2000.
11. H. A. Yanco and J. L. Drury, ""Where am I?" Acquiring situation awareness using a remote robot platform," in Proc. IEEE Conf. Syst., Man, Cybern., Oct. 2004, vol. 3, pp. 2835-2840.
12. A. Jacoff, E. Messina, and J. Evans, "A reference test course for autonomous mobile robots," in Proc. SPIE-AeroSense Conf., Orlando, FL, Apr. 2001.
13. J. L. Drury, J. Scholtz, and H. A. Yanco, "Awareness in human-robot interactions," in Proc. IEEE Int. Conf. Syst., Man, Cybern., Washington, DC, Oct. 2003.
14. B. P. DeJong, J. Colgate, and M. A. Peshkin, "Improving teleoperation: Reducing mental rotations and translations," in Proc. IEEE Int. Conf. Robot. Autom., New Orleans, LA, Apr. 2004.
15. J. D. Lee, B. Caven, S. Haake, and T. L. Brown, "Speech-based interaction with in-vehicle computers: The effect of speech-based email on driver's attention to the roadway," Hum. Factors, vol. 43, pp. 631-640, 2001.
16. J. Scholtz, "Human-robot interactions: Creating synergistic cyber forces," presented at the 2002 NRL Workshop Multirobot Syst., Washington, DC, Mar.
17. H. A. Yanco, J. L. Drury, and J. Scholtz, "Beyond usability evaluation: Analysis of human-robot interaction at a major robotics competition," J. Hum.-Comput. Interact., vol. 19, no. 1 and 2, pp. 117-149, 2004.
18. J. L. Burke, R. R. Murphy, E. Rogers, V. J. Lumelsky, and J. Scholtz, "Final report for the DARPA/NSF interdisciplinary study on human-robot interaction," IEEE Trans. Syst., Man, Cybern. C, vol. 34, no. 2, pp. 103-112, May 2004.
19. R. R. Murphy, "Humans, robots, rubble, and research," Interactions vol. 12, no. 2, pp. 37-39, Mar./Apr. 2005.
20. P. Zahorik and R. L. Jenison, "Presence as being-in-the-world," Presence, vol. 7, no. 1, pp. 78-89, Feb. 1998.
21. T. Fong, C. Thorpe, and C. Baur, "A safeguarded teleoperation controller," in Proc. IEEE Int. Conf. Adv. Robot., Budapest, Hungary, Aug. 2001.
22. E. Krotkov, R. Simmons, F. Cozman, and S. Koenig, "Safeguarded teleoperation for lunar rovers: From human factors to field trials," in Proc. IEEE Planetary Rover Technol. Syst. Workshop, Minneapolis, MN, Apr. 1996.
23. J. Bradshaw, M. Sierhuis, A. Acquisti, P. Fetovich, R. Hoffman, R. Jeffers, D. Prescott, N. Suri, A. Uszok, and R. Hoff, "Adjustable autonomy and human-agent teamwork in practice: An interim report on space applications," in Agent Autonomy. Norwell, MA: Kluwer, 2002, pp. 9-38.
24. D. J. Bruemmer, D. Dudenhoeffer, and J. Marble, "Dynamic autonomy for urban search and rescue," presented at the 2002 AAAI Mobile Robot Workshop, Edmonton, Canada, Aug.
25. M. Goodrich, D. Olsen, Jr., J. Crandall, and T. Palmer, "Experiments in adjustable autonomy," in Proc. IJCAI-01 Workshop Auton., Delegation, Control: Interact. Auton. Agents, 2001, pp. 1624-1629.
26. P. Scerri, D. Pynadath, and M. Tambe, "Adjustable autonomy in real-world multi-agent environments," in Proc. 5th Int. Conf. Auton. Agents, Montreal, Canada, 2001.
27. M. Hearst, "Mixed-initiative interaction - Trends and controversies," IEEE Intell. Syst., vol. 14, no. 5, pp. 14-23, Sep./Oct. 1999.
28. P. Kroft and C. Wickens, "Displaying multi-domain graphical database information: An evaluation of scanning, clutter, display size, and user activity," Inf. Des. J., vol. 11, no. 1, pp. 44-52, 2002.
29. T. W. Fong and C. Thorpe, "Vehicle teleoperation interfaces," Auton. Robots, vol. 11, no. 1, pp. 9-18, Jul. 2001.
30. S. Iba, J. Weghe, C. Paredis, and P. Khosla, "An architecture for gesture based control of mombile robots," in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., Kyongju, Korea, Oct. 1999, pp. 851-857.
31. S. Waldherr, R. Romero, and S. Thrun, "A gesture based interface for human-robot interaction," Auton. Robots, vol. 9, no. 2, pp. 151-173, 2000.
32. N. Diolaiti and C. Meichiorri, "Teleoperation of a mobile robot through haptic feedback," in Proc. IEEE Int. Workshop Haptic Virtual Environ. Appl., Ottawa, ON, Canada, Nov. 2002, pp. 67-72.
33. V. Kulyukin, C. Gharpure, and C. Pentico, "Robots as interfaces to haptic and locomotor spaces," in Proc. ACM/IEEE Int. Conf. Hum.-Robot Interact., Arlington, VA, 2007, pp. 325-331.
34. S. Lee, G. S. Sukhatme, G. J. Kim, and C.-M. Park, "Haptic control of a mobile robot," in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., Lausanne, Switzerland, 2002, pp. 2867-2874.
35. L. Yu, P. Tsui, Q. Zhou, and H. Hu, "A web-based telerobotic system for research and education at Essex," in Proc. IEEE/ASME Int. Conf. Adv. Intell. Mechatron., Como, Italy, Jul. 2001.
36. D. Schulz, W. Burgard, D. Fox, S. Thrun, and A. Creemers, "Web interfaces for mobile robots in public places," IEEE Robot. Autom. Mag., vol. 7, no. 1, pp. 48-56, Mar. 2000.
37. G. Chronis and M. Skubic, "Robot navigation using qualitative landmark states from sketched route maps," in Proc. IEEE 2004 Int. Conf. Robot. Autom., New Orleans, LA, pp. 1530-1535.
38. H. Keskinpala, J. Adams, and K. Kawamura, "A PDA-based human robotic interface," in Proc. Int. Conf. Syst., Man, Cybern., Washington, DC, 2003.
39. M. Skubic, D. Perznowski, S. Blisard, A. Schultz, W. Adams, M. Bugajska, and D. Brock, "Spatial language for human-robot dialogs," IEEE Trans. Syst., Man, Cybern. C, vol. 34, no. 2, pp. 154-167, May 2004.
40. T. Fong, C. Thorpe, and C. Baur, "Robot as partner: Vehicle teleoperation with collaborative control," in Proc. 2002 NRL Workshop Multirobot Syst., Washington, DC, Mar.
41. R. R. Murphy and E. Rogers, "Cooperative assistance for remote robot supervision," Presence, vol. 5, no. 2, pp. 224-240, 1996.
42. P. Dourish and V. Bellotti, "Awareness and coordination in shared workspaces," in Proc. ACM Conf. Comput.-Supported Coop. Work, Toronto, ON, Canada, 1992, pp. 107-114.
43. M. R. Endsley, "Automation and situation awareness," in Automation and Human Performance: Theory and Applications, R. Parasuraman and M. Mouloua, Eds. Mahwah, NJ: Lawrence Erlbaum, 1996.
44. C. Wickens, "Situation awareness and workload in aviation," Current Directions Psychol. Sci., vol. 11, no. 4, pp. 128-133, 2002.
45. S. Nayar, "Catadioptric omnidirectional camera," Bell Labs., Holmdel, NJ, Tech. Rep., 1997.
46. G. Thomas, W. D. Robinson, and S. Dow, "Improving the visual experience for mobile robotics," in Proc. 7th Annu. Iowa Space Grant, Des Moines, IA, Nov. 1997, pp. 10-20.
47. G. Thomas, T. Blackmon, M. Sims, and D. Rasmussen, "Video engraving for virtual environments," in Proc. Electron. Imaging: Sci. Technol., San Jose, CA, Feb. 1997.
48. K. Yamazawa, Y. Yagi, and M. Yachida, "Obstacle avoidance with omnidirectional image sensor hyperomni vision," in Proc. IEEE Int. Conf. Robot. Autom., Nagoya, Japan, 1995, pp. 1062-1067.
49. S. Hughes, J. Manojlovich, M. Lewis, and J. Gennari, "Camera control and decoupled motion for teleoperation," in Proc. 2003 IEEE Int. Conf. Syst., Man, Cybern., Washington, DC, Oct. 5-8.
50. B. Keyes, R. Casey, H. Yanco, B. Maxwell, and Y. Georglev, "Camera placement and multi-camera fusion for remote robot operation," in Proc. IEEE Int. Workshop Safety, Security, Rescue Robot., Gaithersburg, MD, Aug. 2006.
51. M. G. Voshell and D. D. Woods, Breaking the keyhole in human-robot coordination: Method and evaluation. Ohio State Univ., Athens, OH, Tech. Rep., 2005.
52. G. Terrien, T. Fong, C. Thorpe, and C. Baur, "Remote driving with a multisensor user interface," presented at the SAE 30th Int. Conf. Environ. Syst., Toulouse, France, 2000.
53. T. W. Fong, C. Thorpe, and C. Baur, "Advanced interfaces for vehicle teleoperation: Collaborative control, sensor fusion displays, and remote driving tools," Auton. Robots, vol. 11, no. 1, pp. 77-85, Jul. 2001.
54. L. A. Nguyen, M. Bualat, L. J. Edwards, L. Flueckiger, C. Neveu, K. Schwehr, M. D. Wagner, and E. Zbinden, "Virtual reality interfaces for visualization and control of remote vehicles," Auton. Robots, vol. 11, no. 1, pp. 59-68, Jul. 2001.
55. C. Stoker, E. Zbinden, T. Blackmon, B. Kanefsky et al., "Analyzing pathfinder data using virtual reality and superresolved imaging," J. Geophys. Res. - Planets, vol. 104, no. E4, pp. 8889-8906, 1999.
56. P. Milgram and F. Kishino, "A taxonomy of mixed reality visual displays," IEICE Trans. Inf. Syst., vol. E77-D, no. 12, pp. 1321-1329, 1994.
57. R. T. Azuma, "A survey of augmented reality," Presence: Teleoper. Virtual Environ., vol. 6, no. 4, pp. 355-385, Aug. 1997.
58. P. Milgram, S. Zhai, D. Drascic, and J. Grodski, "Applications of augmented reality for human-robot communication," in Proc. Int. Conf. Intell. Robots Syst., Yokohama, Japan, Jul. 1993, pp. 1467-1472.
59. M. Usoh, E. Catena, S. Arman, and M. Slater, "Using presence questionnaires in reality," Presence, vol. 9, no. 5, pp. 497-503, Oct. 2000.
60. J. Steuer, "Defining virtual reality: Dimensions determining telepresence," J. Commun., vol. 42, no. 2, pp. 73-93, 1992.
61. G. Mantovani and G. Riva, "Real presence: How different ontologies generate different criteria for presence, telepresence, and virtual presence," Presence: Teleoper. Virtual Environ., vol. 8, no. 5, pp. 538-548, Oct. 1999.
62. W. Sadowsky and K. Stanney, "Measuring and managing presence in virtual environments," in Handbook of Virtual Environments Technology, K. Stanney, Ed. Hillsdale, NJ: Lawrence Erlbaum, 2002.
63. K. Stanney, R. Mourant, and R. Kennedy, "Human factors issues in virtual environments: A review of the literature," Presence: Teleoper. Virtual Environ., vol. 7, no. 1, pp. 327-351, 1998.
64. D. J. Bruemmer, J. L. Marble, D. A. Few, R. L. Boring, M. C. Walton, and C. W. Nielsen, "Shared understanding for collaborative control," IEEE Trans. Syst., Man, Cybern. A, vol. 35, no. 4, pp. 494-504, Jul. 2005.
65. M. Baker, R. Casey, B. Keyes, and H. A. Yanco, "Improved interfaces for human-robot interaction in urban search and rescue," in Proc. IEEE Conf. Syst., Man Cybern., The Hauge, The Netherlands, Oct. 2004, pp. 2960-2965.
66. J. J. Gibson, The Ecological Approach to Visual Perception. Boston, MA: Houghton Mifflin, 1979.
67. D. A. Norman, "Affordance, conventions, and design," Interactions vol. 6, no. 3, pp. 38-43, May/Jun. 1999.
68. R. R. Murphy, "Case studies of applying gibson's ecological approach to mobile robots," IEEE Trans. Syst., Man, Cybern. A, vol. 29, no. 1, pp. 105-111, Jan. 1999.
69. P. Milgram, D. Drascic, J. J. Grodski, A. Restogi, S. Zhai, and C. Zhou, "Merging real and virtual worlds," in Proc. IMAGINA Conf. 1995, Monte Carlo, NV, Feb.
70. D. Drascic and P. Milgram, "Perceptual issues in augmented reality," Proc SPIE: Stereoscopic Displays Virtual Real. Syst. III, vol. 2653, pp. 123-134, 1996.
71. B. W. Ricks, C. W. Nielsen, and M. A. Goodrich, "Ecological displays for robot interaction: A new perspective," in Proc. Int. Conf. Intell. Robots Syst. IEEE/RSJ, Sendai, Japan, 2004.
72. S. G. Hart and L. E. Staveland, "Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research," in Human Mental Workload, P. A. Hancock and N. Meshkati, Eds., North-Holland, Amsterdam, The Netherlands, 1988, pp. 139-183.
73. O. Nakayama, T. Futami, T. Nakamura, and E. Boer, "Development of a steering entropy method for evaluating driver workload," presented at the Int. Congr. Expo., Detroit, MI, Mar. 1999.
74. K. Konolige, "Large-scale map-making," in Proc. Nat. Conf. AAAI, San Jose, CA, 2004.
75. C. W. Nielsen and M. A. Goodrich, "Comparing the usefulness of video and map information in navigation tasks," in Proc. 2006 Hum.-Robot Interact. Conf., Salt Lake City, UT.
76. J. Wang, M. Lewis, and J. Gennari, "A game engine based simulation of the NIST urban search and rescue arenas," in Proc. 2003 Winter Simul. Conf., vol. 1, pp. 1039-1045.
77. M. C. W. Douglas, A. Few, and D. J. Bruemmer, "Improved human-robot teaming through facilitated initiative," in Proc. 15th IEEE Int. Symp. Robot Hum. Interact. Commun., Hatfield, U.K., Sep. 2006.
78. E. B. Pacis, H. Everett, N. Farrington, and D. J. Bruemmer, "Enhancing functionality and autonomy in man-portable robots,," in Proc. SPIE Unmanned Ground Veh. Technol. VI, Defense Security, Orlando, FL, Apr. 2004, pp. 355-366.
79. C. W. Nielsen and M. A. Goodrich, "Testing the usefulness of a pan-tiltzoom (PTZ) camera in human-robot interactions," in Proc. Hum. Factors Ergon. Soc. 50th Annu. Meet., San Francisco, CA, 2006.
80. C. W. Nielsen, M. A. Goodrich, and R. J. Rupper, "Towards facilitating the use of a pan-tilt camera on a mobile robot," in Proc. 14th IEEE Int. Workshop Robot Hum. Interact. Commun., Nashville, TN, 2005.
81. M. A. Goodrich and D. R. Olsen, Jr., "Seven principles of efficient interaction," in Proc. IEEE Int. Conf. Syst., Man, Cybern. Oct. 5-8, 2003, pp. 3943-3948.
82. C. D. Wickens and J. G. Hollands, Engineering Psychology and Human Performance, 3rd ed. Englewood Cliffs, NJ: Prentice-Hall, 1999.
83. M. R. Endsley, B. Bolté, and D. G. Jones, Designing for Situation Awareness. New York: Taylor & Francis, 2003.
84. S. Thrun, "Robotic mapping: A survey," in Exploring Artificial Intelligence New Millennium, G. Lakemeyer and B. Nebel, Eds. San Mateo, CA: Morgan Kaufmann, 2002.
85. B. P. DeJong, J. E. Colgate, and M. A. Peshkin, "Improving teleoperation: Reducing mental rotations and translations," in Proc. Am. Nuclear Soc. 10th Int. Conf. Robot. Remote Syst. Hazard. Environ. Gainesville, FL, Mar. 2004.
86. R. Shepard and J. Metzler, "Mental rotation of three-dimensional objects," Science, vol. 171, pp. 701-703, 1971.
87. M. A. Goodrich, R. J. Rupper, and C. W. Nielsen, "Perceiving head, shoulders, eyes, and toes in augmented virtuality interfaces for mobile robots," in Proc. 14th IEEE Int. Workshop Robot Hum. Interact. Commun., Nashville, TN, 2005.
88. J. W. Crandall, M. A. Goodrich, D. R. Olsen, Jr., and C. W. Nielsen, "Validating human-robot interaction schemes in multi-tasking environments," IEEE Trans. Syst., Man, Cybern. A, vol. 35, no. 4, pp. 438-449, Jul. 2005.
89. D. R. Olsen, Jr. and S. B. Wood, "Fan-Out: Measuring human control of multiple robots," in Proc. SIGCHI Conf. Hum. Factors Comput. Syst. Vienna, Austria, 2004, pp. 231-238.