3-D integration of robot vision and laser data with semiautomatic calibration in augmented reality stereoscopic visual interface

IEEE Transactions on Industrial Informatics

Volumn 8, Issue 1, 2012, Pages 69-77

  Livatino, Salvatore ^a   Bannò, Filippo ^b   Muscato, Giovanni ^c  

^a UNIVERSITY OF HERTFORDSHIRE   (United Kingdom)

^b SCUOLA SUPERIORE SANT ANNA   (Italy)

^c UNIVERSITY OF CATANIA   (Italy)

Author keywords

Augmented reality; multiple sensors; stereo vision; telerobotics

Indexed keywords

3-D INTEGRATION; 3-D SPACE; 3D GRAPHICS; GRAPHIC ELEMENTS; INNOVATIVE APPROACHES; LASER DATA; LASER MEASUREMENTS; LASER SENSOR; MULTIPLE SENSORS; REAL OBJECTS; REALITY VISUALIZATION; STEREOSCOPIC VIEWING; TELE-ROBOTICS; TELEROBOTIC SYSTEMS; VIDEO INFORMATION; VISUAL INTERFACE;

AUGMENTED REALITY; COMPUTER VISION; DATA VISUALIZATION; INDUSTRIAL ROBOTS; REMOTE CONTROL; SENSORS; THREE DIMENSIONAL; VISUALIZATION;

STEREO VISION;

EID: 84856353445     PISSN: 15513203     EISSN: None     Source Type: Journal    
DOI: 10.1109/TII.2011.2174062     Document Type: Article

References (45)

1. R. Murphy, "Human-robot interaction in rescue robotics," IEEE Trans. Syst., Man, Cybern., C, Appl. Rev., vol. 34, no. 2, pp. 138-153, May 2004.
2. Z. Zheng, M. Shugen, L. Zhenli, and C. Binggang, "Communication mechanism study of a multi-robot planetary exploration system," in IEEE Int. Conf. Robot. Biomimet., Dec. 2006, pp. 49-54.
3. G. Muscato, D. Caltabiano, S. Guccione, D. Longo, M. Coltelli, A. Cristaldi, E. Pecora, V. Sacco, P. Sim, G. S. Virk, P. Briole, A. Semerano, and T. White, "ROBOVOLC: A robot for volcano exploration result of first test campaign," J. Ind. Robot., vol. 30, no. 3, pp. 231-242, 2003.
4. R. Safaric, M. Debevc, R. M. Parkin, and S. Uran, "Telerobotics experiments via internet," IEEE Trans. Ind. Electron., vol. 48, no. 2, pp. 424-431, Apr. 2001. (Pubitemid 32419018)
5. A. Kouba, R. Severino, M. Alves, and E. Tovar, "Improving quality-ofservice in wireless sensor networks by mitigating hidden-node collisions," IEEE Trans. Ind. Informat., vol. 56, no. 3, pp. 299-313, Aug. 2009.
6. X. Shao and D. Sun, "Development of a new robot controller architecture with FPGA-based IC design for improved high-speed performance," IEEE Trans. Ind. Informat., vol. 54, no. 4, pp. 312-321, Nov. 2007.
7. R. Wirz, R. Marin, M. Ferre, J. Barrio, J. M. Claver, and J. Ortego, "Bidirectional transport protocol for teleoperated robots," IEEE Trans. Ind. Electron., vol. 56, no. 9, pp. 3772-3781, Sep. 2009.
8. P. Arena, P. Di Giamberardino, L. Fortuna, F. La Gala, S. Monaco, G. Muscato, A. Rizzo, and R. Ronchini, "Toward a mobile autonomous robotic system for Mars exploration," J. Planet. Space Sci., vol. 52, pp. 23-30, 2004.
9. T. Chang, P. Jaroonsiriphan, M. Bernhardt, and P. Ludden, "Web-based command shaping of cobra 600 robot with a swinging load," IEEE Trans. Ind. Informat., vol. 2, no. 1, pp. 59-69, Feb. 2006.
10. S. Bernardi and J. Campos, "Computation of performance bounds for real-time systems using time Petri Nets," IEEE Trans. Ind. Informat., vol. 5, no. 2, pp. 168-180, May 2009.
11. R. Bautista-Quintero and M. J. Pont, "Implementation of H-infinity control algorithms for sensor-constrained mechatronic systems using low-cost microcontrollers," IEEE Trans. Ind. Informat., vol. 4, no. 3, pp. 175-184, Aug. 2008.
12. G. Astuti, G. Giudice, D. Longo, C. Melita, and G. Muscato, "An overview of the "volcan project": An UAS for exploration of volcanic environments," J. Intell. Robot. Syst., vol. 54, pp. 471-494, 2009.
13. S. Livatino, G. Muscato, C. Koeffel, S. Sessa, C. Arena, A. Pennisi, D. Di Mauro, and E. Malkondu, "Mobile robotic teleguide based on video images," IEEE Robot. Autom. Mag., vol. 14, no. 4, pp. 58-67, Dec. 2008.
14. L. J. Corde, C. R. Caringnan, B. R. Sullivan, D. L. Akin, T. Hunt, and R. Cohen, "Effects of time delay on telerobotic control of neural buoyancy," in Proc. IEEE Int. Conf. Robot. Autom., Washington, DC, 2002, pp. 2874-2879. (Pubitemid 34916322)
15. S. Livatino, G. Muscato, S. Sessa, and V. Neri, "Depth-enhanced mobile robot teleguide based on laser images," Spec. Issue Design Control Methodol. Telerobot., Elsevier Mechatron. J., vol. 20, no. 7, pp. 739-750, 2010.
16. F. Zeiger, M. Schmidt, and K. Schilling, "Remote experiments with mobile-robot hardware via internet at limited link capacity," IEEE Trans. Ind. Electron., vol. 56, no. 12, pp. 4798-4805, Dec. 2009.
17. H. Baltzakis, A. Argyros, and P. Trahanias, "Fusion of Laser and Visual Data for Robot Motion Planning and Collision Avoidance," in Machine Vision and Applications. New York: Springer-Verlag, 2003.
18. F. Ferland, F. Pomerleau, C. T. Le Dinh, and F. Michaud, "Egocentric and exocentric teleoperation interface using real-time, 3D video projection," in Proc. 4th ACM/IEEE Int. Conf. Human Robot Interact., 2009, pp. 37-44.
19. C. W. Nielsen, M. A. Goodrich, and R. W. Ricks, "Ecological interfaces for improving mobile robot teleoperation," IEEE Trans. Robot., vol. 23, no. 5, pp. 927-941, Oct. 2007. (Pubitemid 47577969)
20. R. Meier, T. Fong, C. Thorpe, and C. Baur, "A Sensor Fusion Based User Interface for Vehicle Teleoperation," Field and Service Robots, Pittsburgh, PA, Aug. 1999.
21. G. Terrien, T. Fong, C. Thorpe, and C. Baur, "Remote driving with a multisensor user interface," presented at the presented at the Int. Conf. Environmental Syst., Toulouse, France, Jul. 2000.
22. Fraunhofer IAIS Institute 3DLS: 3D Laser Scanner Dec. 2009 [Online]. Available: http://www.3d-scanner.net
23. J. W. S. Chong, S. K. Ong, A. Y. C. Nee, and K. Youcef-Youmi, "Robot programming using augmented reality: An interactive method for planning collision-free paths," Robot. Comput.-Integr. Manuf., vol. 25, pp. 689-701, 2009.
24. U. Neumann and A. Majoros, "Cognitive, performance, and systems issues for augmented reality applications in manufacturing and maintenance," in Proc. IEEE Virtual Reality Annu. Int. Symp. (VRAIS), Atlanta, GA, Mar. 1998, pp. 4-11.
25. P. Milgram, S. Zhai, D. Drasic, and J. Grodski, "Applications of augmented reality for human robot communication," in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., Yokohama, Japan, Jul. 1993, pp. 1467-1472.
26. A. Rastogi and P. Milgram, "Augmented telerobotic control: A visual interface for unstructured environments," in Proc. KBS/Robot. Conf., Montreal, PQ, Canada, Oct. 1995, pp. 16-18.
27. Intelligent Robot Painting Dec. 2009 [Online]. Available: http://www.inropa.com, Inropa Company
28. T. Pettersen, J. Pretlove, C. Skourup, T. Engedal, and T. Lokstad, "Augmented reality for programming industrial robots," in Proc. Int. Symp. Mixed Augmented Reality, Tokyo, Japan, Oct. 2003, pp. 319-320.
29. J. Molineros and R. Sharma, "Computer vision for guiding manual assembly," in Proc. IEEE Int. Symp. Assembly Task Plann., Fukuoka, Japan, May 2001, pp. 362-368. (Pubitemid 32882753)
30. R. T. Azuma, "A survey of augmented reality," Presence, vol. 6, no. 4, pp. 355-385, 1997.
31. R. Marin, P. J. Sanz, P. Nebot, and R. Wirz, "A multimodal interface to control a robot arm via the web: A case study on remote programming," IEEE Trans. Ind. Electron., vol. 52, no. 6, pp. 1506-1520, Dec. 2005. (Pubitemid 41830021)
32. M. Baker, R. Casey, B. Keyes, and H. A. Yanco, "Improved interfaces for human-robot interaction in urban search and rescue," in Proc. IEEE Int. Conf. Syst., Man Cybern., 2004, vol. 3, pp. 2960-2965. (Pubitemid 40408616)
33. J. Scholtz, J. Young, J. Drury, and H. Yanco, "Evaluation of humanrobot interaction awareness in search and rescue," in IEEE Int. Conf. Robot. Autom., 2004, vol. 3, pp. 2327-2332.
34. H. A. Yanco and J. Drury, "Where am I? Acquiring situation awareness using a remote robot platform," in IEEE Int. Conf. Syst., Man Cybern., 2004, pp. 2835-2840. (Pubitemid 40408595)
35. B. Giesler, T. Salb, P. Steinhaus, and R. Dillmann, "Uning augmented reality to interact with an autonomous mobile platform," in Proc. IEEE Int. Conf. Robot. Autom., 2004, vol. 1, pp. 1009-1014.
36. Y. Bok, Y. Hwang, and I. So Kweon, "Accurate motion estimation and high-precision 3D reconstruction by sensor fusion," in IEEE Int. Conf. Robot. Autom., Roma, Italy, Apr. 2007, pp. 4721-4726. (Pubitemid 350140918)
37. Q. Zhang and R. Pless, "Extrinsic calibration of camera and laser range finder," in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., Sendai, Japan, Sep. 2004, pp. 2301-2306.
38. M. A. Al-Mouhamed, O. Toker, and A. Al-Harthy, "A 3-D visionbased man-machine interface for hand-controlled telerobot," IEEE Trans. Ind. Electron., vol. 52, no. 1, pp. 306-319, Feb. 2005. (Pubitemid 40264779)
39. P. E. Debevec, C. J. Taylor, and J. Malik, "Modeling and rendering architectures from photographs: A hybrid geometry-and image-based approach," in Proc. SIGGRAPH, Aug. 1996, pp. 11-20.
40. E. Williams and B. Andrews, The Non-Designer's Design Book. Berkeley, CA: Peachipt Press, 1994.
41. Open Source Computer Vision: OpenCV [Online]. Available: http://opencv. willowgarage.com 2009, OpenCV Software
42. J. Canny, "A computational approach to edge detection," Readings in Computer Vision: Issues, Problems, Principles, and Paradigms, vol. 184, pp. 87-116, 1987.
43. G. J. Kim, Designing Virtual Reality Systems. The Structured Approach. New York: Springer-Verlag, 2005
44. S. Livatino and C. Koeffel, "Handbook for evaluation studies in virtual reality," in Proc. IEEE Symp. Virtual Environ., Human-Comput. Interface Meas. Syst., Ostuni, Italy, 2007, pp. 1-6.
45. M. Ferre, R. Aracil, and M. A. Sanchez-Uran, "Stereoscopic human interfaces," IEEE Robot. Autom. Mag., vol. 14, no. 4, pp. 50-57, Dec. 2008.