The integration of contactless static pose recognition and dynamic hand motion tracking control system for industrial human and robot collaboration

Industrial Robot

Volumn 42, Issue 5, 2015, Pages 416-428

  Tang, Gilbert ^a   Asif, Seemal ^a   Webb, Phil ^a  

^a CRANFIELD UNIVERSITY   (United Kingdom)

Author keywords

Automatic assembly; Cooperative robots; Flexible manufacturing; Robotic machining; Robotics; Teleoperation

Indexed keywords

AGRICULTURAL ROBOTS; CONTROL SYSTEMS; FACE RECOGNITION; GESTURE RECOGNITION; INDUSTRIAL ROBOTS; MAN MACHINE SYSTEMS; MANUFACTURE; MOTION TRACKING; PALMPRINT RECOGNITION; REMOTE CONTROL; ROBOTIC ASSEMBLY; ROBOTICS;

AUTOMATIC ASSEMBLY; COLLABORATIVE SYSTEMS; COOPERATIVE ROBOTS; DESIGN/METHODOLOGY/APPROACH; FLEXIBLE MANUFACTURING; HAND MOTION TRACKING; MANUFACTURING SOLUTIONS; ROBOTIC MACHINING;

SOCIAL ROBOTS;

EID: 84938796270     PISSN: 0143991X     EISSN: None     Source Type: Journal    
DOI: 10.1108/IR-03-2015-0059     Document Type: Article

References (38)

1. Arai, T., Kato, R. and Fujita, M. (2010), "Assessment of operator stress induced by robot collaboration in assembly", CIRP Annals - Manufacturing Technology, Vol. 59 No. 1, pp. 5-8.
2. Ballester, J. and Pheatt, C. (2013), "Using the Xbox Kinect sensor for positional data acquisition", American Journal of Physics, Vol. 81 No. 1, pp. 71-77.
3. Barratini, P., Morand, C. and Robertson, N.M. (2012), "A proposed gesture set for the control of industrial collaborative robots", RO-MAN 2012 IEEE, Paris, pp. 132-137.
4. Bhurane, A.A. and Talbar, S.N. (2011), "Vision-based authenticated robotic control using face and hand gesture recognition", 3rd International Conference on Electronics Computer Technology (ICECT), Kanyakumari, 8-10 April, Vol. 1, pp. 64-68.
5. British Standards Institution (2010), "BS EN ISO 13855: safety of machinery - positioning of safeguards with respect to the approach speeds of parts of the human body", British Standards Institute, London.
6. British Standards Institution (2011), "BS ISO 10218-2: robots and robotic devices - safety requirements for industrial robots Part 2: robot systems and integration", British Standards Institute, London.
7. British Standards Institution (2012), "BS ISO 16715: cranes - hand signals used with cranes", British Standards Institute, London.
8. Cerlinca, T., Pentiuc, S.G. and Vlad, V. (2013), "Real-time 3D hand gestures recognition for manipulation of industrial robots", Elektronika ir Elektrotechnika, Vol. 19 No. 2, pp. 3-8.
9. Du, G. and Zhang, P. (2014), "Markerless human-robot interface for dual robot manipulators using Kinect sensor", Robotics and Computer-Integrated Manufacturing, Vol. 30 No. 2, pp. 150-159.
10. Du, G. and Zhang, P. (2015), "A markerless human-robot interface using particle filter and Kalman filter for dual robots", IEEE Transactions on Industrial Electronics, Vol. 62 No. 4, pp. 2257-2264.
11. Du, G.L., Zhang, P. and Li, D. (2014), "Human-manipulator Interface based on multisensory process via Kalman filters", IEEE Transactions on Industrial Electronics, Vol. 61 No. 10, pp. 5411-5418.
12. Du, G.L., Zhang, P., Yang, L.Y. and Su, Y.B. (2010), "Robot teleoperation using a vision-based manipulation method", 2010 International Conference on Audio Language and Image Processing (ICALIP), Shanghai, 23-25 November, pp. 945-949.
13. Elgendi, M., Picon, F. and Magenant-Thalmann, N. (2012), "Real-time speed detection of hand gesture using Kinect", Workshop on Autonomous Social Robots and Virtual Humans, the 25th Annual Conference on Computer Animation and Social Agents (CASA 2012), 9-11 May.
14. Green, S.A., Billinghurst, M., Chen, X. and Chase, G.J. (2008), "Human-robot collaboration: a literature review and augmented reality approach in design", International Journal of Advanced Robotic Systems, Vol. 5 No. 1, pp. 1-18.
15. Guan, T., Duan, L.Y., Yu, J.Q., Chen, Y.J. and Zhang, X. (2011), "Real-time camera pose estimation for wide-area augmented reality applications", IEEE Computer Graphics and Applications, Vol. 31 No. 3, pp. 56-68.
16. Guan, T., He, Y., Gao, J., Yang, J. and Yu, J. (2013), "On-device mobile visual location recognition by integrating vision and inertial sensors", IEEE Transactions on Multimedia, Vol. 15 No. 7, pp. 1688-1699.
17. Han, J., Shao, L., Xu, D. and Shotton, J. (2013), "Enhanced computer vision with microsoft Kinect sensor: a review", IEEE Transactions on Cybernetics, Vol. 43 No. 5, pp. 1318-1334.
18. Hasanuzzaman, M.D., Zhang, T., Ampornaramveth, V., Gotoda, H., Shirai, Y. and Ueno, H. (2007), "Adaptive visual gesture recognition for human-robot interaction using a knowledge-based software platform", Robotics and Autonomous Systems, Vol. 55 No. 8, pp. 643-657.
19. Hasanuzzaman, M.D., Zhang, T., Ampornaramveth, V. and Ueno, H. (2006), "Gesture-based human-robot interaction using a knowledge-based software platform", Industrial Robot: An International Journal, Vol. 33 No. 1, pp. 37-49.
20. Karlsson, N., Karlsson, B. and Wide, P. (1998), "A glove equipped with finger flexion sensors as a command generator used in a fuzzy control system", IEEE Transactions on Instrumentation and Measurement, Vol. 47 No. 5, pp. 1330-1334.
21. Khoshelham, K. and Elberink, S.O. (2012), "Accuracy and resolution of Kinect depth data for indoor mapping applications", Sensors, Vol. 12 No. 2, pp. 1437-1454.
22. Kofman, J., Wu, X., Luu, T. and Verma, S. (2005), "Teleoperation of a robot manipulator using a vision-based human-robot interface", IEEE Transactions on Industrial Electronics, Vol. 52 No. 5, pp. 1206-1219.
23. Kornuta, T. and Zielinski, C. (2011), "Behavior-based control system of a robot actively recognizing hand postures", 15th International Conference on Advanced Robotics (ICAR), Tallinn, 20-23 June, pp. 265-270.
24. Kruse, D., Wen, J.T. and Radke, R.J. (2015), "A sensor-based dual-arm tele-robotic system", IEEE Transactions on Automation Science and Engineering, Vol. 12 No. 1, pp. 4-18.
25. Kuklinski, K., Fischer, K., Marhenke, I., Kirstein, F., aus der Wieschen, M.V., Solvason, D., Kruger, N. and Savarimuthu, T.R. (2014), "Teleoperation for learning by demonstration: data glove versus object manipulation for intuitive robot control", 2014 6th International Congress on Ultra Modern Telecommunications and Control Systems and Workshops (ICUMT), St. Petersburg, FL, 6-8 October, pp. 346-351.
26. Luo, Y., Guan, T., Wei, B., Pan, H. and Yu, J. (2015), "Fast terrain mapping from low altitude digital imagery", Neurocomputing, Vol. 156, pp. 105-116.
27. Neto, P., Pires, J. and Moreira, A. (2010), "High-level programming and control for industrial robotics: using a hand-held accelerometer-based input device for gesture and posture recognition", Industrial Robot: An International Journal, Vol. 37 No. 2, pp. 137-147.
28. Nguyen, K.C. and Perdereau, V. (2011), "Arm-hand movement: imitation of human natural gestures with tenodesis effect", 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), San Francisco, CA, pp. 1459-1464.
29. Obdrzalek, S., Kurillo, G., Ofli, F., Bajcsy, R., Seto, E., Jimison, H. and Pavel, M. (2012), "Accuracy and robustness of Kinect pose estimation in the context of coaching of elderly population", 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), San Diego, CA, 28 August-1 September, pp. 1188-1193.
30. Paredes-Madrid, L. and de Santos, P.G. (2013), "Dataglove-based interface for impedance control of manipulators in cooperative human-robot environments", Measurement Science and Technology, Vol. 24 No. 2.
31. Parkale, Y.V. (2012), "Gesture based operating system control", 2012 Second International Conference on Advanced Computing & Communication Technologies (ACCT), Rohtak, 7-8 January, pp. 318-323.
32. Schlömer, T., Poppinga, B., Henze, N. and Boll, B. (2008), "Gesture recognition with a Wii controller", Proceedings of the 2nd International Conference on Tangible and Embedded Interaction, Bonn, 18-20 February.
33. Stanton, C., Bogdanovych, A. and Ratanasena, E. (2012), "Teleoperation of a humanoid robot using full-body motion capture, example movements, and machine learning", Proceedings of Australasian Conference on Robotics and Automation, Victoria University of Wellington, 3-5 December.
34. Turk, M. and Pentland, A. (1991), "Eigenfaces for recognition", Journal of Cognitive Neuroscience, Vol. 3 No. 1, pp. 71-86.
35. Walton, M., Webb, P. and Poad, M. (2011), "Applying a concept for robot-human cooperation to aerospace equipping processes", SAE Technical Paper 2011-01-2655, SAE International.
36. Wan, T., Wang, Y. and Li, J. (2012), "Hand gesture recognition system using depth data", 2012 2nd International Conference on Consumer Electronics, Communications and Networks (CECNet), Yichang, 21-23 April, pp. 1063-1066.
37. Wigdor, D. and Wixon, D. (2011), "Chapter 19 - properties of a gesture language", Brave NUI World, Morgan Kaufmann, pp. 137-144.
38. Yanik, P.M., Manganelli, J., Merino, J., Threatt, A.L., Brooks, J.O., Green, K.E. and Walker, I.D. (2012), "Use of Kinect depth data and growing neural gas for gesture based robot control", Proceeding of the 6th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth 2012), La Jolla, CA, pp. 283-290.