2D human gesture tracking and recognition by the fusion of MEMS inertial and vision sensors

IEEE Sensors Journal

Volumn 14, Issue 4, 2014, Pages 1160-1170

  Zhou, Shengli ^a   Fei, Fei ^b   Zhang, Guanglie ^b   Mai, John D ^b   Liu, Yunhui ^a   Liou, Jay Y J ^c   Li, Wen J ^b  

^a CHINESE UNIVERSITY OF HONG KONG   (Hong Kong)

^b CITY UNIVERSITY OF HONG KONG   (Hong Kong)

^c HONG KONG APPLIED SCIENCE AND TECHNOLOGY RESEARCH INSTITUTE   (Hong Kong)

Author keywords

Gesture recognition; MEMSbased motion tracking; Sensor calibration; Sensor fusion

Indexed keywords

HAND GESTURE TRACKING; LOW RESOLUTION IMAGING; MICRO ELECTROMECHANICAL SYSTEM (MEMS); MOTION TRACKING; RECONSTRUCTED TRAJECTORY; SENSOR CALIBRATION; SENSOR FUSION; VISUAL DATA PROCESSING;

ADAPTIVE ALGORITHMS; COMPUTER VISION; COSINE TRANSFORMS; ELECTROMECHANICAL DEVICES; FEATURE EXTRACTION; IMAGE RECONSTRUCTION; INERTIAL NAVIGATION SYSTEMS; MEMS; OPTICAL DATA PROCESSING; SENSORS; UNITS OF MEASUREMENT;

GESTURE RECOGNITION;

EID: 84896748774     PISSN: 1530437X     EISSN: None     Source Type: Journal    
DOI: 10.1109/JSEN.2013.2288094     Document Type: Article

References (49)

1. S. Mitra and T. Acharya, "Gesture recognition: A survey," IEEE Trans. Syst., Man, Cybern., Part C, Appl. Rev., vol. 37, no. 3, pp. 311-324, May 2007. (Pubitemid 46621535)
2. J. S. Wang and F. C. Chuang, "An accelerometer-based digital pen with a trajectory recognition algorithm for handwritten digit and gesture recognition," IEEE Trans. Ind. Electron., vol. 59, no. 7, pp. 2998-3007, Jul. 2012.
3. D. H. Shin and W. S. Jang, "Utilization of ubiquitous computing for construction AR technology," Autom. Construction, vol. 18, no. 8, pp. 1063-1069, 2009.
4. G. Welch and E. Foxlin, "Motion tracking: No silver bullet, but a respectable arsenal," IEEE Comput. Graph. Appl., vol. 22, no. 6, pp. 24-38, Sep. 2002. (Pubitemid 35421716)
5. S. Zhou, Q. Shan, F. Fei, W. J. Li, C. P. Kwong, P. C. K. Wu, et al., "Gesture recognition for interactive controllers using MEMS motion sensors," in Proc. 4th IEEE Int. Conf. Nano/Micro Eng. Molecular Syst., Jan. 2009, pp. 935-940.
6. C. Hu, M. Li, S. Song, R. Zhang, and M. Q. H. Meng, "A cubic 3- axis magnetic sensor array for wirelessly tracking magnet position and orientation," IEEE Sensors J., vol. 10, no. 5, pp. 903-913, May 2010.
7. J. Lee, S. Hong, N. Moon, and S. J. Oh, "Acoustic sensor-based multiple object tracking with visual information association," EURASIP J. Adv. Signal Process., vol. 2010, pp. 90-93, Aug. 2010.
8. M. A. Amin and H. Yan, "Sign language finger alphabet recognition from gabor-PCA representation of hand gestures," in Proc. 6th Int. Conf. Mach. Learn. Cybern., 2007, pp. 2218-2223.
9. M. Elmezain, A. Al-Hamadi, S. S. Pathan, and B. Michaelis, "Spatiotemporal feature extraction-based hand gesture recognition for isolated american sign language and arabic numbers," in Proc. 6th Int. Symp. Image Signal Process. Anal., 2009, pp. 254-259.
10. A. Seniuk and D. Blostein, "Pen acoustic emissions for text and gesture recognition," in Proc. 10th Int. Conf. Document Anal. Recognit., 2009, pp. 872-876.
11. P. Asadzadeh, L. Kulik, and E. Tanin, "Gesture recognition using RFID technology," Pers. Ubiquitous Comput., vol. 16, no. 3, pp. 225-234, 2012.
12. M. G. Ceruti, V. V. Dinh, N. X. Tran, H. Van Phan, L. T. Duffy, T. A. Ton, et al., "Wireless communication glove apparatus for motion tracking, gesture recognition, data transmission, and reception in extreme environments," in Proc. ACM Symp. Appl. Comput., 2009, pp. 172-176.
13. C. Sung-do and L. Soo-Young, "3D stroke reconstruction and cursive script recognition with magnetometer-aided inertial measurement unit," IEEE Trans. Consumer Electron., vol. 58, no. 2, pp. 661-669, May 2012.
14. W. De Vries, H. Veeger, C. Baten, and F. Van Der Helm, "Magnetic distortion in motion labs, implications for validating inertial magnetic sensors," Gait Posture, vol. 29, no. 4, pp. 535-541, 2009.
15. N. Phuong, H. J. Kang, Y. S. Suh, and Y. S. Ro, "A DCM based orientation estimation algorithm with an inertial measurement unit and a magnetic compass," J. Univ. Comput. Sci., vol. 15, no. 4, pp. 859-876, 2009.
16. H. M. Schepers, D. Roetenberg, and P. H. Veltink, "Ambulatory human motion tracking by fusion of inertial and magnetic sensing with adaptive actuation," Med. Biol. Eng. Comput., vol. 48, no. 1, pp. 27-37, 2010.
17. W. T. Faulkner, R. Alwood, D. W. A. Taylor, and J. Bohlin, "GPS-denied pedestrian tracking in indoor environments using an IMU and magnetic compass," in Proc. Int. Tech. Meeting Inst. Navigat., 2010, pp. 198-204.
18. A. Jimenez, F. Seco, C. Prieto, and J. Guevara, "A comparison of pedestrian dead-reckoning algorithms using a low-cost MEMS IMU," in Proc. IEEE Int. Symp. Intell. Signal Process., Aug. 2009, pp. 37-42.
19. D. Dusha and L. Mejias, "Error analysis and attitude observability of a monocular GPS/visual odometry integrated navigation filter," Int. J. Robot. Res., vol. 31, no. 6, pp. 714-737, 2012.
20. J. Naranjo, F. Jiménez, F. Aparicio, and J. Zato, "GPS and inertial systems for high precision positioning on motorways," J. Navigat., vol. 62, no. 2, pp. 351-363, 2009.
21. C. H. Kang, S. Y. Kim, and C. G. Park, "Improvement of a low cost mems inertial-GPS integrated system using wavelet denoising techniques," Int. J. Aeronautical Space Sci., vol. 12, no. 4, pp. 371-378, 2011.
22. R. E. Hopkins, N. M. Barbour, D. E. Gustafson, and P. Sherman, Miniature Inertial and Augmentation Sensors for Integrated Inertial/GPS Based Navigation Applications. Cambridge, MA, USA, NATO RTO Lecture Series, 2010.
23. J. Hol, "Sensor fusion and calibration of inertial sensors, vision, ultrawideband and GPS," Ph.D. dissertation, Linköping Studies Sci. Technol., Linköping Univ., Linköping, Sweden, 2011.
24. G. Bleser and D. Stricker, "Advanced tracking through efficient image processing and visual-inertial sensor fusion," Comput. Graph., vol. 33, no. 1, pp. 59-72, 2009.
25. G. Bleser and G. Hendeby, "Using optical flow for filling the gaps in visual-inertial tracking," in Proc. EUSIPCO, 2010, pp. 1057-1063.
26. R. Xu, S. Zhou, and W. J. Li, "MEMS accelerometer based nonspecificuser hand gesture recognition," IEEE Sensors J., vol. 12, no. 5, pp. 1166-1173, May 2012.
27. R. Z. Khan and N. A. Ibraheem, "Comparative study of hand gesture recognition system," in Proc. Comput. Sci. Inf. Technol., 2012, pp. 203-213.
28. Y. Tao, H. Hu, and H. Zhou, "Integration of vision and inertial sensors for 3D arm motion tracking in home-based rehabilitation," Int. J. Robot. Res., vol. 26, no. 6, pp. 607-624, 2007. (Pubitemid 47011241)
29. Y. Suya, U. Neumann, and R. Azuma, "Hybrid inertial and vision tracking for augmented reality registration," in Proc. Virtual Reality, 1999, pp. 260-267.
30. S. Giannarou, Z. Zhiqiang, and Y. Guang-Zhong, "Deformable structure from motion by fusing visual and inertial measurement data," in Proc. IEEE/RSJ Int. Conf. Intell. Robot. Syst., Oct. 2012, pp. 4816-4821.
31. P. Gemeiner, P. Einramhof, and M. Vincze, "Simultaneous motion and structure estimation by fusion of inertial and vision data," Int. J. Robot. Res., vol. 26, no. 6, pp. 591-605, 2007. (Pubitemid 47011240)
32. L. Armesto, J. Tornero, and M. Vincze, "Fast ego-motion estimation with multi-rate fusion of inertial and vision," Int. J. Robot. Res., vol. 26, no. 6, pp. 577-589, 2007. (Pubitemid 47018080)
33. M. Li and A. I. Mourikis, "Vision-aided inertial navigation for resourceconstrained systems," in Proc. IEEE/RSJ Int. Conf. Intell. Robot. Syst., Oct. 2012, pp. 1057-1063.
34. M. Achtelik, T. Zhang, K. Kuhnlenz, and M. Buss, "Visual tracking and control of a quadcopter using a stereo camera system and inertial sensors," in Proc. Int. Conf. Mechatron. Autom., 2009, pp. 2863-2869.
35. G. Bishop and G. Welch, "An introduction to the Kalman filter," in Proc. ACM SIGGRAPH Course 8, Los Angeles, CA, USA, 2001.
36. J. A. Corrales Ramón, F. A. Candelas Herías, and F. Torres Medina, "Kalman filtering for sensor fusion in a human tracking system," 2010.
37. S. J. Julier and J. K. Uhlmann, "New extension of the Kalman filter to nonlinear systems," in Proc. AeroSense, 11th Int. Symp. Aerosp./Defense Sens., Simul. Controls, 1997, pp. 182-193.
38. L. Armesto, J. Tornero, and M. Vincze, "On multi-rate fusion for nonlinear sampled-data systems: Application to a 6D tracking system," Robot. Auto. Syst., vol. 56, no. 8, pp. 706-715, 2008.
39. P. Saeedi, P. D. Lawrence, and D. G. Lowe, "Vision-based 3-D trajectory tracking for unknown environments," IEEE Trans. Robot., vol. 22, no. 1, pp. 119-136, Feb. 2006. (Pubitemid 43245261)
40. O. J. Woodman, "An introduction to inertial navigation," Dept. Comput. Lab., Univ. Cambridge, Cambridge, MA, USA, Tech. Rep. UCAMCLTR- 696, 2007.
41. D. Titterton, J. Weston, D. Titterton, and J. Weston, Strapdown Inertial Navigation Technology. 2nd ed. London, U.K.: Inst. Electr. Eng., 2004.
42. J. Lobo and J. Dias, "Relative pose calibration between visual and inertial sensors," Int. J. Robot. Res., vol. 26, no. 6, pp. 561-575, 2007. (Pubitemid 47011239)
43. G. T. Marzan and H. M. Karara, "A computer program for direct linear transformation solution of collinearity condition, and some applications of it," in Proc. Symp. Close-Range Photogram. Syst., 1975, pp. 420-476.
44. T. Huck, A. Westenberger, M. Fritzsche, T. Schwarz, and K. Dietmayer, "Precise timestamping and temporal synchronization in multi-sensor fusion," in Proc. IEEE IV Symp., Sep. 2011, pp. 242-247.
45. J. D. Hol, T. B. Schön, H. Luinge, P. J. Slycke, and F. Gustafsson, "Robust real-time tracking by fusing measurements from inertial and vision sensors," J. Real-Time Image Process., vol. 2, no. 2, pp. 149-160, 2007. (Pubitemid 350027503)
46. H. Sato, "Moving average filter," U.S. Patent no. 6 304 133, Oct. 16, 2001.
47. J. F. Ehlers, Rocket Science for Traders: Digital Signal Processing Applications. New York, NY, USA: Wiley, 2001.
48. P. Senin, Dynamic Time Warping Algorithm Review. New York, NY, USA: Wiley, 2008.
49. S. A. Khayam, "The discrete Cosine transform (DCT): Theory and application," Dept. Electr. Comput. Eng., Michigan State Univ., Lansing, MI, USA, 2003.