Hand gesture recognition in real time for automotive interfaces: A multimodal vision-based approach and evaluations

IEEE Transactions on Intelligent Transportation Systems

Volumn 15, Issue 6, 2014, Pages 2368-2377

  Ohn Bar, Eshed ^a   Trivedi, Mohan Manubhai ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Depth cue analysis; driver assistance systems; hand gesture recognition; human machine interaction; infotainment

Indexed keywords

AUTOMOBILE DRIVERS; HUMAN COMPUTER INTERACTION; MAN MACHINE SYSTEMS;

DEPTH CUE; DRIVER ASSISTANCE SYSTEM; HAND-GESTURE RECOGNITION; HUMAN MACHINE INTERACTION; INFOTAINMENT;

GESTURE RECOGNITION;

EID: 84908519903     PISSN: 15249050     EISSN: None     Source Type: Journal    
DOI: 10.1109/TITS.2014.2337331     Document Type: Article

References (61)

1. J.M. Gregers,M. B. Skov, and N. G. Thomassen, "You can touch, but you can't look: Interacting with in-vehicle systems," in Proc. SIGCHI Conf. Hum. Factors Comput. Syst., 2008, pp. 1139-1148
2. M. Alpern and K. Minardo, "Developing a car gesture interface for use as a secondary task," in Proc. CHI Hum. Factors Comput. Syst., 2003, pp. 932-933
3. F. Parada-Loira, E. González-Agulla, and J. L. Alba-Castro, "Hand gestures to control infotainment equipment in cars," in Proc. IEEE Intell. Veh. Symp., 2014, pp. 1-6
4. C. A. Pickering, K. J. Burnham, and M. J. Richardson, "A research study of hand gesture recognition technologies and applications for human vehicle interaction," in Proc. Inst. Eng. Technol. Conf. Autom. Electron., 2007, pp. 1-15
5. W. J. Horrey, "Assessing the effects of in-vehicle tasks on driving performance," Ergonom. Design, vol. 19, no. 4, pp. 4-7, 2011
6. G. Jahn, J. F. Krems, and C. Gelau, "Skill acquisition while operating in-vehicle information systems: Interface design determines the level of safety-relevant distractions," Hum. Factors Ergonom. Soc., vol. 51, no. 2, pp. 136-151, Apr. 2009
7. S. Klauer, F. Guo, J. Sudweeks, and T. Dingus, "An Analysis of Driver Inattention Using a Case-Crossover Approach on 100-Car Data: Final Report," Nat. Highway Traffic Safety Admin., Washington, DC, USA, Tech. Rep. DOT HS 811 334, 2010
8. E. Ohn-Bar, A. Tawari, S. Martin, and M. M. Trivedi, "Predicting driver maneuvers by learning holistic features," in Proc. IEEE Intell. Veh. Symp., 2014, pp. 719-724
9. E. Ohn-Bar, S. Martin, and M. M. Trivedi, "Driver hand activity analysis in naturalistic driving studies: Challenges, algorithms, experimental studies," J. Electron. Imag., vol. 22, no. 4, p. 041119, 2013
10. C. Tran and M. M. Trivedi, "Vision for driver assistance: Looking at people in a vehicle," in Visual Analysis of Humans. London, U.K.: Springer-Verlag, 2011, pp. 597-614
11. C. Tran and M. Trivedi, "Driver assistance for "keeping hands on the wheel and eyes on the road"," in Proc. IEEE Conf. Veh. Electron. Safety, 2009, pp. 97-101
12. E. Ohn-Bar, A. Tawari, S. Martin, and M. M. Trivedi, "Vision on wheels: Looking at driver, vehicle, surround for on-road maneuver analysis," in Proc. IEEE Conf. Comput. Vis. Pattern Recog. Workshops-Mobile Vis., 2014, pp. 185-190
13. I. Oikonomidis, N. Kyriazis, and A. A. Argyros, "Efficient model-based 3D tracking of hand articulations using Kinect," in Proc. Brit. Mach. Vis. Conf., 2011, pp. 101.1-101.11
14. C. Qian, X. Sun, Y. Wei, X. Tang, and J. Sun, "Realtime and robust hand tracking from depth," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2014, pp. 1-8
15. E. Ohn-Bar and M. M. Trivedi, "Joint angles similarities and HOG2 for action recognition," in Proc. IEEE Conf. Comput. Vis. Pattern Recog. Workshops-Hum. Activity Understanding 3D Data, 2013, pp. 465-470
16. A. Kläser, M. Marszalek, and C. Schmid, "A spatio-temporal descriptor based on 3D-gradients," in Proc. Brit. Mach. Vis. Conf., 2008, pp. 1-10
17. W. Heng, A. Kläser, C. Schmid, and C.-L. Liu, "Dense trajectories and motion boundary descriptors for action recognition," Int. J. Comput. Vis., vol. 103, no. 1, pp. 60-79, May 2013
18. C.-C. Chang and C.-J. Lin, "LIBSVM: A library for support vector machines," ACM Intell. Syst. Tech., vol. 2, pp. 27:1-27:27, 2011
19. M. B. Holte, C. Tran, M. M. Trivedi, and T. B. Moeslund, "Human pose estimation and activity recognition from multi-view videos: Comparative explorations of recent developments," IEEE J. Sel. Topics Signal Process., vol. 6, no. 5, pp. 538-552, Sep. 2012
20. R. Poppe, "A survey on vision-based human action recognition," Image Vis. Comput., vol. 28, no. 6, pp. 976-990, Jun. 2010
21. S. Sathyanarayana, G. Littlewort, and M. Bartlett, "Hand gestures for intelligent tutoring systems: Dataset, techniques and evaluation," in Proc. IEEE Int. Conf. Comput. Vis. Workshops, 2013, pp. 769-776
22. S. Hadfield and R. Bowden, "Hollywood 3D: Recognizing actions in 3D natural scenes," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2013, pp. 3398-3405
23. Y. Zhu, W. Chen, and G. Guo, "Evaluating spatiotemporal interest point features for depth-based action recognition," Image Vis. Comput., vol. 32, no. 8, pp. 453-464, Aug. 2014
24. N. Neverova et al., "A multi-scale approach to gesture detection and recognition," in Proc. IEEE Intl. Conf. Comput. Vis. Workshops, 2013, pp. 484-491
25. A. Erol, G. Bebis, M. Nicolescu, R. D. Boyle, and X. Twombly, "Visionbased hand pose estimation: A review," Comput. Vis. Image Understanding, vol. 108, no. 1/2, pp. 52-73, Oct./Nov. 2007
26. C. Keskin, F. Kirac, Y. Kara, and L. Akarun, "Real time hand pose estimation using depth sensors," in Proc. IEEE Intl. Conf. Comput. Vis. Workshops, 2011, pp. 1228-1234
27. G. Evangelidis, G. Singh, and R. Horaud, "Skeletal quads: Human action recognition using joint quadruples," in Proc. IEEE Intl. Conf. Pattern Recog., 2014, pp. 1-6
28. R. Vemulapalli, F. Arrate, and R. Chellappa, "Human action recognition by representing 3D human skeletons as points in a lie group," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2014, pp. 1-8
29. J. Wang, Z. Liu, and Y. Wu, "Learning actionlet ensemble for 3D human action recognition," in Human Action Recognition With Depth Cameras. Cham, Switzerland: Springer-Verlag, 2014, ser. SpringerBriefs in Computer Science, pp. 11-40
30. I. Kapsouras and N. Nikolaidis, "Action recognition on motion capture data using a dynemes and forward differences representation," J. Vis. Commun. Image Represent., vol. 25, no. 6, pp. 1432-1445, Aug. 2014
31. N. Pugeault and R. Bowden, "Spelling it out: Real-time ASL fingerspelling recognition," in Proc. IEEE Intl. Conf. Comput. Vis. Workshops, 2011, pp. 1114-1119
32. C. Helen, H. Brian, and B. Richard, "Sign language recognition," in Visual Analysis of Humans. London, U.K.: Springer-Verlag, 2011, pp. 539-562
33. D. Uebersax, J. Gall, M. V. den Bergh, and L. V. Gool, "Real-time sign language letter and word recognition from depth data," in Proc. IEEE Intl. Conf. Comput. Vis. Workshops, 2011, pp. 383-390
34. P. Doliotis, A. Stefan, C. McMurrough, D. Eckhard, and V. Athitsos, "Comparing gesture recognition accuracy using color and depth information," in Proc. ACM Conf. Pervasive Technol. Related Assist. Environ., 2011, pp. 1-7
35. E. Ohn-Bar, S. Martin, A. Tawari, and M. M. Trivedi, "Head, eye, hand patterns for driver activity recognition," in Proc. IEEE Intl. Conf. Pattern Recog., 2014, pp. 1-6
36. E. Ohn-Bar and M. M. Trivedi, "The power is in your hands: 3D analysis of hand gestures in naturalistic video," in Proc. IEEE Conf. Comput. Vis. Pattern Recog. Workshops-Anal. Model. Faces Gestures, 2013, pp. 912-917, (best paper award
37. R.-D. Vatavu, "User-defined gestures for free-hand tv control," in Proc. Eur. Conf. Interact. TV Video, 2012, pp. 45-48
38. G. Panger, "Kinect in the kitchen: Testing depth camera interactions in practical home environments," in Proc. ACM Hum. Factors Comput. Syst., 2012, pp. 1985-1990
39. J. P.Wachs, M. Kölsch, H. Stern, and Y. Edan, "Vision-based hand-gesture applications," ACM Commun., vol. 54, no. 2, pp. 60-71, Feb. 2011
40. C. Kirmizibayrak et al., "Evaluation of gesture based interfaces for medical volume visualization tasks," in Proc. Virtual Real. Continuum Appl. Ind., 2011, pp. 69-74
41. L. Gallo, A. Placitelli, and M. Ciampi, "Controller-free exploration of medical image data: Experiencing the Kinect," in Proc. Comput.-Based Med. Syst., 2011, pp. 1-6
42. E. Saba, E. Larson, and S. Patel, "Dante vision: In-air and touch gesture sensing for natural surface interaction with combined depth and thermal cameras," in Proc. IEEE Conf. Emerg. Signal Process. Appl., 2012, pp. 167-170
43. C.-Y. Kao and C.-S. Fahn, "A human-machine interaction technique: Hand gesture recognition based on hidden Markov models with trajectory of hand motion," Proc. Eng., vol. 15, pp. 3739-3743, 2011
44. E. Ozcelik and G. Sengul, "Gesture-based interaction for learning: Time to make the dream a reality," Brit. J. Educ. Technol., vol. 43, no. 3, pp. 86-89, May 2012
45. Z. Ren, J. Meng, and J. Yuan, "Depth camera based hand gesture recognition and its applications in human-computer-interaction," in Proc. IEEE Conf. Inf., Commun. Signal Process., 2011, pp. 1-5
46. J. M. Teixeira, B. Reis, S. Macedo, and J. Kelner, "Open/closed hand classification using Kinect data," in Proc. IEEE Symp. Virtual Augmented Real., 2012, pp. 18-25
47. M.V. den Bergh et al., "Real-time 3Dhand gesture interactionwith a robot for understanding directions from humans," in Proc. IEEE RO-MAN, Aug. 2011, pp. 357-362
48. G. Bailly, R. Walter, J. Mller, T. Ning, and E. Lecolinet, "Comparing free hand menu techniques for distant displays using linear, marking and finger-count menus," in Hum.-Comput. Interact.-INTERACT, 2011, vol. 6947, pp. 248-262
49. C. Keskin, A. Cemgil, and L. Akarun, "DTW based clustering to improve hand gesture recognition," in Human Behavior Unterstanding, vol. 7065. Berlin, Germany: Springer-Verlag, 2011, pp. 72-81
50. D. Minnen and Z. Zafrulla, "Towards robust cross-user hand tracking and shape recognition," in Proc. IEEE Int. Conf. Comput. Vis. Workshops, 2011, pp. 1235-1241
51. O. Oreifej and Z. Liu, "HON4D: Histogram of oriented 4D normals for activity recognition from depth sequences," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2013, pp. 716-723
52. P. Dollár, V. Rabaud, G. Cottrell, and S. Belongie, "Behavior recognition via sparse spatio-temporal features," in Proc. IEEE Int. Conf. Comput. Vis. Workshops, 2005, pp. 65-72
53. L. Xia and J. Aggarwal, "Spatio-temporal depth cuboid similarity feature for activity recognition using depth camera," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2013, pp. 2834-2841
54. M. Zobl, R. Nieschulz, M. Geiger, M. Lang, and G. Rigoll, "Gesture components for natural interaction with in-car devices," in Gesture-Based Communication in Human-Computer Interaction, vol. 2915. Berlin, Germany: Springer-Verlag, 2004, pp. 367-368
55. A. Riener et al., "Standardization of the in-car gesture interaction space," in Proc. ACM Automot. User Interf. Interact. Veh. Appl., 2013, pp. 14-21
56. F. Althoff, R. Lindl, and L. Walchshaeusl, "Robust multimodal hand and head gesture recognition for controlling automotive infotainment systems," in Proc. VDI-Tagung, Der Fahrer 21, 2005, pp. 1-10
57. C. Endres, T. Schwartz, and C. A. Müller, "Geremin": 2D microgestures for drivers based on electric field sensing," in Proc. ACM Conf. Intell. User Interf., 2011, pp. 327-330
58. S. Y. Cheng and M. M. Trivedi, "Vision-based infotainment user determination by hand recognition for driver assistance," IEEE Trans. Intell. Transp. Syst., vol. 11, no. 3, pp. 759-764, Sep. 2010
59. N. Dalal and B. Triggs, "Histograms of oriented gradients for human detection," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2005, pp. 886-893
60. E. Ohn-Bar and M. M. Trivedi, "In-vehicle hand activity recognition using integration of regions," in IEEE Intell. Veh. Symp., 2013, pp. 1034-1039
61. E. Ohn-Bar, C. Tran, and M. Trivedi, "Hand gesture-based visual user interface for infotainment," in ACM Automot. User Interf. Interact. Veh. Appl., 2012, pp. 111-115