Online driver distraction detection using long short-term memory

IEEE Transactions on Intelligent Transportation Systems

Volumn 12, Issue 2, 2011, Pages 574-582

  Wollmer, Martin ^a   Blaschke, Christoph ^b   Schindl, Thomas ^a   Schuller, Bjrn ^a   Farber, Berthold ^c   Mayer, Stefan ^b   Trefflich, Benjamin ^b  

^a TECHNICAL UNIVERSITY OF MUNICH   (Germany)

^b AUDI ELECTRONICS VENTURE GMBH   (Germany)

^c BUNDESWEHR UNIVERSITY MUNICH   (Germany)

Author keywords

Driver assistance systems; driver state estimation; long short term memory (LSTM); recurrent neural networks (RNNs)

Indexed keywords

ASSISTANCE SYSTEM; CONVENTIONAL APPROACH; DRIVER ASSISTANCE SYSTEM; DRIVER DISTRACTIONS; DRIVER STATE ESTIMATION; HEAD TRACKING; LONG SHORT-TERM MEMORY (LSTM); NOVEL TECHNIQUES; ON-LINE DETECTION; SHORT TERM MEMORY;

AUTOMOBILE DRIVERS; BRAIN; ONLINE SYSTEMS; SUPPORT VECTOR MACHINES;

RECURRENT NEURAL NETWORKS;

EID: 79958176949     PISSN: 15249050     EISSN: None     Source Type: Journal    
DOI: 10.1109/TITS.2011.2119483     Document Type: Conference Paper

References (40)

1. J. Wang, R. Knipling, and M. Goodman, "The role of driver inattention in crashes; new statistics from the 1995 crashworthiness data system (CDS)", in Proc. 40th Annu. Assoc. Advancement Automot. Med., 1996, pp. 377-392.
2. T. Ranney, E. Mazzae, R. Garrott, and M. Goodman, "NHTSA driver distraction research: Past, present and future", Nat. Highway Traffic Safety Admin., Washington, DC, 2000.
3. T. Dingus, S. Klauer, V. Neale, A. Petersen, S. Lee, J. Sudweeks, M. Perez, J. Hankey, D. Ramsey, S. Gupta, C. Bucher, Z. Doerzaph, J. Jermeland, and R. Knipling, "The 100-car naturalistic driving study, phase II-Results of the 100-car field experiment", Transp. Res. Board Nat. Acad., Washington, DC, 2006.
4. Y. Sugimoto and C. Sauer, "Effectiveness estimation method for advanced driver assistance system and its application to collision mitigation brake system", in Proc. 19th Int. Tech. Conf. Enhanced Safety Vehicles, 2005, pp. 1-8.
5. R. Freymann, "The role of driver assistance systems in a future traffic scenario", in Proc. IEEE Int. Conf. Control Appl., Munich, Germany, 2006, pp. 2269-2274.
6. M. Rimini-Döring, T. Altmüller, U. Ladstätter, and M. Rossmeier, "Effects of lane departure warning on drowsy drivers' performance and state in a simulator", in Proc. 3rd Int. Driving Symp. Hum. Factors Driver Assessment, Train. Vehicle Des., Rockport, ME, 2005.
7. T. Alkim, G. Bootsma, and S. Hoogendoorn, "Field operational test 'The assisted driver'", in Proc. Intell. Vehicles Symp., Istanbul, Turkey, 2007, pp. 1198-1203.
8. K. Kozak, J. Pohl, W. Birk, J. Greenberg, B. Artz, M. Blommer, L. Cathey, and R. Curry, "Evaluation of lane departure warnings for drowsy drivers", in Proc. Hum. Factors Ergonom. Soc. 50th Annu. Meeting, San Francisco, CA, 2006, pp. 2400-2404. (Pubitemid 351742146)
9. C. Blaschke, F. Breyer, B. Färber, J. Freyer, and R. Limbacher, "Driver distraction based lane-keeping assistance", Transp. Res. Part F: Traffic Psychol. Behav., vol. 12, no. 4, pp. 288-299, Jul. 2009.
10. K. Torkkola, N. Massey, and C. Wood, "Detecting driver inattention in the absence of driver monitoring sensors", in Proc. Int. Conf. Mach. Learn. Appl., Louisville, KY, 2004, pp. 220-226. (Pubitemid 40893326)
11. D. de Waard, K. A. Brookhuis, and N. Hernandez-Gress, "The feasibility of detecting phone-use related driver distraction", Int. J. Vehicle Des., vol. 26, no. 1, pp. 85-95, 2001. (Pubitemid 32534700)
12. H. Zhang, M. R. H. Smith, and G. J. Witt, "Identification of real-time diagnostic measures of visual distraction with an automatic eye-tracking system", Hum. Factors, vol. 48, no. 4, pp. 805-821, 2006. (Pubitemid 46036207)
13. Y. Liang, M. L. Reyes, and J. D. Lee, "Real-time detection of driver cognitive distraction using support vector machines", IEEE Trans. Intell. Transp. Syst., vol. 8, no. 2, pp. 340-350, Jun. 2007. (Pubitemid 46873214)
14. Y. Liang, J. D. Lee, and M. L. Reyes, "Nonintrusive detection of driver cognitive distraction in real time using Bayesian networks", Transp. Res. Rec.: J. Transp. Res. Board, vol. 2018, pp. 1-8, 2007. (Pubitemid 351345443)
15. Q. Ji, Z. Zhu, and P. Lan, "Real-time nonintrusive monitoring and prediction of driver fatigue", IEEE Trans. Veh. Technol., vol. 53, no. 4, pp. 1052-1068, Jul. 2004.
16. T. D'Orazio, M. Leo, C. Guaragnella, and A. Distante, "A visual approach for driver inattention detection", Pattern Recognit., vol. 40, no. 8, pp. 2341-2355, Aug. 2007. (Pubitemid 46574771)
17. Q. Ji, P. Lan, and C. Looney, "A probabilistic framework for modeling and real-time monitoring human fatigue", IEEE Trans. Syst., Man, Cybern. A, Syst., Humans, vol. 36, no. 5, pp. 862-875, Sep. 2006.
18. J. A. Healey and R. W. Picard, "Detecting stress during real-world driving tasks using physiological sensors", IEEE Trans. Intell. Transp. Syst., vol. 6, no. 2, pp. 156-166, Jun. 2005. (Pubitemid 40925766)
19. Y. Liang and J. D. Lee, Driver Cognitive Distraction Detection Using Eye Movements. Berlin, Germany: Springer-Verlag, 2008, pp. 285-300.
20. M. H. Kutila, M. Jokela, T. Mäkinen, J. Viitanen, G. Markkula, and T. W. Victor, "Driver cognitive distraction detection: Feature estimation and implementation", Proc. Inst. Mech. Eng., Part D: J. Automobile Eng., vol. 221, no. 9, pp. 1027-1040, 2007.
21. T. Kumagai and M. Akamatsu, "Prediction of human driving behavior using dynamic Bayesian networks", IEICE Trans. Inf. Syst., vol. E89-D, no. 2, pp. 857-860, Feb. 2006.
22. A. Pentland and A. Liu, "Modeling and prediction of human behavior", Neural Comput., vol. 11, no. 1, pp. 229-242, Jan. 1999.
23. L. Qiao, M. Sato, and H. Takeda, "Learning algorithm of environmental recognition in driving vehicle", IEEE Trans. Syst., Man, Cybern., vol. 25, no. 6, pp. 917-925, Jun. 1995.
24. S. Hochreiter, Y. Bengio, P. Frasconi, and J. Schmidhuber, "Gradient flow in recurrent nets: The difficulty of learning long-term dependencies", in A Field Guide to Dynamical Recurrent Neural Networks, S. C. Kremer and J. F. Kolen, Eds. Piscataway, NJ: IEEE Press, 2001.
25. S. Hochreiter and J. Schmidhuber, "Long short-term memory", Neural Comput., vol. 9, no. 8, pp. 1735-1780, Nov. 1997. (Pubitemid 127462305)
26. K. Young, M. Regan, and M. Hammer, "Driver distraction: A review of literature", Monash Univ. Accident Res. Center, Victoria, Australia, 2003.
27. P. Smith, M. Shah, and N. da Vitoria Lobo, "Determining driver visual attention with one camera", IEEE Trans. Intell. Transp. Syst., vol. 4, no. 4, pp. 205-218, Dec. 2003.
28. A. Graves, "Supervised sequence labelling with recurrent neural networks", Ph. D. dissertation, Techn. Univ. München, Munich, Germany, 2008.
29. F. Eyben, M. Wöllmer, and B. Schuller, "openEAR-Introducing the Munich open-source emotion and affect recognition toolkit", in Proc. ACII, Amsterdam, The Netherlands, 2009, pp. 576-581.
30. M. A. Hall, "Correlation-based feature selection for machine learning", Ph. D. dissertation, Univ. Waikato, Hamilton, New Zealand, 1999.
31. I. H. Witten and E. Frank, Data Mining: Practical Machine Learning Tools and Techniques., 2nd ed. San Francisco, CA: Morgan Kaufmann, 2005.
32. M. Wöllmer, F. Eyben, J. Keshet, A. Graves, B. Schuller, and G. Rigoll, "Robust discriminative keyword spotting for emotionally colored spontaneous speech using bidirectional LSTM networks", in Proc. ICASSP, Taipei, Taiwan, 2009, pp. 3949-3952.
33. A. Graves and J. Schmidhuber, "Framewise phoneme classification with bidirectional LSTM and other neural network architectures", Neural Netw., vol. 18, no. 5/6, pp. 602-610, Jun. 2005. (Pubitemid 43186580)
34. M. Wöllmer, B. Schuller, F. Eyben, and G. Rigoll, "Combining long shortterm memory and dynamic Bayesian networks for incremental emotionsensitive artificial listening", IEEE J. Sel. Topics Signal Process., vol. 4, no. 5, pp. 867-881, Oct. 2010.
35. M. Wöllmer, F. Eyben, A. Graves, B. Schuller, and G. Rigoll, "Bidirectional LSTM networks for context-sensitive keyword detection in a cognitive virtual agent framework", Cogn. Comput., vol. 2, no. 3, pp. 180-190, Sep. 2010.
36. R. Fan, P. Chen, and C. Lin, "Working set selection using the second order information for training SVM", J. Mach. Learn. Res., vol. 6, pp. 1889-1918, 2005. (Pubitemid 41798130)
37. T. Wu, C. Lin, and R. C. Weng, "Probability estimates for multi-class classification by pairwise coupling", J. Mach. Learn. Res., vol. 5, pp. 975-1005, 2004.
38. M. Schuster and K. K. Paliwal, "Bidirectional recurrent neural networks", IEEE Trans. Signal Process., vol. 45, no. 11, pp. 2673-2681, Nov. 1997. (Pubitemid 127766336)
39. M. Wöllmer, M. Al-Hames, F. Eyben, B. Schuller, and G. Rigoll, "A multidimensional dynamic time warping algorithm for efficient multimodal fusion of asynchronous data streams", Neurocomput., vol. 73, no. 1-3, pp. 366-380, Dec. 2009.
40. Y. Yao, G. L. Marcialis, M. Pontil, P. Frasconi, and F. Roli, "Combining flat and structured representations for fingerprint classification with recursive neural networks and support vector machines", Pattern Recognit., vol. 36, no. 2, pp. 397-406, Feb. 2003. (Pubitemid 35212836)