A Multi-Modal Driver Fatigue and Distraction Assessment System

International Journal of Intelligent Transportation Systems Research

Volumn 14, Issue 3, 2016, Pages 173-194

  Craye, Céline ^a   Rashwan, Abdullah ^a   Kamel, Mohamed S ^a   Karray, Fakhri ^a  

^a UNIVERSITY OF WATERLOO   (Canada)

Author keywords

Advanced safety systems; Driver behavior; Driver distraction; Driver drowsiness; Driver fatigue; Driver inattention; Driver states detection; Multi modal data fusion; Safety driving assist

Indexed keywords

BAYESIAN NETWORKS; BEHAVIORAL RESEARCH; DATA FUSION; HIDDEN MARKOV MODELS; IMAGE RETRIEVAL; MARKOV PROCESSES; MODAL ANALYSIS; SIGNAL DETECTION;

DRIVER BEHAVIOR; DRIVER DISTRACTIONS; DRIVER DROWSINESS; DRIVER FATIGUE; DRIVER INATTENTION; MULTI-MODAL DATA;

ADVANCED DRIVER ASSISTANCE SYSTEMS;

EID: 84982710909     PISSN: None     EISSN: 18688659     Source Type: Journal    
DOI: 10.1007/s13177-015-0112-9     Document Type: Article

References (62)

1. Stanton, N.A., Salmon, P.M.: Human error taxonomies applied to driving: A generic driver error taxonomy and its implications for intelligent transport systems. Saf. Sci. 47(2), 227–237 (2009)
2. NHTSA: Distraction.gov, official us government website for distracted driving. http://www.distraction.gov (2013)
3. Lee, J.D., Young, K.L., Regan, M.A.: Defining driver distraction: Driver distraction: Theory, effects, and mitigation (2008)
4. Lal, S.K., Craig, A.: A critical review of the psychophysiology of driver fatigue. Biol. Psychol. 55(3), 173–194 (2001)
5. Strohl, K., Merritt, S., Blatt, J., Pack, A., Council, F., Rogus, S.: Drowsy driving and automobile crashes. nccdr/nhtsa expert panel on driver fatigue and sleepiness. http://www.nhtsa.gov/people/injury/drowsy_driving1/Drowsy.html (2004)
6. Klauer, S.G., Dingus, T.A., Neale, V.L., Sudweeks, J., Ramsey, D.: The impact of driver inattention on near-crash/crash risk: An analysis using the 100-car naturalistic driving study data, Washington DC: National Highway Traffic Safety Administration. Tech. Rep. HS-810, 594 (2006)
7. Brock, J., Robinson, A., Robinson, B., Percer, J.: Traffic safety facts, research note,” Washington DC: National Highway Traffic Safety Administration. Tech. Rep. HS-811, 737 (2013)
8. Olson, R.L., Hanowski, R.J., Hickman, J.S., Bocanegra, J.L.: Driver distraction in commercial vehicle operations, federal motor carrier safety administration, USDOT, Tech. Rep. FMCSA-RRT-09-042 (2009)
9. Williamson, A., Chamberlain, T.: Review of on-road driver fatigue monitoring devices. NSW Injury Risk Management Research Center, University of New South Wales,Tech. Rep. (2005)
10. Young, K., Regan, M., Hammer, M.: Driver distraction: A review of the literature, pp. 379–405 (2007)
11. Dong, Y., Hu, Z., Uchimura, K., Murayama, N.: Driver inattention monitoring system for intelligent vehicles: A review. Intell Trans Syst IEEE Trans on 12(2), 596–614 (2011)
12. EyeAlert: Distracted driving and fatigue sentinels. http://www.eyealert.com/ (2012)
13. Edenborough, N., Hammoud, R., Harbach, A., Ingold, A., Kisacanin, B., Malawey, P., Newman, T., Scharenbroch, G., Skiver, S., Smith, M., Wilhelm, A., Witt, G., Yoder, E., Zhang, H.: Driver state monitor from delphi, in Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Comput Soc Conf on 2, 1206–1207 (2005)
14. Lexus: Driver monitoring system. http://www.lexus.eu/car-models/ls/ls-600h/index.tmex (2006)
15. InSight: Sensomotoric instruments gmbh. http://www.smivision.com/en/gaze-and-eye-tracking-systems/services/smi-eye-tracking-roadshow.html (2011)
16. Car, M.V.: Driver alert control. http://www.media.volvocars.com (2007)
17. Alert, F.D.: Ford’s wake-up call for europe’s sleepy drivers. http://media.ford.com (2011)
18. Volkswagen: Driver alert, driver assistance, experience. http://www.volkswagen.co.uk/new/passat-vii/explore/experience/driver-assistance/driver-alert (2011)
19. Healey, J.A., Picard, R.W.: Detecting stress during real-world driving tasks using physiological sensors. Intell. Trans. Syst. IEEE Trans. on 6(2), 156–166 (2005)
20. Egelund, N.: Spectral analysis of heart rate variability as an indicator of driver fatigue. Ergonomics 25(7), 663–672 (1982)
21. Patel, M., Lal, S., Kavanagh, D., Rossiter, P.: Applying neural network analysis on heart rate variability data to assess driver fatigue. Expert Syst. Appl. 38(6), 7235–7242 (2011)
22. Miyaji, M., Danno, M., Kawanaka, H., Oguri, K.: Driver? cognitive distraction detection using adaboost on pattern recognition basis, in Vehicular Electronics and Safety, 2008. ICVES 2008, IEEE International Conference on, pp. 51–56. IEEE (2008)
23. Shiwu, L., Linhong, W., Zhifa, Y., Bingkui, J., Feiyan, Q., Zhongkai, Y.: An active driver fatigue identification technique using multiple physiological features, in Mechatronic Science, Electric Engineering and Computer (MEC), 2011 International Conference on, pp. 733–737. IEEE (2011)
24. Lal, S.K., Craig, A.: Driver fatigue: electroencephalography and psychological assessment. Psychophysiology 39(3), 313–321 (2002)
25. Jones, C.M., Jonsson, I.-M.: Automatic recognition of affective cues in the speech of car drivers to allow appropriate responses. In: Proceedings of the 17th Australia conference on Computer-Human Interaction: Citizens Online: Considerations for Today and the Future, ser. OZCHI ’05. Computer-Human Interaction Special Interest Group (CHISIG) of Australia, pp. 1–10 (2005). Available: http://dl.acm.org/citation.cfm?id=1108368.1108397
26. Tawari, A., Trivedi, M.: Speech based emotion classification framework for driver assistance system, in Intelligent Vehicles Symposium (IV), 2010 IEEE, pp. 174– 178. IEEE (2010)
27. Lugger, M., Yang, B. In: Mihelic, F., Zibert, J. (eds.): Psychological motivated multi-stage emotion classification exploiting voice quality features, in Speech Recognition, Technologies and Applications (2008). intech
28. Greeley, H.P., Friets, E., Wilson, J.P., Raghavan, S., Picone, J., Berg, J.: Detecting fatigue from voice using speech recognition, in Signal Processing and Information Technology, 2006 IEEE International Symposium on, pp. 567–571 (2006)
29. Krajewski, J., Trutschel, U., Golz, M., Sommer, D., Edwards, D.: Estimating fatigue from predetermined speech samples transmitted by operator communication systems. In: Proceedings of the International Driving Symposium on Human Factors in Driver Assessment. Train. Veh. Des. 5, 468– 474 (2009)
30. Dhupati, L., Kar, S., Rajaguru, A., Routray, A.: A novel drowsiness detection scheme based on speech analysis with validation using simultaneous eeg recordings, in, Automation Science and Engineering (CASE), 2010 IEEE Conference on (2010)
31. Bergasa, L.M., Buenaposada, J.M., Nuevo, J., Jimenez, P., Baumela, L.: Analysing driver’s attention level using computer vision, in Intelligent Transportation Systems, 2008. ITSC 2008. 11th International IEEE Conference on, 1149–1154 (2008)
32. Ji, Q., Zhu, Z., Lan, P.: Real-time nonintrusive monitoring and prediction of driver fatigue. IEEE Trans Veh Technol 53(4), 1052–1068 (2004)
33. Ji, Q., Lan, P., Looney, C.: A probabilistic framework for modeling and real-time monitoring human fatigue. Syst Man and Cybern Part A: Syst Humans, IEEE Tran on 36(5), 862–875 (2006)
34. Bergasa, L., Nuevo, J., Sotelo, M., Barea, R., Lopez, M.: Real-time system for monitoring driver vigilance, intelligent transportation systems, vol. 7, pp. 63 –77 (2006)
35. Li, L., Werber, K., Calvillo, C.F., Dinh, K.D., Guarde, A., König, A.: Multi-sensor soft-computing system for driver drowsiness detection, Online conference on soft computing in industrial applications, pp. 1–10 (2012)
36. Angell, L., Auflick, J., Austria, P., Kochhar, D., Tijerina, L., Biever, W., Diptiman, T., J. Hogsett Jr, Kiger, S.: Driver workload metrics project, national highway traffic safety administration. Tech. Rep. HS 810, 635 (2006)
37. Dinges, D.F., Grace, R.: Perclos: A valid psychophysiological measure of alertness as assessed by psychomotor vigilance, Federal Highway Administration. Office of motor carriers, Tech. Rep. MCRT-98-006 (1998)
38. Senaratne, R., Hardy, D., Vanderaa, B., Halgamuge, S., Fei, S., Hou, Z., Zhang, H., Sun, C. In: Liu, D. (ed.): Driver fatigue detection by fusing multiple cues, in Advances in Neural Networks - ISNN 2007, ser. Lecture Notes in Computer Science, vol. 4492, pp. 801–809. Springer Berlin Heidelberg (2007)
39. Smith, P., Shah, M., da Vitoria Lobo, N.: Determining driver visual attention with one camera. Int. Trans. Syst. IEEE Trans. on 4(4), 205–218 (2003)
40. Damousis, I.G., Tzovaras, D.: Fuzzy fusion of eyelid activity indicators for hypovigilance-related accident prediction. Intell. Trans. Sys. IEEE Trans. on 9(3), 491–500 (2008)
41. Daza, I., Hernandez, N., Bergasa, L., Parra, I., Yebes, J., Gavilan, M., Quintero, R., Llorca, D., Sotelo, M.: Drowsiness monitoring based on driver and driving data fusion, in Intelligent Transportation Systems (ITSC), 2011 14th International IEEE Conference on, pp. 1199–1204. IEEE (2011)
42. Pilutti, T., Ulsoy, A. G.: Identification of driver state for lane-keeping tasks: experimental results, in American Control Conference, 1997. Proceedings of the 1997, vol. 5, pp. 3370–3374. IEEE (1997)
43. Singh, H., Bhatia, J., Kaur, J.: Eye tracking based driver fatigue monitoring and warning system. In: Power Electronics (IICPE), 2010 India International Conference on, pp. 1–6. IEEE (2011)
44. Yeo, M. V., Li, X., Shen, K., Wilder-Smith, E. P.: Can svm be used for automatic eeg detection of drowsiness during car driving? Saf. Sci. 47(1), 115–124 (2009)
45. Liu, J., Zhang, C., Zheng, C.: Eeg-based estimation of mental fatigue by using kpca–hmm and complexity parameters. Biomed. Sig. Process. Control 5(2), 124–130 (2010)
46. Chua, C.-P., McDarby, G., Heneghan, C.: Combined electrocardiogram and photoplethysmogram measurements as an indicator of objective sleepiness. Physiol. Meas. 29(8), 857 (2008)
47. Fletcher, L., Loy, G., Barnes, N., Zelinsky, A.: Correlating driver gaze with the road scene for driver assistance systems. Robot. Auton. Syst. 52(1), 71–84 (2005)
48. Ji, Q., Lan, P., Looney, C.: A probabilistic framework for modeling and real-time monitoring human fatigue. Syst. Man Cybern. Part A: Syst. Hum. IEEE Trans. on 36(5), 862–875 (2006)
49. Craye, C.: A framework for context-aware driver status assessment systems,Master’s thesis, Electrical and Computer Engineering department, University of Waterloo (2013)
50. Kinect, M.: Kinect face tracking, http://msdn.microsoft.com/en-us/library/jj130970.aspx (2013)
51. Tian, Q.-C., Pan, Q., Cheng, Y.-M., Gao, Q.-X.: Fast algorithm and application of hough transform in iris segmentation, in Machine Learning and Cybernetics, 2004. Procee. 2004 Int. Conf. on 7, 3977–3980 (2004)
52. Kawaguchi, T., Rizon, M.: Iris detection using intensity and edge information. Pattern Recog. 36(2), 549–562 (2003)
53. Zhang, Y., Sun, N., Gao, Y., Cao, M.: A new eye location method based on ring gabor filter,” in Automation and Logistics, 2008. ICAL 2008. IEEE International Conference on, pp. 301–305. IEEE (2008)
54. Edwards, G. J., Taylor, C. J., Cootes, T. F.: Interpreting face images using active appearance models,” in Automatic Face and Gesture Recognition, 1998. Proceedings. Third IEEE International Conference on, pp. 300–305. IEEE (1998)
55. Ahlberg, J.: Ahlberg. Candide-3-an updated parameterised face (2001)
56. Ramirez, J., Gorriz, J. M., Segura, J. C.: Voice activity detection fundamentals and speech recognition system robustness (2007)
57. Rashwan, A.: Automatic driver fatigue monitoring using hidden markov models and bayesian networks Master’s thesis, University of Waterloo (2013)
58. Young, S. J., Evermann, G., Gales, M. J. F., Hain, T., Kershaw, D., Moore, G., Odell, J., Ollason, D., Povey, D., Valtchev, V., Woodland, P. C.: The HTK Book, version 3.4. Cambridge UK:Cambridge University Engineering Department (2006)
59. Zhang, N., Poole, D.: A simple approach to Bayesian network computations. In: Proceedings of the Tenth Canadian Conference on Artificial Intelligence, pp. 171–178 (1994)
60. car driving, C.: City car driving - car driving simulator, car game, http://citycardriving.com/ (2013)
61. Chang, C.-C., Lin, C.-J.: LIBSVM: A library for support vector machines. ACM Trans. Intell. Syst. Technol. 2, 27:1– 27:27 (2011). software available at, http://www.csie.ntu.edu.tw/cjlin/libsvm
62. Linde, Y., Buzo, A., Gray, R.: An algorithm for vector quantizer design. Commun. IEEE Trans. on 28 (1), 84–95 (1980)