Vision-based method for detecting driver drowsiness and distraction in driver monitoring system

Optical Engineering

Volumn 50, Issue 12, 2011, Pages

  Jo, Jaeik ^a   Lee, Sung Joo ^a   Jung, Ho Gi ^b   Park, Kang Ryoung ^c   Kim, Jaihie ^a  

^a Yonsei University   (South Korea)

^b Hanyang University   (South Korea)

^c Dongguk University   (South Korea)

Author keywords

blink detection; computer vision; distractions; driver monitoring systems; drowsiness; drowsy driver detection; feature selection; inattention; machine learning

Indexed keywords

DISCRIMINANT ANALYSIS; ERROR DETECTION; EYE PROTECTION; FEATURE EXTRACTION; LEARNING SYSTEMS; PRINCIPAL COMPONENT ANALYSIS; SIGNAL DETECTION; TEMPLATE MATCHING;

BLINK DETECTIONS; DISTRACTION; DRIVER MONITORING SYSTEM; DROWSINESS; DROWSY DRIVER; DROWSY DRIVER DETECTION; EYE DETECTION; FEATURES SELECTION; INATTENTION; MACHINE-LEARNING;

COMPUTER VISION;

EID: 85040195302     PISSN: 00913286     EISSN: 15602303     Source Type: Journal    
DOI: 10.1117/1.3657506     Document Type: Article

References (70)

1. Rosekind M. R., Co E. L., Gregory K. B., and Miller D. L., " Crew factors in flight operations XIII: a survey of fatigue factors in corporate/executive aviation operations.," NASA, Ames Research Center, NASA/TM-2000-209 610 (2000).
2. Awake Consortium (IST 2000-28062), " System for effective assessment of driver vigilance and warning according to traffic risk estimation (AWAKE).," September 2001-2004. < http://www.awake-eu.org > (April 16, 2011).
3. U.S. Department of Transportation, " 2007 FARS/GES annual report (final ed.).," < http://www.nhtsa.dot.gov/portal/site/nhtsa/menuitem.6a6eaf83cf719ad24ec86e10dba046a0/ > (April 16, 2011).
4. Wierwille W., Tijerina L., Kiger S., Rockwell T., Lauber E., and Bittne A., " Final report supplement-heavy vehicle driver workload assessment-task 4: review of workload and related research.," USDOT, Tech. Rep. No. DOT HS 808 467(4) (1996).
5. Saito H., Ishiwaka T., Sakata M., and Okabayashi S., " Applications of driver's line of sight to automobiles-what can driver's eye tell.," in Proc. of Vehicle Navigation and Information Systems Conf., Yokohama, pp. 21-26 (1994).
6. Ueno H., Kaneda M., and Tsukino M., " Development of drowsiness detection system.," in Proc. of Vehicle Navigation and Information Systems Conf., Yokohama, pp. 15-20 (1994).
7. Boverie S., Leqellec J. M., and Hirl A., " Intelligent systems for video monitoring of vehicle cockpit.," in Proc. of Int. Congress and Exposition ITS, Advanced Controls and Vehicle Navigation Systems, pp. 1-5 (1998).
8. Kaneda M., Ueno H., and Tsukino M., " Development of a drowsiness warning system.," in Proc. of 11th Int. Conf. on Enhanced Safety of Vehicle, Munich (1994).
9. Onken R., " Daisy, an adaptive knowledge-based driver monitoring and warning system.," in Proc, of Vehicle Navigation and Information Systems Conf., Yokohama, pp. 3-10 (1994).
10. Ishii T., Hirose M., and Iwata H., " Automatic recognition of driver's facial expression by image analysis.," Journal of the Society of Automotive Engineers of Japan, Vol. 41, pp. 1398-1403 (1987).
11. Yammamoto K., and Higuchi S., " Development of a drowsiness warning system.," Journal of the Society of Automotive Engineers of Japan, Vol. 46, pp. 127-133 (1992).
12. Dinges D. F., Mallis M., Maislin G., and Powell J. W., " Evaluation of techniques for ocular measurement as an index of fatigue and the basis for alertness management.," Department of Transportation Highway Safety Publication 808 762, April (1998).
13. Grace R., Byrne V. E., Bierman D. M., and Legrand J. M., " A drowsy driver detection system for heavy vehicles.," in Proceedings of Digital Avionics Systems Conference, 17th DASC, Vol. 2, pp. I36/1-I36/8 (1998).
14. Cleveland D., " Unobtrusive eyelid closure and visual of regard measurement system.," Conference on Ocular Measures of Driver alertness, April (1999).
15. Fukuda J., Adachi K., Nishida M., and Akutsu E., " Development of driver's drowsiness detection technology.," Toyota Tech. Rev. 45, 34-40 (1995).
16. Richardson J. H., " The development of a driver alertness monitoring system.," in Fatigue and Driving: Driver Impairment, Driver Fatigue and Driver Simulation, Harrtley L., Ed., Taylor & Francis, London (1995).
17. D'Orazio T., Leo M., Guaragnella C., and Distante A., " A visual approach for driver inattention detection.," Pattern Recogn. 40, 2341-2355 (2007).
18. Murphy-Chutorian E., and Trivedi M. M., " Head pose estimation in computer vision: a survey.," IEEE Trans. Pattern Anal. Mach. Intell. 31 (4), 607-626 (2009).
19. Hansen D. W., and Ji Q., " In the eye of the beholder: a survey of models for eyes and gaze.," IEEE Trans. Pattern Anal. Mach. Intell. 32, 478-500 (2010).
20. Hattori A., Tokoro S., Miyashita M., Tanakam I., Ohue K., and Uozumi S., " Development of forward collision warning system using the driver behavioral information.," presented at 2006 SAE World Congress, Detroit, Michigan (2006).
21. Murphy-Chutorian E., Doshi A., and Trivedi M. M., " Head pose estimation for driver assistance systems: a robust algorithm and experimental evaluation.," in Proc. of 10th Int. IEEE Conf. Intelligent Transportation Systems, pp. 709-714 (2007).
22. Smith P., Shah M., and da Vitoria Lobo N., " Determining driver visual attention with one camera.," IEEE Trans. Intell. Transp. Syst. 4 (4), 2058-2218 (2003).
23. Kaminski J. Y., Knaan D., and Shavit A., " Single image face orientation and gaze detection.," Mach. Vis. Appl. 21, 85-98 (2009).
24. Stutts J. C., Reinfurt D. W., Staplin L., and Rodgman E. A., " The role of driver distraction in traffic crashes.," AAA Foundation for Traffic Safety, Washington, DC, (2001).
25. Torkkola K., Massey N., and Wood C., " Driver inattention detection through intelligent analysis of readily available sensors.," in Proc. of IEEE Conf. on Intelligent Transportation Systems, Washington, DC, pp. 326-331 (2004).
26. Yang G., Lin Y., and Bhattacharya P., " A driver fatigue recognition model using fusion of multiple features.," in Proc. of IEEE Int. Conf. on Syst., Man and Cybernetics, Hawaii Vol. 2, pp. 1777-1784 (2005).
27. Park S., and Trivedi M. M., " Driver activity analysis for intelligent vehicles: issues and development framework.," in Proc. of IEEE Intelligent Vehicles Symp., Las Vegas, pp. 795-800 (2005).
28. Vural E., Cetin M., Ercil A., Littlewort G., Bartlett M. S., and Movellan J. R., " Drowsy driver detection though facial movement analysis.," IEEE Int. Conf. on Computer Vision-Human Computer Interaction, pp. 6-18 (2007).
29. Saradadevi M., and Bajaj P., " Driver fatigue detection using mouth and yawning analysis.," Int. J. Comput. Sci. Netw. Security 8 (6), 183-188 (2008).
30. Beragasa L. M., Nuevo J., Sotelo M. A., Barea R., and Lopez E., " Real-time system for monitoring driver vigilance.," IEEE Trans. Intell. Transport. Syst. 7 (1), 1524-1538 (2006).
31. Senaratne R., Hardy D., Vanderaa B., and Halgamuge S., " Driver fatigue detection by fusing multiple cues.," Lect. Notes Comput. Sci. 4492, 801-809 (2007).
32. Kircher K., Kircher A., and Claezon F., " Distraction and drowsiness-a field study.," Tech. Rep., VTI (The Swedish National Road and Transport Research Institute) (2009).
33. Lee S. J., Jo J., Jung H. G., Park K. R., and Kim J., " Real-time gaze estimator based on driver's head orientation for forward collision warning system.," IEEE Trans. Intell. Transp. Syst. 12 (1), 254-267 (2011).
34. Specification of illuminator, < http://roithner-laser.com/multi/led_multi_to.html ≥ (July 25, 2011).
35. Camera (EC650), < http://www.pyramidimaging.com/specs/Prosilica/700009AA-EC650%20User%20Manual.pdf > (August 25, 2011).
36. Camera lens (ML0614), < http://www.phoeniximaging.com/ml-0614.htm ≥ (July 25, 2011).
37. Narrow bandpass interference filter (NT43-148), < http://www.edmundoptics.com/onlinecatalog/displayproduct.cfm?productid=3198&PageNum=5&Sort=displayOrder&Order=asc#products > (July 25, 2011).
38. SUV (GIA motors Sportage), < http://www.kia.com/#/sportage/explore/ > (August 25, 2011).
39. Sedan (Hyundai Motors Grandeur), < http://grandeur.hyundai.com ≥ (August 25, 2011).
40. Ito T., Mita S., Kozuka K., Nakano T., and Yamamoto S., " Driver blink measurement by the motion picture processing and its application to drowsiness detection.," in Proc. of Int. Conf. Intelligent Transportation Systems, Singapore, pp. 168-173 (2002).
41. Ohue K., Yamada Y., Uozumi S., Tokoro S., Hattori A., and Hayashi T., " Development of a new pre-crash safety system.," SAE Tech. Paper Series, Doc. No. 2006-01-1461 (2006).
42. Specification of motion tracking device called Patriot, < http://www.polhemus.com/polhemus_editor/assets/Tech%20compare%20with%20ResolutionvsRange%20graphs.pdf > (February 15, 2011).
43. Scheirer W. J., Rocha A., Heflin B., and Boult T. E., " Difficult detection: a comparison of two different approaches to eye detection for unconstrained environments.," in Proc. of IEEE 3rd Int. Conf. on Biometrics: Theory, Applications, and Systems, pp. 1-8 (2009).
44. Whitelam C., Jafri Z., and Bourlai T., " Multispectral eye detection: a preliminary study.," in Proc. of 2010 Int. Conf. on Pattern Recognition, pp. 209-212 (2010).
45. Viola P., and Jones M., " Rapid object detection using a boosted cascade of simple features.," in Proc. of IEEE CVPR, Vol. 1, pp. I-511-I-518 (2001).
46. Wei S. D, and Lai S. H, " Fast template matching based on normalized cross correlation with adaptive multilevel winner update.," IEEE Trans. Image Process. 17 (11), 2227-2235 (2008).
47. Open source of blob detection, < http://opencv.willowgarage.com/wiki/cvBlobsLib#Algorithm > (July 30, 2011).
48. Gonzalez R. C., and Woods R. E., Digital Image Processing, 2nd ed., pp. 523-532, Prentice-Hall, Englewood Cliffs, NJ (2002).
49. Donato G., Bartlett M. S., Hager J. C., Ekman P., and Sejnowski T. J., " Classifying facial actions.," IEEE Trans. Pattern Anal. Mach. Intell. 21, 974-989 (1999).
50. Bellhumeur P. N., Hespanha J. P., and Kriegman D. J., " Eigenfaces vs. fisherfaces: recognition using class specific linear projection.," IEEE Trans. Pattern Anal. Mach. Intell. 19, 711-720 (1997).
51. Martinez A. M., and Kak A. C., " PCA versus LDA.," IEEE Trans. Pattern Anal. Mach. Intell. 23, 228-233 (2001).
52. Turk M., and Pentland A., " Eigenfaces for recognition.," J. Cognitive Neuro-sci. 3, 71-86 (1991).
53. Deng H., Jin L., Zhen L., and Huang J., " A new facial expression recognition method nased on local gabor filter bank and PCA plus LDA.," in Proc. of Int. Conf. on Intelligent Computing, August 23-26, Hefei, China, pp. 6-7 (2005).
54. Vapnik V., " Support vector estimation of functions.," in Statistical Learning Theory, pp. 375-570, Wiley, Hoboken, NJ (1998).
55. Hsu C., Chang C., and Lin C.,. " A practical guide to support vector classification.," Tech. Rep., Department of Computer Science and Information Engineering, National Taiwan University, Taipei, < http://w.csie.org/∼cjlin/papers/guide/guide.pdf > (March. 23, (2011).
56. Lang L., and Qi H., " The study of driver fatigue monitor algorithm combined PERCLOS and AECS.," in Proc. of Int. Conf. on Comput. Sci. and Software Eng., pp. 349-352 (2008).
57. Harley L., Horberry T., Mabbott N., and Krueger G., " Review of fatigue detection and prediction technologies.," National Road Transport Commission, (2000).
58. Jain R., Kasturi R., and Schunck B. G., Machine Vision, pp. 78-84 McGraw-Hill, New York (1995).
59. Hoyer P. O., " Non-negative matrix factorization with sparseness constraints.," J. Mach. Learning Res. 5, 1457-1469 (2004).
60. Hyvarinen A., and Oja E., " Independent component analysis.," Neural Netw. 13 (4-5), 411-430 (2000).
61. Ji Q., and Yang X., " Real-time eye, gaze, and face pose tracking for monitoring driver vigilance.," Real-Time Imaging 8 (5), 357-377 (2002).
62. Dinges D. F., Mallis M. M., Maislin G., and Powell J. W., " Evaluation of techniques for ocular measurement as an index of fatigue and the basis for alertness management.," U.S. Department of Transportation: National Highway Traffic Safety Administration, DOT HS 808 762 (1998).
63. Smart Eye Pro, < http://www.smarteye.se > (25 July 2011).
64. Kawato S., and Ohya J., " Real-time detection of nodding and head-shaking by directly detecting and tracking the between-eyes.," in Proc. of 4th Int'l IEEE Conf. Automatic Face and Gesture Recognition, pp. 40-45 (2000).
65. Ma B., Shan S., Chen X., and Gao W., " Head yaw estimation from asymmetry of facial appearance.," IEEE Trans. Syst., Man, Cybern. Part B 38 (6), 1501-1512 (2008).
66. Watta P., Lakshmanan S., and Hou Y., " Nonparametric approaches for estimating driver pose.," IEEE Trans. Vehicular Technol. 56 (4), 2028-2041 (2007).
67. Wu J., and Trivedi M. M., " A two-stage head pose estimation framework and evaluation.," Pattern Recogn. 41, 1138-1158 (2008).
68. Murphy-Chutorian E., and Trivedi M. M., " HyHOPE: hybrid head orientation and position estimation for vision-based driver head tracking.," in Proc. of IEEE Intell. Vehicles Symp., pp. 512-517 (2008).
69. Batista J. P., " A real-time driver visual attention monitoring system.," Lect. Notes Comput. Sci. 3522, 200-208 (2005). 10.1007/11492429_25
70. Ratha N. K., and Govindaraju V., Advances in Biometrics: Sensors, Algorithms and System, Springer, New York (2008).