Real-time nonintrusive monitoring and prediction of driver fatigue

IEEE Transactions on Vehicular Technology

Volumn 53, Issue 4, 2004, Pages 1052-1068

  Ji, Qiang ^a   Zhu, Zhiwei ^a   Lan, Peilin ^b  

^a RENSSELAER POLYTECHNIC INSTITUTE   (United States)

^b University of Nevada   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ACCIDENT PREVENTION; AUTOMOBILE DRIVERS; EYE MOVEMENTS; HUMAN ENGINEERING; MATHEMATICAL MODELS; MONITORING; MOTOR TRANSPORTATION; PROBABILITY; REAL TIME SYSTEMS; VIDEO CAMERAS;

DRIVER VIGILANCE; GAZE MOVEMENT; HEAD MOVEMENT; HUMAN FATIGUE; PROBABILISTIC MODEL; REAL TIME MONITORING; VISUAL CUES;

COMPUTER VISION;

EID: 3943085654     PISSN: 00189545     EISSN: None     Source Type: Journal    
DOI: 10.1109/TVT.2004.830974     Document Type: Article

References (56)

1. M. R. Rosekind, E. L. Co, K. B. Gregory, and D. L. Miller, "Crew factors in flight operations XIII: A survey of fatigue factors in corporate/executive aviation operations," National Aeronautics and Space Administration, Ames Research Center, Moffett Field, CA, NASA/TM-2000-209 610, 2000.
2. H. Saito, T. Ishiwaka, M. Sakata, and S. Okabayashi, "Applications of driver's line of sight to automobiles-what can driver's eye tell," in Proc. Vehicle Navigation Information Systems Conf., Yokohama, Japan, Aug. 1994, pp. 21-26.
3. H. Ueno, M. Kaneda, and M. Tsukino, "Development of drowsiness detection system," in Proc. Vehicle Navigation Information Systems Conf., Yokohama, Japan, Aug. 1994, pp. 15-20.
4. S. Boverie, J. M. Leqellec, and A. Hirl, "Intelligent systems for video monitoring of vehicle cockpit," in Proc. Int. Congr. Expo. ITS: Advanced Controls Vehicle Navigation Systems, 1998, pp. 1-5.
5. M. K. et al., "Development of a drowsiness warning system," presented at the Proc. 11th Int. Conf. Enhanced Safety Vehicle, Munich, Germany, 1994.
6. R. Onken, "Daisy, an adaptive knowledge-based driver monitoring and warning system," in Proc. Vehicle Navigation Information Systems Conf., Yokohama, Japan, Aug. 1994, pp. 3-10.
7. J. Feraric, M. Kopf, and R. Onken, "Statistical versus neural bet approach for driver behavior description and adaptive warning," Proc. 11th Eur. Annual Manual, pp. 429-436, 1992.
8. T. Ishii, M. Hirose, and H. Iwata, "Automatic recognition of driver's facial expression by image analysis," J. Soc. Automotive Eng. Japan, vol. 41, no. 12, pp. 1398-1403, 1987.
9. K. Yammamoto and S. Higuchi, "Development of a drowsiness warning system," J. Soc. Automotive Eng. Japan, vol. 46, no. 9, pp. 127-133, 1992.
10. D. Dinges and M. Mallis, "Managing fatigue by drowsiness detection: Can technological promises be realized? in managing fatigue in transportation," in Managing Fatigue in Transportation: Selected Papers from the 3rd Fatigue in Transportation Conference, Fremantle, Western Australia, L. R. Hartley, Ed. Oxford, U.K.: Elsevier, 1998, pp. 209-229.
11. S. Charlton and P. Baas, "Fatigue and fitness for duty of New Zealand truck drivers," presented at the Road Safety Conf., Wellington, New Zealand, 1998.
12. T. Akerstedt and S. Folkard, "The three-process model of alertness and its extension to performance, sleep latency and sleep length," Chronobio. Int., vol. 14, no. 2, pp. 115-123, 1997.
13. G. Belenky, T. Balkin, D. Redmond, H. Sing, M. Thomas, D. Thorne, and N. Wesensten, "Sustained performance during continuous operations: The us armys sleep management system. In managing fatigue in transportation," in Managing Fatigue in Transportation: Selected Papers from the 3rd Fatigue in Transportation Conference, Fremantle, Western Australia, L. R. Hartley, Ed. Oxford, U.K.: Elsevier, 1998.
14. D. Dawson, N. Lamond, K. Donkin, and K. Reid, "Quantitative similarity between the cognitive psychomotor performance decrement associated with sustained wakefulness and alcohol intoxication. In managing fatigue in transportation," in Managing Fatigue in Transportation: Selected Papers from the 3rd Fatigue in Transportation Conference, Fremantle, Western Australia, L. R. Hartley, Ed. Oxford, U.K.: Elsevier, 1998, pp. 231-256.
15. P. Artaud, S. Planque, C. Lavergne, H. Cara, P. de Lepine, C. Tarriere, and B. Gueguen, "An on-board system for detecting lapses of alertness in car driving," presented at the 14th Int. Conf. Enhanced Safety of Vehicles, vol. 1, Munich, Germany, 1994.
16. N. Mabbott, M. Lydon, L. Hartley, and P. Arnold, "Procedures and devices to monitor operator alertness whilst operating machinery in open-cut coal mines. Stage 1: State-of-the-art review," ARRB Transport Res. Rep. RC 7433, 1999.
17. C. Lavergne, P. De Lepine, P. Artaud, S. Planque, A. Domont, C. Tarriere, C. Arsonneau, X. Yu, A. Nauwink, C. Laurgeau, J. Alloua, R. Bourdet, J. Noyer, S. Ribouchon, and C. Confer, "Results of the feasibility study of a system for warning of drowsiness at the steering wheel based on analysis of driver eyelid movements," presented at the Proc. 15th Int. Tech. Conf. Enhanced Safety Vehicles, vol. 1, Melbourne, Australia, 1996.
18. E. Grandjean, Fitting the Task to the Man, 4th ed. London, U.K.: Taylor & Francis, 1988.
19. D. Cleveland, "Unobtrusive eyelid closure and visual point of regard measurement system," presented at the Proc. Ocular Measures Driver Alertness Tech. Confe., Herndon, VA, Apr. 26-27, 1999.
20. R. J. E Carroll, "Ocular measures of driver alertness technical conference proceedings," Federal Highway Administration, Office of Motor Carrier and Highway Safety, Washington, DC, FHWA Tech. Rep. FHWA-MC-99-136, 1999.
21. L. Hartley, T. Horberry, N. Mabbott, and G. Krueger, Review of Fatigue Detection and Prediction Technologies. Melbourne, Australia: National Road Transport Commission, 2000.
22. S. Saito, "Does fatigue exist in a quantitative of eye movement ?" Ergonomics, vol. 35, pp. 607-615, 1992.
23. Appl. Sci. Lab., "PERCLOS and eyetracking: Challenge and opportunity," Tech. Rep., Appl. Sci. Lab., Bedford, MA, 1999.
24. Conf. Ocular Measures of Driver Alertness, Washington, DC, Apr. 26-27, 1999.
25. D. F. Dinges, M. Mallis, G. Maislin, and J. W. Powell, "Evaluation of techniques for ocular measurement as an index of fatigue and the basis for alertness management," Dept. Transp. Highway Safety, pub. 808 762, 1998.
26. R. Grace, "A drowsy driver detection system for heavy vehicles," presented at the Conf. Ocular Measures Driver Alertness, Apr. 1999.
27. D. Cleveland, "Unobtrusive eyelid closure and visual of regard measurement system," presented at the Conf. Ocular Measures Driver Alertness, Apr. 1999.
28. J. Fukuda, K. Adachi, M. Nishida, and A. E., "Development of driver's drowsiness detection technology," Toyota Techn. Rev., vol. 45, pp. 34-40, 1995.
29. J. H. Richardson, "The development of a driver alertness monitoring system," in Fatigue and Driving: Driver Impairment, Driver Fatigue and Driver Simulation, L. Harrtley, Ed. London, U.K.: Taylor & Francis, 1995.
30. J. Dowdall, I. Pavlidis, and G. Bebis, "A face detection method based on multi-band feature extraction in the near-IR spectrum," presented at the IEEE Workshop Computer Vision Beyond Visible Spectrum, Honolulu, HI, Dec. 14, 2001.
31. A. Haro, M. Flickner, and I. Essa, "Detecting and tracking eyes by using their physiological properties, dynamics, and appearance," presented at the Proc. IEEE Conf. Computer Vision and Pattern Recognition, Hilton Head Island, SC, 2000.
32. W. Mullally and M. Betke, "Preliminary investigation of real-time monitoring of a driver in city traffic," presented at the IEEE Int. Conf. Intelligent Vehicles, Dearborn, MI, 2000.
33. Z. Zhu, Q. Ji, K. Fujimura, and K. c. Lee, "Combining Kalman filtering and mean shift for real time eye tracking under active IR illumination," presented at the Int. Conf. Pattern Recognition, Quebec, PQ, Canada, 2002.
34. Z. Zhu, K. Fujimura, and Q. Ji, "Real-time eye detection and tracking under various light conditions," presented at the Symp. Eye Tracking Research Applications, New Orleans, LA, 2002.
35. C. Cortes and V. Vapnik, "Support-vector networks," Machine Learning, vol. 20, pp. 273-297, 1995.
36. J. Huang, D. Ii, X. Shao, and H. Wechsler, "Pose discrimination and eye detection using support vector machines (SVM's)," in Proc. NATO Advanced Study Institute (ASI) Face Recognition: From Theory to Applications, 1998, pp. 528-536.
37. Q. Ji and X. Yang, "Real time visual cues extraction for monitoring driver vigilance," presented at the Proc. Int. Workshop Computer Vision Systems, Vancouver, BC, Canada, 2001.
38. Z. Zhu and Q. Ji, "3D face pose tracking from an uncalibrated monocular camera," presented at the Int. Conf. Pattern Recognition, Cambridge, U.K., 2004.
39. Y. Ebisawa, "Unconstrained pupil detection technique using two light sources and the image difference method," in Visualization and Intelligent Design in Engineering and Architecture II. Boston, MA: Computational Mechanics, 1989, pp. 79-89.
40. T. E. Hutchinson, "Eye Movement Detection with Improved Calibration and Speed," U.S. Patent 4 950 069, 1988.
41. T. E. Hutchinson, K. White, J. R. Worthy, N. Martin, C. Kelly, R. Lisa, and A. Frey, "Human-computer interaction using eye-gaze input," IEEE Trans. Syst., Man, Cybern., vol. 19, pp. 1527-1533, Nov./Dec. 1989.
42. T. Ohno, N. Mukawa, and A. Yoshikawa, "Freegaze: A gaze tracking system for everyday gaze interaction," presented at the Eye Tracking Research Applications Symp., New Orleans, LA, Mar. 25-27, 2002.
43. D. Koons and M. Flickner. IBM Blue Eyes Project. [Online]. Available: http://www.almaden.ibm.com/cs/blueeyes
44. C. H. Morimoto, D. Koons, A. Amir, and M. Flickner, "Frame-rate pupil detector and gaze tracker," presented at the IEEE Int. Conf. Computer Vision (ICCV) Frame-Rate Workshop, 1999.
45. Y. Ebisawa, "Improved video-based eye-gaze detection method," IEEE Trans. Instrum. Meas., vol. 47, pp. 948-955, Apr. 1998.
46. Q. Ji and Z. Zhu, "Eye and gaze tracking for interactive graphic display," presented at the 2nd Int. Symp. Smart Graphics, Hawthorne, NY, 2002.
47. Q. Ji and X. Yang, "Real time 3D face pose discrimination based on active IR illumination (oral)," presented at the Int. Conf. Pattern Recognition, 2002.
48. D. F. Specht, "A general regression neural network," IEEE Trans. Neural Networks, vol. 3, pp. 568-576, Nov. 1991.
49. T. Lee, "Image representation using 2D Gabor wavelets," IEEE Trans. Pattern Anal. Machine Intell., vol. 18, pp. 959-971, Oct. 1996.
50. H. Gu, Q. Ji, and Z. Zhu, "Active facial tracking for fatigue detection," presented at the IEEE Workshop Applications of Computer Vision, Orlando, FL, 2002.
51. M. I. Jordan, Learning in Graphical Models. Cambridge, MA: MIT Press, 1999.
52. E. L. Co, K. B. Gregory, J. M. Johnson, and M. R. Rosekind, "Crew factors in flight operations XI: A survey of fatigue factors in regional airline operations," Nat. Aeron. Space Admin., Ames Res. Center, Moffett Field, CA, NASA/TM-1999-208 799, 1999.
53. P. Sherry, Fatigue Countermeasures in the Railroad Industry-Past and Current Developments. Denver, CO: Univ. Denver, Intermodal Transportation Institute, Counseling Psychology Program, 2000.
54. M. R. Rosekind, K. B. Gregory, E. L. Co, D. L. Miller, and D. F. Dinges, "Crew factors in flight operations XII: A survey of sleep quantity and quality in on-board crew rest facilities," Nat. Aeron. Space Admin., Ames Res. Center, Moffett Field, CA, NASA/TM-2000-209 611, 2000.
55. F. V. Jensen, "Bayesian networks and decision graphs," in Statistics for Engineering and Information Science. New York: Springer-Verlag, 2001.
56. Online MSBNx Editor Manual and Software Download. M. R. Center. [Online]. Available: http://research.microsoft.com/adapt/MSBNx/