红外与激光工程, 2018, 47 (2): 0203009, 网络出版: 2018-04-26  

基于多形态红外特征与深度学习的实时驾驶员疲劳检测

Real-time driver fatigue detection based on morphology infrared features and deep learning
作者单位
1 天津市光电检测技术与系统重点实验室, 天津 300387
2 天津工业大学 电子与信息工程学院, 天津 300387
3 天津工业大学 计算机科学与软件学院, 天津 300387
摘要
疲劳驾驶是导致车祸的重要诱因, 严重危害道路交通安全, 而车辆行驶过程中的光照条件变化、驾驶员姿态调整和眼镜遮挡等因素将对疲劳检测任务产生不利的影响。针对以上问题, 提出了基于深度学习的驾驶员疲劳检测算法。首先, 使用850 nm红外光源补光, 在复杂光照和遮挡形态下采集驾驶员的面部图像; 其次, 利用红外图像中的多种特征, 通过级联CNN确定人脸边框和特征点位置, 提取眼睛区域并识别眼睛的睁闭状态; 最后, 将眼睛状态识别结果和连续图像中的特征点坐标差值输出至LSTM网络, 检测驾驶员疲劳状态。实验结果表明: 该疲劳检测算法的准确率可达94.48%, 平均检测时间为65.64 ms。
Abstract
Fatigue driving is the main cause or reason for traffic accidents, which has a huge influence on social safety. Considering the fact that light change and glasses could significantly increase the difficulty to monitor human eyes, fatigue detection was still an unsolved problem. A new driver fatigue method based on morphology infrared features and deep learning were proposed. Using 850 nm infrared light source, the facial image was obtained. Human faces and landmarks which indicated the area of eyes were located by Convolution Neural Network(CNN) with morphology features in infrared image. In the next step, a filter module which measured head displacement was added, aiming at reducing the impact of posture change. In the following, the collected facial states were transformed into sequential data. Finally, the sequential data was passed to the Long Short Term Memory(LSTM) network to detect fatigue state by analyzing the sequential correlations. Experimental results show that the accuracy of the fatigue detection algorithm can reach 94.48% with an average detection time of 65.64 ms.
参考文献

[1] Katsis C D, Ntouvas N E, Bafas C G, et al. Assessment of muscle fatigue during driving using surface EMG[C]// Proceedings of the IASTED International Conference on Biomedical Engineering, 2004: 259-262.

[2] Ohsuga M, Kamakura Y, Roongroj Nopsuwanchai, et al. Classification of blink waveforms toward the assessment of driver′s arousal levels-an EOG approach and the correlation with physiological measures[C]//Engineering Psychology and Cognitive Ergonomics, 2007: 787-795.

[3] Takahashi I, Yokoyama K. Development of a feedback stimulation for drowsy driver using heartbeat rhythms[C]//Engineering in Medicine and Biology Society, 2011: 4153-4158.

[4] Jin Xue. Research on the detection method of fatigue driving based on driving behavior[D]. Beijing: Beijing University of Technology, 2015. (in Chinese)

[5] Celenk M, Eren H, Poyraz M. Prediction of driver head movement via Bayesian learning and ARMA modeling[C]//Intelligent Vehicles Symposium, 2009: 542-547.

[6] Friedrichs F, Yang B. Drowsiness monitoring by steering and lane data based features under real driving conditions[C]//European Signal Processing Conference, 2010: 209-213.

[7] Tao H, Zhao Y. Real-time driver fatigue detection based on face alignment[C]//International Conference on Digital Image Processing, 2017: 1042003.

[8] Meng C N, Bai J J, Zhang T N, et al. Eye movement analysis for activity recognition based on one web camera[J]. Acta Physica Sinica, 2013, 62(17): 116-121.

[9] Cheng Ruzhong, Zhao Yong, Dai Yong, et al. An on-board embedded driver fatigue warning system based on adaboost method[J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2012, 48(5): 719-726. (in Chinese)

[10] Kuang Wenteng, Mao Kuancheng, Hong Jiacai, et al. Fatigue driving detection based on sclera Gaussian model[J]. Journal of Image and Graphics, 2016, 21(11): 1515-1522.

[11] Wang Jizhou, Li Hong, Xiong Yuqing, et al. Design and preparation of one induced transmission filter with anlistatig and anti-infrared character[J]. Infrared and Laser Engineering, 2015, 44(10): 3005-3009. (in Chinese)

[12] Boser B, Denker J S. Backpropagation applied to handwritten zip code recognition[J]. Neural Computation, 1989, 1(4): 541-551.

[13] Farhan M, Yli-Harja O, Niemist A. A novel method for splitting clumps of convex objects incorporating image intensity and using rectangular window-based concavity point-pair search [J]. Pattern Recognition, 2013, 46(3):741-751.

[14] Alex Krizhevsky, Llya Sutskever, Geoffrey E, et al. ImageNet classification with deep convolutional neural networks[J]. Communications of the Acm, 2012, 60(2): 2012.

[15] Girshick R, Donahue J, Darrell T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]//Computer Vision and Pattern Recognition, 2014: 580-587.

[16] Girshick R. Fast R-CNN[C]//IEEE International Conference on Computer Vision, 2015: 1440-1448.

[17] Ren S, He K, Girshick R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[C]// International Conference on Neural Information Processing Systems, 2015: 91-99.

[18] Redmon J, Divvala S, Girshick R, et al. You only look once: unified real-time object detection [C]//Computer Vision and Pattern Recognition, 2015, arxiv: 1506, 02640.

[19] Donahue J, Hendricks L A, Rohrbach M, et al. Long-term recurrent convolutional networks for visual recognition and description[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2014, 39(4): 677-691.

[20] Zhang K, Zhang Z, Li Z, et al. Joint face detection and alignment using multitask cascaded convolutional networks[J]. IEEE Signal Processing Letters, 2016, 23(10): 1499-1503.

[21] Li Rui. Facial expression recognition method based on feature fusion[D]. Hefei: Hefei University of Technology, 2013. (in Chinese)

[22] Hu Feng. Multi-source information fusion application to driving fatigue detection[D]. Hefei: Anhui University of Science and Technology, 2014. (in Chinese)

[23] Chen Mingchu. Research on driver fatigue detection based on eye state[D]. Chongqing: Chongqing University, 2012.(in Chinese)

[24] Ma Ying. Research on driver fatigue alarm system based on facial features[D]. Hefei: Anhui University of Technology,2016. (in Chinese)

[25] Zhang F, Su J, Geng L, et al. Driver fatigue detection based on eye state recognition[C]//International Conference on Machine Vision and Information Technology, 2017: 105-110.

[26] Sepp Hochreiter, Jürgen Schmidhuber. Long short-term memory[J]. Neural Computation, 1997, 9(8): 1735-1780.

[27] Ma X, Hovy E. End-to-end sequence labeling via Bi-directional LSTM-CNNs-CRF[C]//Meeting of the Association for Computational Linguistics, 2016: 1064-1074.

[28] He K, Zhang X, Ren S, et al. Delving deep into rectifiers: surpassing human-level performance on imageNet classification[C]//IEEE International Conference on Computer Vision, 2016: 1026-1034.

耿磊, 梁晓昱, 肖志涛, 李月龙. 基于多形态红外特征与深度学习的实时驾驶员疲劳检测[J]. 红外与激光工程, 2018, 47(2): 0203009. Geng Lei, Liang Xiaoyu, Xiao Zhitao, Li Yuelong. Real-time driver fatigue detection based on morphology infrared features and deep learning[J]. Infrared and Laser Engineering, 2018, 47(2): 0203009.

关于本站 Cookie 的使用提示

中国光学期刊网使用基于 cookie 的技术来更好地为您提供各项服务,点击此处了解我们的隐私策略。 如您需继续使用本网站,请您授权我们使用本地 cookie 来保存部分信息。
全站搜索
您最值得信赖的光电行业旗舰网络服务平台!