Buscar en
Revista Iberoamericana de Automática e Informática Industrial RIAI
Toda la web
Inicio Revista Iberoamericana de Automática e Informática Industrial RIAI Sistema Automático Para la Detección de Distracción y Somnolencia en Conducto...
Información de la revista
Vol. 14. Núm. 3.
Páginas 307-328 (Julio - Septiembre 2017)
Compartir
Compartir
Descargar PDF
Más opciones de artículo
Visitas
3154
Vol. 14. Núm. 3.
Páginas 307-328 (Julio - Septiembre 2017)
Open Access
Sistema Automático Para la Detección de Distracción y Somnolencia en Conductores por Medio de Características Visuales Robustas
Automatic System to Detect Both Distraction and Drowsiness in Drivers Using Robust Visual Features
Visitas
3154
Alberto Fernández Villána,
Autor para correspondencia
alberto.fernandez@grupotsk.com

Autor para correspondencia.
, Rubén Usamentiaga Fernándezb, Rubén Casado Tejedorb
a Grupo TSK, Parque Científico y Tecnológico de Gijón, 33203 Gijón, Asturias, España
b Universidad de Oviedo, Campus de Viesques, 33204 Gijón, Asturias, España
Este artículo ha recibido

Under a Creative Commons license
Información del artículo
Resumen
Texto completo
Bibliografía
Descargar PDF
Estadísticas
Resumen

De acuerdo con un reciente estudio publicado por la Organización Mundial de la Salud (OMS), se estima que 1.25 millones de personas mueren como resultado de accidentes de tráfico. De todos ellos, muchos son provocados por lo que se conoce como inatención, cuyos principales factores contribuyentes son tanto la distracción como la somnolencia. En líneas generales, se calcula que la inatención ocasiona entre el 25% y el 75% de los accidentes y casi-accidentes. A causa de estas cifras y sus consecuencias se ha convertido en un campo ampliamente estudiado por la comunidad investigadora, donde diferentes estudios y soluciones han sido propuestos, pudiendo destacar los métodos basados en visión por computador como uno de los más prometedores para la detección robusta de estos eventos de inatención. El objetivo del presente artículo es el de proponer, construir y validar una arquitectura especialmente diseñada para operar en entornos vehiculares basada en el análisis de características visuales mediante el empleo de técnicas de visión por computador y aprendizaje automático para la detección tanto de la distracción como de la somnolencia en los conductores. El sistema se ha validado, en primer lugar, con bases de datos de referencia testeando los diferentes módulos que la componen. En concreto, se detecta la presencia o ausencia del conductor con una precisión del 100%, 90.56%, 88.96% por medio de un marcador ubicado en el reposacabezas del conductor, por medio del operador LBP, o por medio del operador CS-LBP, respectivamente. En lo que respecta a la validación mediante la base de datos CEW para la detección del estado de los ojos, se obtiene una precisión de 93.39% y de 91.84% utilizando una nueva aproximación basada en LBP (LBP_RO) y otra basada en el operador CS-LBP (CS-LBP_RO). Tras la realización de varios experimentos para ubicar la cámara en el lugar más adecuado, se posicionó la misma en el salpicadero, pudiendo aumentar la precisión en la detección de la región facial de un 86.88% a un 96.46%. Las pruebas en entornos reales se realizaron durante varios días recogiendo condiciones lumínicas muy diferentes durante las horas diurnas involucrando a 16 conductores, los cuales realizaron diversas actividades para reproducir síntomas de distracción y somnolencia. Dependiendo del tipo de actividad y su duración, se obtuvieron diferentes resultados. De manera general y considerando de forma conjunta todas las actividades se obtiene una tasa media de detección del 93.11%.

Palabras clave:
Detección distracción y somnolencia
Visión por computador
Percepción y reconocimiento
Aprendizaje automático
Monitorización y supervisión
Abstract

According to the most recent studies published by the World Health Organization (WHO) in 2013, it is estimated that 1.25 million people die as a result of traffic crashes. Many of them are caused by what it is known as inattention, whose main contributing factors are both distraction and drowsiness. Overall, it is estimated that inattention causes between 25% and 75% of the crashes and near-crashes. That is why this is a thoroughly studied field by the research community, where solutions to combat distraction and drowsiness, in particular, and inattention, in general, can be classified into three main categories, and, where computer vision has clearly become a non-obtrusive effective tool for the detection of both distraction and drowsiness. The aim of this paper is to propose, build and validate an architecture based on the analysis of visual characteristics by using computer vision techniques and machine learning to detect both distraction and drowsiness in drivers. Firstly, the modules have been tested with all its components independently using several datasets. More specifically, the presence/absence of the driver is detected with an accuracy of 100%, 90.56%, 88.96% by using a marker positioned onto the headrest, the LBP operator and the CS-LBP operator, respectively. Regarding the eye closeness validation with CEW dataset, an accuracy of 93.39% and 91.84% is obtained using a new method using both LBP (LBP_RO) and CS-LBP (CS-LBP_RO). After performing several tests, the camera is positioned on the dashboard, increasing the accuracy of face detection from 86.88% to 96.46%. In connection with the tests performed in real-world settings, 16 drivers were involved performing several activities imitating different sings of sleepiness and distraction. Overall, an accuracy of 93.11%is obtained considering all activities and all drivers.

Keywords:
Distraction and drowsiness detection
Computer vision
Perception and recognition
Machine learning
Monitoring and supervision
Texto completo
Referencias no citadas

Abtahi et al., 2014, Ahlstrom and Dukic, 2010, Ahonen et al., 2006, Asthana et al., 2011, Berri et al., 2014, Bolme et al., 2009, Boyraz et al., 2012, Chang and Lin, 2011, Dalal and Triggs, 2005, Daniluk et al., 2014, Dasgupta et al., 2013, Devi and Bajaj, 2008, Dinges and Grace, 1998, Dong et al., 2011, Fernandez et al., 2017, Fernández et al., 2015a, Fernández et al., 2015b, Fernández et al., 2015c, Fernández et al., 2016, Flores et al., 2010, Flores et al., 2011, Forsman et al., 2013, Hadid and Pietikäinen, 2013, Hammoud et al., 2005, Hansen and Ji, 2010, Hattori et al., 2006, Heikkilä et al., 2009, Hong and Qin, 2007, Hsu et al., 2003, Jain and Learned-Miller, 2010, Jo et al., 2014, Jung et al., 2016, Lee et al., 2011, Li et al., 2015, Liu et al., 2009, López Romero, 2016, Losada et al., 2013, Lu et al., 2011, Markuš et al., 2014, Martin, 2006, Mbouna et al., 2013, Murphy-Chutorian and Trivedi, 2010, Noori and Mikaeili, 2016, Nuevo et al., 2010, Ojala et al., 1996, Ojala et al., 2002, Organization, 2016, Pan et al., 2007, Peden et al., 2016, Phillips et al., 2000, RACE y l., 2016, Regan et al., 2011, Sahayadhas et al., 2012, Selvakumar et al., 2015, Shan, 2012, Shan et al., 2009, Sigari, 2009, Slawiñski et al., 2015, Song et al., 2013, Song et al., 2014, StopChatear, 2016, Talbot et al., 2013, Tan and Triggs, 2010, Timm and Barth, 2011, Uřičář et al., 2012, Vapnik, 1998, Vicente et al., 2015, Villan et al., 2016, Viola and Jones, 2004, Vural et al., 2007, You et al., 2013 and Zhang and Zhang, 2006.

Referencias
[Abtahi et al., 2014]
Abtahi, S., Omidyeganeh, M., Shirmohammadi, S., Hariri, B., 2014. Yawdd: a yawning detection dataset. In: Proceedings of the 5th ACM Multimedia Systems Conference. ACM, pp. 24-28.
[Ahlstrom and Dukic, 2010]
Ahlstrom, C., Dukic, T., 2010. Comparison of eye tracking systems with one and three cameras. In: Proceedings of the 7th International Conference on Methods and Techniques in Behavioral Research. ACM, p. 3.
[Ahonen et al., 2006]
T. Ahonen, A. Hadid, M. Pietikainen.
Face description with local binary patterns: Application to face recognition.
IEEE transactions on pattern analysis and machine intelligence, 28 (2006), pp. 2037-2041
[Asthana et al., 2011]
Asthana, A., Marks, T.K., Jones, M.J., Tieu, K.H., Rohith, M., 2011. Fully automatic pose-invariant face recognition via 3d pose normalization. In: 2011 International Conference on Computer Vision. IEEE, pp. 937-944.
[Berri et al., 2014]
Berri, R.A., Silva, A.G., Parpinelli, R.S., Girardi, E., Arthur, R., 2014. A pattern recognition system for detecting use of mobile phones while driving. In: Computer Vision Theory and Applications (VISAPP), 2014 International Conference on. Vol. 2. IEEE, pp. 411-418.
[Bolme et al., 2009]
Bolme, D.S., Draper, B.A., Beveridge, J.R., 2009. Average of synthetic exact filters. In: Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on. IEEE, pp. 2105-2112.
[Boyraz et al., 2012]
Boyraz, P., Yang, X., Hansen, J.H., 2012. Computer vision systems for context-aware active vehicle safety and driver assistance. In: Digital Signal Processing for In-Vehicle Systems and Safety. Springer, pp. 217-227.
[Chang and Lin, 2011]
C.-C. Chang, C.-J. Lin.
Libsvm: a library for support vector machines.
ACM Transactions on Intelligent Systems and Technology (TIST), 2 (2011), pp. 27
[Dalal and Triggs, 2005]
Dalal, N., Triggs, B., 2005. Histograms of oriented gradients for human detection. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05). Vol. 1. IEEE, pp. 886-893.
[Daniluk et al., 2014]
Daniluk, M., Rezaei, M., Nicolescu, R., Klette, R., 2014. Eye status based on eyelid detection: A driver assistance system. In: International Conference on Computer Vision and Graphics. Springer, pp. 171-178.
[Dasgupta et al., 2013]
A. Dasgupta, A. George, S. Happy, A. Routray, T. Shanker.
An onboard vision based system for drowsiness detection in automotive drivers.
International Journal of Advances in Engineering Sciences and Applied Mathematics, 5 (2013), pp. 94-103
[Devi and Bajaj, 2008]
Devi, M.S., Bajaj, P.R., 2008. Driver fatigue detection based on eye tracking. In: 2008 First International Conference on Emerging Trends in Engineering and Technology. IEEE, pp. 649-652.
[Dinges and Grace, 1998]
Dinges, D.F., Grace, R., 1998. Perclos: A valid psychophysiological measure of alertness as assessed by psychomotor vigilance. US Department of Transportation, Federal Highway Administration, Publication Number FHWA-MCRT-98-006.
[Dong et al., 2011]
Y. Dong, Z. Hu, K. Uchimura, N. Murayama.
Driver inattention monitoring system for intelligent vehicles: A review.
IEEE transactions on intelligent transportation systems, 12 (2011), pp. 596-614
[Fernandez et al., 2017]
A. Fernandez, J. Carus, R. Usamentiaga, E. Alvarez, R. Casado.
Wearable and ambient sensors to health monitoring using computer vision and signal processing techniques.
Journal of Networks, (2017),
In press
[Fernández et al., 2015a]
Fernández, A., Carús, J.L., Usamentiaga, R., Alvarez, E., Casado, R., 2015a. Unobtrusive health monitoring system using video-based physiological information and activity measurements. In: Computer, Information and Telecommunication Systems (CITS), 2015 International Conference on. IEEE, pp. 1-5.
[Fernández et al., 2015b]
Fernández, A., Casado, R., Usamentiaga, R., 2015b. A real-time big data architecture for glasses detection using computer vision techniques. In: Future Internet of Things and Cloud (FiCloud), 2015 3rd International Conference on. IEEE, pp. 591-596.
[Fernández et al., 2015c]
A. Fernández, R. García, R. Usamentiaga, R. Casado.
Glasses detection on real images based on robust alignment.
Machine Vision and Applications, 26 (2015), pp. 519-531
[Fernández et al., 2016]
A. Fernández, R. Usamentiaga, J.L. Carús, R. Casado.
Driver distraction using visual-based sensors and algorithms.
Sensors, 16 (2016), pp. 1805
[Flores et al., 2010]
M.J. Flores, J.M. Armingol, A. de la Escalera.
Real-time warning system for driver drowsiness detection using visual information.
Journal of Intelligent & Robotic Systems, 59 (2010), pp. 103-125
[Flores et al., 2011]
M.J. Flores, A. de la Escalera, et al.
Sistema avanzado de asistencia a la conducción para la detección de la somnolencia.
Revista Iberoamericana de Automática e Informática Industrial RIAI, 8 (2011), pp. 216-228
[Forsman et al., 2013]
P.M. Forsman, B.J. Vila, R.A. Short, C.G. Mott, H.P. Van Dongen.
Efficient driver drowsiness detection at moderate levels of drowsiness.
Accident Analysis & Prevention, 50 (2013), pp. 341-350
[Hadid and Pietikäinen, 2013]
A. Hadid, M. Pietikäinen.
Demographic classification from face videos using manifold learning.
Neurocomputing, 100 (2013), pp. 197-205
[Hammoud et al., 2005]
Hammoud, R.I., Wilhelm, A., Malawey, P., Witt, G.J., 2005. Eficient real-time algorithms for eye state and head pose tracking in advanced driver support systems. In: Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on. Vol. 2. IEEE, pp. 1181–vol.
[Hansen and Ji, 2010]
D.W. Hansen, Q. Ji.
In the eye of the beholder: A survey of models for eyes and gaze.
IEEE Transactions on pattern analysis and machine intelligence, 32 (2010), pp. 478-500
[Hattori et al., 2006]
Hattori, A., Tokoro, S., Miyashita, M., Tanaka, I., Ohue, K., Uozumi, S., 2006. Development of forward collision warning system using the driver behavioral information. Tech. rep., SAE Technical Paper.
[Heikkilä et al., 2009]
M. Heikkilä, M. Pietikäinen, C. Schmid.
Description of interest regions with local binary patterns.
Pattern recognition, 42 (2009), pp. 425-436
[Hong and Qin, 2007]
Hong, T., Qin, H., 2007. Drivers drowsiness detection in embedded system. In: Vehicular Electronics and Safety, 2007. ICVES. IEEE International Conference on. IEEE, pp. 1-5.
[Hsu et al., 2003]
Hsu, C.-W., Chang, C.-C., Lin, C.-J. et al., 2003. A practical guide to support vector classification.
[Jain and Learned-Miller, 2010]
Jain, V., Learned-Miller, E.G., 2010. Fddb: A benchmark for face detection in unconstrained settings. UMass Amherst Technical Report.
[Jo et al., 2014]
J. Jo, S.J. Lee, K.R. Park, I.-J. Kim, J. Kim.
Detecting driver drowsiness using feature-level fusion and user-specific classification.
Expert Systems with Applications, 41 (2014), pp. 1139-1152
[Jung et al., 2016]
J.-Y. Jung, S.-W. Kim, C.-H. Yoo, W.-J. Park, S.-J. Ko.
Lbp-ferns-based feature extraction for robust facial recognition.
IEEE Transactions on Consumer Electronics, 62 (2016), pp. 446-453
[Lee et al., 2011]
S.J. Lee, J. Jo, H.G. Jung, K.R. Park, J. Kim.
Real-time gaze estimator based on driver's head orientation for forward collision warning system.
IEEE Transactions on Intelligent Transportation Systems, 12 (2011), pp. 254-267
[Li et al., 2015]
Li, H., Lin, Z., Shen, X., Brandt, J., Hua, G., 2015. A convolutional neural network cascade for face detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 5325-5334.
[Liu et al., 2009]
C.C. Liu, S.G. Hosking, M.G. Lenné.
Predicting driver drowsiness using vehicle measures: Recent insights and future challenges.
Journal of safety research, 40 (2009), pp. 239-245
[López Romero, 2016]
López Romero, W.L., 2016. Sistema de control del estado de somnolencia en conductores de vehículos.
[Losada et al., 2013]
Losada, D.G., López, G.A. R., Acevedo, R.G., Villán, A.F., 2013. Aviueartificial vision to improve the user experience. In: New Concepts in Smart Cities: Fostering Public and Private Alliances (SmartMILE), 2013 International Conference on. IEEE, pp. 1-6.
[Lu et al., 2011]
Lu, L., Ning, X., Qian, M., Zhao, Y., 2011. Close eye detected based on synthesized gray projection. In: Advances in Multimedia, Software Engineering and Computing Vol. 2. Springer, pp. 345-351.
[Markuš et al., 2014]
Markuš, N., Frljak, M., Pandžić, I.S., Ahlberg, J., Forchheimer, R., 2014. Object detection with pixel intensity comparisons organized in decision trees. arXi**v preprint arXi***v:1305.4537.
[Martin, 2006]
E. Martin.
Breakthrough research on real-world driver behavior released.
National Highway Traffic Safety Administration, (2006),
[Mbouna et al., 2013]
R.O. Mbouna, S.G. Kong, M.-G. Chun.
Visual analysis of eye state and head pose for driver alertness monitoring.
IEEE transactions on intelligent transportation systems, 14 (2013), pp. 1462-1469
[Murphy-Chutorian and Trivedi, 2010]
E. Murphy-Chutorian, M.M. Trivedi.
Head pose estimation and augmented reality tracking: An integrated system and evaluation for monitoring driver awareness.
IEEE Transactions on intelligent transportation systems, 11 (2010), pp. 300-311
[Noori and Mikaeili, 2016]
S.M.R. Noori, M. Mikaeili.
Driving drowsiness detection using fusion of electroencephalography, electrooculography, and driving quality signals.
Journal of medical signals and sensors, 6 (2016), pp. 39
[Nuevo et al., 2010]
J. Nuevo, L.M. Bergasa, P. Jiménez.
Rsmat: Robust simultaneous modeling and tracking.
Pattern Recognition Letters, 31 (2010), pp. 2455-2463
of Transportation, D., 2016. Pennsylvania driver's manual. https://goo.gl/XCER8C, accessed: 2016-09-018
[Ojala et al., 1996]
T. Ojala, M. Pietikäinen, D. Harwood.
A comparative study of texture measures with classification based on featured distributions.
Pattern recognition, 29 (1996), pp. 51-59
[Ojala et al., 2002]
T. Ojala, M. Pietikainen, T. Maenpaa.
Multiresolution gray-scale and rotation invariant texture classification with local binary patterns.
IEEE Transactions on pattern analysis and machine intelligence, 24 (2002), pp. 971-987
[Organization, 2016]
Organization, W.H., 2016. Global status report on road safety 2015. http://goo.gl/jMoJ4l, accessed: 2016-07-01.
[Pan et al., 2007]
Pan, G., Sun, L., Wu, Z., Lao, S., 2007. Eyeblink-based anti-spoofing in face recognition from a generic webcamera. In: Computer Vision, 2007. ICCV 2007. IEEE 11th International Conference on. IEEE, pp. 1-8.
[Peden et al., 2016]
M. Peden, T. Toroyan, E. Krug, K. Iaych, et al.
The status of global road safety: The agenda for sustainable development encourages urgent action.
Journal of the Australasian College of Road Safety, 27 (2016), pp. 37
[Phillips et al., 2000]
P.J. Phillips, H. Moon, S.A. Rizvi, P.J. Rauss.
The feret evaluation methodology for face-recognition algorithms.
IEEE Transactions on pattern analysis and machine intelligence, 22 (2000), pp. 1090-1104
[RACE y l., 2016]
RACE, A. y. l. D., 2016. Los conductores españoles reconocen sufrir más somnolencia al volante que los usuarios europeos. http://goo.gl/mui9S3, accessed: 2016-07-01.
[Regan et al., 2011]
M.A. Regan, C. Hallett, C.P. Gordon.
Driver distraction and driver inattention: Definition, relationship and taxonomy.
Accident Analysis & Prevention, 43 (2011), pp. 1771-1781
[Sahayadhas et al., 2012]
A. Sahayadhas, K. Sundaraj, M. Murugappan.
Detecting driver drowsiness based on sensors: a review.
Sensors, 12 (2012), pp. 16937-16953
[Selvakumar et al., 2015]
K. Selvakumar, J. Jerome, K. Rajamani, N. Shankar.
Real-time vision based driver drowsiness detection using partial least squares analysis.
Journal of Signal Processing Systems, (2015), pp. 1-12
[Shan, 2012]
C. Shan.
Learning local binary patterns for gender classification on real-world face images.
Pattern Recognition Letters, 33 (2012), pp. 431-437
[Shan et al., 2009]
C. Shan, S. Gong, P.W. McOwan.
Facial expression recognition based on local binary patterns: A comprehensive study.
Image and Vision Compuing, 27 (2009), pp. 803-816
[Sigari, 2009]
Sigari, M.H., 2009. Driver hypo-vigilance detection based on eyelid behavior. In: Advances in Pattern Recognition, 2009. ICAPR’09. Seventh International Conference on. IEEE, pp. 426-429.
[Slawiñski et al., 2015]
E. Slawiñski, V. Mut, F. Penizzotto.
Sistema de alerta al conductor basado en realimentación vibro-táctil.
Revista Iberoamericana de Automática e Informática Industrial RIAI, 12 (2015), pp. 36-48
[Song et al., 2013]
F. Song, X. Tan, S. Chen, Z.-H. Zhou.
A literature survey on robust and efficient eye localization in real-life scenarios.
Pattern Recognition, 46 (2013), pp. 3157-3173
[Song et al., 2014]
F. Song, X. Tan, X. Liu, S. Chen.
Eyes closeness detection from still images with multi-scale histograms of principal oriented gradients.
Pattern Recognition, 47 (2014), pp. 2825-2838
[StopChatear, 2016]
StopChatear, 2016. Uso de los smartphones en la conducción. http://goo.gl/67dvtn, accessed: 2016-07-01.
[Talbot et al., 2013]
R. Talbot, H. Fagerlind, A. Morris.
Exploring inattention and distraction in the safetynet accident causation database.
Accident Analysis & Prevention, 60 (2013), pp. 445-455
[Tan and Triggs, 2010]
X. Tan, B. Triggs.
Enhanced local texture feature sets for face recognition under difficult lighting conditions.
IEEE transactions on image processing, 19 (2010), pp. 1635-1650
[Timm and Barth, 2011]
F. Timm, E. Barth.
Accurate eye centre localisation by means of gradients.
VISAPP, 11 (2011), pp. 125-130
[Uřičář et al., 2012]
M. Uřičář, V. Franc, V. Hlaváč.
Detector of facial landmarks learned by the structured output svm.
VIsAPP, 12 (2012), pp. 547-556
[Vapnik, 1998]
Vapnik, V., 1998. Statistical learning theory wiley new york google scholar.
[Vicente et al., 2015]
F. Vicente, Z. Huang, X. Xiong, F. De la Torre, W. Zhang, D. Levi.
Driver gaze tracking and eyes off the road detection system.
IEEE Transactions on Intelligent Transportation Systems, 16 (2015), pp. 2014-2027
[Villan et al., 2016]
A.F. Villan, J.L.C. Candas, R.U. Fernandez, R.C. Tejedor.
Face recognition and spoofing detection system adapted to visually-impaired people.
IEEE Latin America Transactions, 14 (2016), pp. 913-921
[Viola and Jones, 2004]
P. Viola, M.J. Jones.
Robust real-time face detection.
International journal of computer vision, 57 (2004), pp. 137-154
[Vural et al., 2007]
Vural, E., Cetin, M., Ercil, A., Littlewort, G., Bartlett, M., Movellan, J., 2007. Drowsy driver detection through facial movement analysis. In: International Workshop on Human-Computer Interaction. Springer, pp. 6-18.
[You et al., 2013]
You, C.-W., Lane, N.D., Chen, F., Wang, R., Chen, Z., Bao, T.J., Montes-de Oca, M., Cheng, Y., Lin, M., Torresani, L. et al., 2013. Carsafe app: alerting drowsy and distracted drivers using dual cameras on smartphones. In: Proceeding of the 11th annual international conference on Mobile systems, applications, and services. ACM, pp. 13-26.
[Zhang and Zhang, 2006]
Zhang, Z., Zhang, J.-s., 2006. Driver fatigue detection based intelligent vehicle control. In: 18th International Conference on Pattern Recognition (ICPR’06). Vol. 2. IEEE, pp. 1262-1265.
Opciones de artículo
Herramientas