Comparative Analysis of Facial Feature Extraction in RGB and Near-Infrared Images Using YOLOv11 for Edge-Deployed Driver Monitoring
DOI:
https://doi.org/10.61255/decoding.v4i2.1397Keywords:
Computer vision, Microsleep, Near-infrared, RGB, YOLOv11Abstract
Purpose – This study proposes a robust edge-computed Driver Monitoring System (DMS) using the YOLOv11 architecture to detect driver fatigue across daytime RGB and nighttime near-infrared (NIR) environments.
Methods – A lightweight YOLOv11 model was trained on an augmented multi-spectral dataset of 2,289 images containing two critical fatigue markers: drowsy_eye and open_mouth. For real-time deployment on a resource-constrained Raspberry Pi 4B, the model was compiled into an optimized ONNX format with a 240 × 320 pixel input matrix. A Temporal State Machine using strict logical conjunction (AND logic) was integrated to process sequential frame updates and reduce false-positive alerts caused by micro-blinking.
Findings – Under live multi-spectral stationary cabin hardware evaluation, the integrated prototype achieved real-time inference of 22.9–72.4 FPS in daytime RGB conditions and 20.7–28.7 FPS in nighttime NIR conditions. In total darkness, NIR feature extraction remained stable, with empirical confidence ranges of 0.70–0.82 for drowsy_eye and 0.93–0.94 for open_mouth. The state machine successfully confirmed microsleep events lasting more than two seconds and triggered synchronized voice alerts with a randomized LED array as a chaotic counter-fatigue sensory stimulus.
Research implications – The system demonstrates the feasibility of deploying advanced AI-based DMS models on low-power, standalone, cloudless edge hardware for automotive safety applications.
Originality – This study presents a multi-illumination RGB–NIR comparative evaluation of an ONNX-optimized YOLOv11 model integrated with an active randomized LED counter-fatigue intervention loop.
Abstract views: 3
, 
PDF downloads: 1
Downloads
References
C. Xu, C. Fu, and X. Jiang, “Advances in Vehicle Safety and Crash Avoidance Technologies,” Applied Sciences, vol. 15, no. 11, p. 5955, May 2025, doi: 10.3390/app15115955.
S. M. Saleem, “Risk assessment of road traffic accidents related to sleepiness during driving: a systematic review,” Eastern Mediterranean Health Journal, vol. 28, no. 9, pp. 695–700, 2022.
I. Nasri, M. Karrouchi, K. Kassmi, and A. Messaoudi, “A Review of Driver Drowsiness Detection Systems: Techniques, Advantages and Limitations,” High School of Technology, Mohammed First University, 2022.
J. Singh, R. Kanojia, R. Singh, R. Bansal, and S. Bansal, “Driver Drowsiness Detection System - An Approach By Machine Learning Application,” Journal of Pharmaceutical Negative Results, vol. 13, no. Special Issue 10, pp. 3002–3012, 2022, doi: 10.47750/pnr.2022.13.510.361.
F. Liu, D. Chen, J. Zhou, and F. Xu, “A review of driver fatigue detection and its advances on the use of RGB-D camera and deep learning,” Engineering Applications of Artificial Intelligence, vol. 116, p. 105399, 2022, doi: https://doi.org/10.1016/j.engappai.2022.105399.
Y. Albadawi, H. Takruri, and M. Awad, “A Review of Recent Developments in Driver Drowsiness Detection Systems,” Sensors, vol. 22, no. 5, 2022.
T. Fonseca and E. Al., “Drowsiness Detection in Drivers: A Systematic Review of Deep Learning Approaches,” Applied Sciences, vol. 15, no. 16, 2025.
R. M. Salman, M. Rashid, R. Roy, M. M. Ahsan, and Z. Siddique, “Driver Drowsiness Detection Using Ensemble Convolutional Neural Networks on YawDD,” 2021.
S. Fu and E. Al., “Advancements in the Intelligent Detection of Driver Fatigue and Distraction Based on Deep Learning,” Applied Sciences, vol. 14, no. 7, 2024.
O. F. Hassan and E. Al., “Real-Time Driver Drowsiness Detection Using Transformer Architectures and Transfer Learning,” Scientific Reports, vol. 15, 2025.
N. Lin and E. Al., “Advancing Driver Fatigue Detection in Diverse Lighting Conditions Using Deep Learning,” Scientific Reports, vol. 14, 2024.
N. Zrira and E. Al., “GCBAM-UNet: Sun Glare Segmentation Using Deep Learning,” Fire, vol. 7, no. 6, 2024.
S. Liawatimena and N. Isworo, “Annotated drowsiness detection dataset captured using Raspberry Pi 5,” Data in Brief, vol. 63, p. 112211, 2025, doi: 10.1016/j.dib.2025.112211.
R. Florez, F. Palomino-Quispe, A. B. Alvarez, R. J. Coaquira-Castillo, and J. C. Herrera-Levano, “A Real-Time Embedded System for Driver Drowsiness Detection Based on Visual Analysis of the Eyes and Mouth Using Convolutional Neural Network and Mouth Aspect Ratio,” Sensors, vol. 24, no. 19, p. 6261, 2024, doi: 10.3390/s24196261.
S. Raghavendran and E. Al., “Corneal Reflection Based Eye Tracking Technology,” in AIP Conference Proceedings, 2023. doi: doi.org/10.1063/5.0142351.
S. Abd El-Nabi, W. El-Shafai, E. S. M. El-Rabaie, K. Ramadan, and S. M. Fathi, “Machine learning and deep learning techniques for driver fatigue and drowsiness detection: a review,” Multimedia Tools and Applications, 2023, doi: 10.1007/s11042-023-15054-0.
S. Jadhav, S. Jagdale, N. Jangale, and S. Raut, “Driver Drowsiness Detection System Using Raspberry Pi,” International Journal for Research in Applied Science and Engineering Technology, vol. 10, no. 12, 2022, doi: doi.org/10.22214/ijraset.2022.47891.
A. A. D. Prasetyo and E. Al., “Smart Alarm Driver Assistance as an Early Warning of Driver Drowsiness Using Raspberry Pi 4 Model B,” Journal of Electrical Technology, 2025.
L. Zhang and Y. Liu, “Lightweight Object Detection Deployment via ONNX Quantization for Automotive Edge Computing,” Engineering Applications of Artificial Intelligence, vol. 132, p. 107954, 2024, doi: 10.1016/j.engappai.2024.107954.
J. Wang and E. Al., “Acceleration and Optimization of Deep Learning Inference on Edge Devices Using ONNX Runtime,” IEEE Access, vol. 11, pp. 45210–45222, 2023, doi: 10.1109/ACCESS.2023.3273410.
U. D. Maharani, A. S. Handayani, and L. Lindawati, “Analisis Deteksi Mata Kantuk di Wajah Pengemudi Menggunakan Support Vector Machine (SVM) Berbasis Citra Real-Time,” Building of Informatics, Technology and Science (BITS), vol. 6, no. 2, pp. 940–949, 2024, doi: 10.47065/bits.v6i2.5701.
I. I. Maulana et al., “Dampak Penggunaan Data Augmentasi Terhadap Akurasi MobileNetV2 Dalam Deteksi Mikrosleep Berbasis Rasio Aspek Mata,” Building of Informatics, Technology and Science (BITS), vol. 7, no. 3, pp. 1797–1808, 2025, doi: 10.47065/bits.v7i3.8719.
L. Anifah, N. Nurhayati, H. Haryanto, and M. S. Zuhrie, “Automatic Microsleep Detection Approach for Car Drivers Using YOLO5 Based on Image Feature,” TEM Journal, vol. 14, no. 3, pp. 1984–1991, Aug. 2025.
A. Karakan, “Real-Time and Deep Learning-Based Fatigue Detection for Drivers,” Mugla Journal of Science and Technology, vol. 10, no. 2, 2024, doi: 10.22531/muglajsci.1481648.
K. Kotwal et al., “Domain-Specific Adaptation of CNN for Detecting Face Presentation Attacks in NIR,” IEEE Transactions on Biometrics, Behavior, and Identity Science, vol. 4, no. 3, pp. 356–366, 2022, doi: 10.1109/TBIOM.2022.3168393.
M. A. Farooq, W. Shariff, D. O’Callaghan, A. Merla, and P. Corcoran, “On the Role of Thermal Imaging in Automotive Applications: A Critical Review,” IEEE Access, vol. 11, pp. 25147–25175, 2023, doi: 10.1109/ACCESS.2023.3255110.
S. N. Tien, T. L. E. Tien, and H. Ly-thanh, “ONNX-based Architectures for Post-Training Quantization Face Detection on Edge Devices,” pp. 1–11, 2026, doi: 10.15598/aeee.v24ix.251003.
O. K. Akinde, T. A. Olaleye, M. O. Ibitoye, V. Rizama, S. Taiwo, and M. O. Adetona, “An Intelligent-Based Algorithm for Determining Drowsy Drivers and Prevention of Road Accidents,” Uniosun Journal of Engineering and Environmental Sciences (UJEES), vol. 7, no. 2, 2025, doi: 10.64980/ujees.v7i2.452.
