Convolutional Two-Stream Network Using Multi-Facial Feature Fusion for Driver Fatigue Detection

Weihuang Liu, Jinhao Qian, Zengwei Yao, Xintao Jiao and Jiahui Pan
Additional contact information
Weihuang Liu: School of Software, South China Normal University, Guangzhou 510641, China
Jinhao Qian: School of Software, South China Normal University, Guangzhou 510641, China
Zengwei Yao: School of Software, South China Normal University, Guangzhou 510641, China
Xintao Jiao: School of Software, South China Normal University, Guangzhou 510641, China
Jiahui Pan: School of Software, South China Normal University, Guangzhou 510641, China

Future Internet, 2019, vol. 11, issue 5, 1-13

Abstract: Road traffic accidents caused by fatigue driving are common causes of human casualties. In this paper, we present a driver fatigue detection algorithm using two-stream network models with multi-facial features. The algorithm consists of four parts: (1) Positioning mouth and eye with multi-task cascaded convolutional neural networks (MTCNNs). (2) Extracting the static features from a partial facial image. (3) Extracting the dynamic features from a partial facial optical flow. (4) Combining both static and dynamic features using a two-stream neural network to make the classification. The main contribution of this paper is the combination of a two-stream network and multi-facial features for driver fatigue detection. Two-stream networks can combine static and dynamic image information, while partial facial images as network inputs can focus on fatigue-related information, which brings better performance. Moreover, we applied gamma correction to enhance image contrast, which can help our method achieve better results, noted by an increased accuracy of 2% in night environments. Finally, an accuracy of 97.06% was achieved on the National Tsing Hua University Driver Drowsiness Detection (NTHU-DDD) dataset.

Keywords: fatigue detection; multi-task cascaded convolutional networks; optical flow; gamma correction; feature fusion (search for similar items in EconPapers)
JEL-codes: O3 (search for similar items in EconPapers)
Date: 2019
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/1999-5903/11/5/115/pdf (application/pdf)
https://www.mdpi.com/1999-5903/11/5/115/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jftint:v:11:y:2019:i:5:p:115-:d:230960

Access Statistics for this article

Future Internet is currently edited by Ms. Grace You

More articles in Future Internet from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().