According to the National Highway Traffic Safety Administration, every year about 100,000 police-reported crashes involve drowsy driving. These crashes result in more than 1,550 fatalities and 71,000 injuries. The real number may be much higher, however, as it is difficult to determine whether a driver was drowsy at the time of a crash. So, we tried to build a system, that detects whether a person is drowsy and alert him.
We need to create an enivronment in order to install dlib, as it cannot be directly installed using pip. So, follow this commands in order to install dlib into your system if you haven't installed it previously. Make sure you have Anaconda installed, as we will be doing everyting in Anaconda Prompt.
conda update conda
conda update anaconda
conda create -n env_dlib
conda activate env_dlib
conda install -c conda-forge dlib
If all these steps are completed successfully, then dlib will be installed in the virtual environment env_dlib. Make sure to use this environment to run the entire project.
conda deactivate
Clone the repository into your system by:
git clone https://github.com/Abhi10398/Drowsiness-Detection-System.git
Or directly download the zip.
Download the file shape_predictor_68_face_landmarks.dat here. Make sure you download it in the same folder.
Install all the system requirments by:
pip install -r requirements.txt
After the system has been setup. Run the command:
python app1.py
Open your browser and in the search bar type: localhost:5000 as this port is mostly used by flask. In case, this port is not available in your system, flask will try to use another port. The port number will be displayed in the command prompt. So, type in the same port number in that case as: localhost:<port_number>.
After all these steps have been completed successfully, you will see a web page opening up in the browser. Now you are free to explore the system.
The basic thing about drowsiness detection is pretty simple. We first detect a face using dlib's frontal face detector. Once the face is detected , we try to detect the facial landmarks in the face using the dlib's landmark predictor. The landmark predictor returns 68 (x, y) coordinates representing different regions of the face, namely - mouth, left eyebrow, right eyebrow, right eye, left eye, nose and jaw. Ofcourse, we don't need all the landmarks, here we need to extract only the eye and the mouth region.
Now, after extraxting the landmarks we calculate the Eye Aspect Ratio (EAR) as:
def eye_aspect_ratio(eye):
# Vertical eye landmarks
A = dist.euclidean(eye[1], eye[5])
B = dist.euclidean(eye[2], eye[4])
# Horizontal eye landmarks
C = dist.euclidean(eye[0], eye[3])
# The EAR Equation
EAR = (A + B) / (2.0 * C)
return EAR
The eye region is marked by 6 coordinates. These coordinates can be used to find whether the eye is open or closed if the value of EAR is checked with a certain threshold value.
In the same way I have calculated the aspect ratio for the mouth to detect if a person is yawning. Although, there is no specific metric for calculating this, so I have taken for points, 2 each from the upper and lower lip and calculated the mean distance between them as:
def mouth_aspect_ratio(mouth):
A = dist.euclidean(mouth[13], mouth[19])
B = dist.euclidean(mouth[14], mouth[18])
C = dist.euclidean(mouth[15], mouth[17])
MAR = (A + B + C) / 3.0
return MAR
Note: Learn more about dlib here.
We have used and Android App available for free in Play Store, named IP Webcam. It can be downloaded from this link. After downloading it, open the app and scroll down to the option Start Server. It will look like:
After starting the server, an IP will be displayed on the screen. Open the file android_cam.py. In line 36 put the given IP.
url = "http://<YOUR_IP_HERE>/shot.jpg"
Also, make sure that the phone and PC/Laptop is connected to the same internet connection.
Then, run the system in the same way as mentioned above. Click on the Run Using Phone Cam button to see the results:
Also, in order to toggle between the front and back camera, type the IP upto "http://<YOUR_IP_HERE>" in the search bar of yor browser and explore the page which will look like this:
We are looking to convert this into an Android App which will be more portable and easily accessible. Also, we faced some issues while hosting this on Heroku. Any leads on these will be extremely helpful.
[1]Facial landmarks with dlib, OpenCV and Python: https://www.pyimagesearch.com/2017/04/03/facial-landmarks-dlib-opencv-python/
[2]Eye blink detection with OpenCV, Python, and dlib: https://www.pyimagesearch.com/2017/04/24/eye-blink-detection-opencv-python-dlib/
[3]Drowsiness Detection with OpenCV: https://www.pyimagesearch.com/2017/05/08/drowsiness-detection-opencv/
[4]Real-Time Eye Blink Detection using Facial Landmarks: http://vision.fe.uni-lj.si/cvww2016/proceedings/papers/05.pdf