A PROJECT PHASE-2 REPORT

ON

DRIVER DROWSINESS DETECTION SYSTEM

Submitted in partial fulfillment for the award of the degree of

BACHELOR OF ENGINEERING

IN

COMPUTER SCIENCE AND ENGINEERING

BY

AKASH S (1NH16CS003)
AKHIL RAMACHANDRAN (1NH16CS004)
NANDU S KUMAR(INH16CS068)

Under the guidance of

Dr. PAMELA VINITHA

Assistant Professor,
Dept. of CSE, NHCE

DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING

(Autonomous Institution Affiliated to VTU & Approved by AICTE)

Accredited by NAAC ‘A’, Accredited by NBA
Outer Ring Road,Panathur Post,Kadubisanahalli,
Bangalore - 560103
 DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING

CERTIFICATE

It is hereby certified that the project phase I work entitled “DRIVER DROWSINESS DETECTION SYSTEM” is a
bonafide work carried out by AKASH S (1NH16CS003),AKHIL RAMACHANDRAN (1NH16CS004), NANDU S
KUMAR (1NH16CS068) in partial fulfillment for the award of Bachelor of Engineering in COMPUTER SCIENCE
AND ENGINEERING of the New Horizon College of Engineering during the year 2019-2020. It is certified that all
corrections/suggestion indicated for Internal Assessment have been incorporated in the Report deposited in the
departmental library. The project report has been approved as it satisfies the academic requirements in respect
of project work prescribed for the said degree.

………………………..… ………………………..…

Signature of Guide Signature of HOD

(Dr. Pamela Vinitha) (Dr. B.Rajalakshmi)

External Viva

Name of Examiner Signature with date

1. ……………………………………….. ……………………………….

2. …………………………………………… ………………………………….
DRIVER DROWSINESS DETECTION SYSTEM
ORIGINALITY REPORT

10 %
SIMILARITY INDEX
%
INTERNET SOURCES
10%
PUBLICATIONS
%
STUDENT PAPERS

PRIMARY SOURCES

P. Karthika, R. Ganesh Babu, A. Nedumaran.

1
"Machine Learning Security Allocation in IoT", 1%
2019 International Conference on Intelligent
Computing and Control Systems (ICCS), 2019
Publication

Advances in Intelligent Systems and Computing,

2
2015. 1%
Publication

"Advances in Visual Informatics", Springer

3
Science and Business Media LLC, 2019 1%
Publication

"Emotion Detection using Facial Expressions

4
with Convolution Neural Networks", International 1%
Journal of Recent Technology and Engineering,
2019
Publication

Hisham El-Amir, Mahmoud Hamdy. "Deep

5
Learning Pipeline", Springer Science 1%
DRIVER DROWSINESS DETECTION SYSTEM

ABSTRACT
Nowadays, more and more professions require long-term concentration. Drivers must keep a close
eye on the road, so they can react to sudden events immediately. Driver fatigue often becomes a
direct cause of many traffic accidents. Therefore, there is a need to develop the systems that will
detect and notify a driver of her/him bad psychophysical condition, which could significantly reduce
the number of fatigue-related car accidents. However, the development of such systems
encounters many difficulties related to fast and proper recognition of a driver’s fatigue symptoms.
One of the technical possibilities to implement driver drowsiness detection systems is to use the
vision-based approach.The technical aspects of using the vision system to detect a driver
drowsiness are also discussed.Drowsiness and Fatigue of drivers are amongst the significant causes
of road accidents. Every year, they increase the amounts of deaths and fatalities injuries globally. In
this paper, a module for Advanced Driver Assistance System (ADAS) is presented to reduce the
number of accidents due to drivers fatigue and hence increase the transportation safety; this
system deals with automatic driver drowsiness detection based on visual information and Artificial
Intelligence. We propose an algorithm to locate, track, and analyze both the drivers face and eyes, a
scientifically supported measure of drowsiness associated with slow eye closure.

Dept.of CSE,NHCE I
DRIVER DROWSINESS DETECTION SYSTEM

ACKNOWLEDGEMENT

The satisfaction and euphoria that accompany the successful completion of any task would be
impossible without the mention of the people who made it possible, whose constant guidance and
encouragement crowned our efforts with success.

I have great pleasure in expressing my deep sense of grattitude to Dr.Mohan Manghnani, Chariman
of New Horizon Educational Institutions for providing necessary infrastructure and creating good
environment.

I take this opportunity to express my profound gratitude to Dr. Manjunatha, Principal NHCE, for his
consent support and encouragement.

I am grateful to Dr. Prashanth C.S.R, Dean Academics, for his unfailing encouragemnt and
suggestions, given to me in the course of my project work.

I would also like to thank Dr. B. Rajalakshmi, Professor and Head, Department of Computer Science
and Engineering, for her constant support.

I express my gratitude to Dr. Pamela Vinitha, Professor, my project guide, for constantly
monitoring the development of the project and setting up precise deadlines. His valuable
suggestions were the motivating factors in completing the work.

Finally, a note of thanks to the teaching and non teaching staff of Department of Computer Science
and Engineering, for their cooperation extended to me, and my friends, who helped me directly or
indirectly in the course of the project work.
AKASH S (1NH16CS003)
AKHIL RAMACHANDRAN (1NH16CS004)
NANDU S KUMAR (1NH16CS068)

Dept.of CSE,NHCE II
DRIVER DROWSINESS DETECTION SYSTEM

TABLE OF CONTENTS
CHAPTER NO TITLE PAGE NO
ABSTRACT I
ACKNOWLEDGEMENT II
LIST OF FIGURES V
1. INTRODUCTION 1-5
1.1 Machine Learning 1-2
1.2 Problem Statement 2
1.3 Project Purpose 3-4
1.4 Project Features 4
1.5 Modules Description 4
1.5.1 Data Collection 4
1.5.2 Applying Algorithms 5
1.5.3 Verification/Testing Phase 5
2. LITERATURE SURVEY 6-27
2.1 Data Mining 6
2.2 Existing System 7-8
2.3 Proposed System 9-12
2.3.1 Face an eye detection by OpenCv Algorithm 9-10
2.3.2 Face Detection 10-12
2.4 System Description 12-17
2.4.1 Eye Detection 12-14
2.4.2 Recognition of Eye’s State 14-15
2.4.3 Eye Aspected Ratio Calculation 15
2.4.4 Eye State Determination 15
2.4.5 Drowsiness Detection 16
2.4.6 Face Detection 17
2.5 Technology Used 18-20
2.5.1 Tensor Flow 18
2.5.2 Machine Learning 18

Dept.of CSE,NHCE III

DRIVER DROWSINESS DETECTION SYSTEM

2.5.3 Open CV 18
2.5.4 Tools and image processing libraries 19-20
2.6 Software Description 20
2.6.1 Python 20-21
2.6.2 Spyder/Anaconda 21-25
2.6.3 PANDAS 25-26
2.6.4 SKLEARN 26
2.6.5 HTML and CSS 26
2.6.6 SQL SERVER 27
3 REQUIREMENT ANALYSIS 28-29
3.1 Functional Requirement 28
3.2 Non-Functional Requirement 28-29
3.3 Hardware Requirement 29
3.4 Software Requirement 28
4 Design 30-38
4.1 System Architecture 32
4.1.1 Class Diagram 32-33
4.1.2 State Diagram 33-34
4.1.3 Sequence Diagram 35
4.1.4 Data Flow Diagram 36-37
4.2 Expected Outcome 38
5 TESTING 39-40
5.1 Importance of testing 39-40
5.2 Types of testing 40
5.2.1 Functional Testing 40-41
5.2.2 Non-Functional Testing 42
5.2.3 Regression Testing 42
5.2.4 Performance Testing 42
6 OUTPUT SCREENSHOT 43
7 CONCLUSION 44
8 REFERENCES 45

Dept.of CSE,NHCE IV
DRIVER DROWSINESS DETECTION SYSTEM

LIST OF FIGURES
FIGURE NO: FIGURE NAME PAGE NO:
2.1 Flowchart of Driver Drowsiness
Detection System 10
2.2 Rectangle Features 11
2.3 Cascade of Classifiers 12
2.4 Visualizing the 68 Facial
Landmark Coordinates 14
2.5 Anaconda Architecture 23
4.1 Class Diagram 32
4.2 State Diagram 33
4.3 Sequence Diagram 35
6.1 Open Eyes Detection 43
6.2 Closed Eyes Detection and
Alerting 43

Dept.of CSE,NHCE V
DRIVER DROWSINESS DETECTION SYSTEM

CHAPTER 1
INTRODUCTION
1.1 MACHINE LEARNING
Machine learning is Associate in drowsiness detection of computer science (AI) that gives
systems the flexibility to mechanically learn and improve from expertise while not being
expressively programmed. The most probable difference between Python and programming
is largely in programming we have a tendency to use conditional statements to expressively
tell the program to figure victimization those conditions whereas, Python mechanically
learns and improves from expertise primarily the user ought to train the dataset and also
the machine language learns mechanically by means independently. Python focuses on the
event of pc programs which will access knowledge and use it learn for themselves. The
process of learning begins with observations or knowledge, like examples, direct expertise,
or instruction, so as to seem for patterns in knowledge and create higher choices in the
future supported the examples that we offer. This type of process of learning is
predominantly classified as supervised learning. The main objective of this type of learning is
that the system already knows its output it’s just the system structures itself towards the
output in this type of learning also shows that the system learns even through feedback too.
The essential point is to permit the PCs adapt naturally without human intercession or help
and change activities likewise. Python algorithms are classified as:
1. Supervised ML Algorithms these algorithms can utilize what has been realized in the past to the
new information being utilized by utilizing hailed models which will be useful in future scopes.
Beginning with preparing the dataset while preparing the dataset the AI calculation sets up
hindrances like fundamental capacities to make expectation of the yield esteems for future
scopes by setting up these impediments the calculation consequently. takes in and develops
itself from these obstructions. This sort of Learning needs an adequate measure of preparing
to anticipate yields and to give yields to the particular new sources of info. This sort of
learning calculation contrasts it's yield and the ideal yield and redresses itself and learns
without anyone else by discover blunders and changing them.

2. Unsupervised Python Algorithms these algorithms square measure utterly completely different
from supervised Python algorithms these algorithms square measure used once the dataset

Dept.of CSE,NHCE 1
DRIVER DROWSINESS DETECTION SYSTEM

to coach is neither classified nor a flagged. So, the dataset ought to be cleansed before
coaching it. knowledge clean-up is largely classifying and labelling the dataset. during this sort
the system doesn't turn out the proper output however investigation {the knowledge info the
information}set and attracts conclusion on the idea of the data clean up done before coaching
the dataset and classifying and tired examples and describe the hidden structures from the
dataset. unsupervised learning is that the coaching of machine mistreatment info that's
neither classified nor labelled and permitting the rule to act on it info while not steerage.
Here the task of machine is to cluster unsorted info per similarities, patterns and variations
with none previous coaching of knowledge. the most distinction between supervised and
unsupervised algorithms is that supervised learning is well-versed steerage whereas
unsupervised learning isn't.

3. Semi-administered Learning it falls among regulated and solo learning this type of learning uses
both labelled and unlabelled data. Typically larger proportionate of un labelled data. Semi
supervised learning requires both the types of data as when the labelled data requires more
resources which are totally dependent on unlabelled data.

4. Reinforcement Learning is a type of dynamic learning this trains the algorithm using a system of
reward and punishment it interacts with its environment to learn. This is inspired behavioral
psychology it is similar to supervised learning but as in supervised learning it is explicitly told
how to perform a task and what the desired outcome will be. As, in this type of learning it
works through the problem on its own and finds a way to perform the task and get the
desired.

1.2 PROBLEM STATEMENT

On road driver’s fatigue and drowsiness is contributing more than 30%[1] of reported road
accidents. Driver drowsiness can be estimated by monitoring biomedical signals ,visual
assessment of driver’s bio-behavior from face images , by Rmonitoring drivers performance
or by combines all above techniques. Proposed algorithm is based on live monitoring of EAR
(Eye aspect Ratio) by application of Image processing. HD live video is decomposed in
continues frames and facial landmarks has been detected using pre trained Neural Network
based Dlib functions. Dlib functions are trained using Cascading algorithm. Intel’s Open

Dept.of CSE,NHCE 2
DRIVER DROWSINESS DETECTION SYSTEM

source Image processing libraries (OPEN CV) is used as primary Image processing tool.
Python Language is used as main codding language.

1.3 PROJECT PURPOSE

Real Time Drowsiness behaviors which are related to fatigue are in the form of eye closing,
head nodding or the brain activity. Hence, we can either measure change in physiological
signals, such as brain waves, heart rate and blinking to monitor drowsiness or consider
physical changes such as sagging leaning of driver’s head and open/closed state of eyes.
The former technique,while more accurate, is not realistic since highly sensitive electrodes
would have to be attached directly on the driver’ body and hence which can be annoying
and distracting to the driver. In addition, long time working would result in perspiration on
the sensors, diminishing their ability to monitor accurately. The second technique is to
measure physical changes (i.e. open/closed eyes to detect fatigue) is well suited for real
world conditions since it is non-intrusive by using a video camera to detect changes. In
addition, micro sleeps that are short period of sleeps lasting 2 to 3 minutes are good
indicators of fatigue. Thus, by continuously monitoring the eyes of the driver one can
detect the sleepy state of driver and a timely warning is issued.
Driver in-alertness is an important resulting from sleep deprivation or sleep disorders and
is an important factor in the increasing number of the accidents on today’s roads. Drowsy
driver warning system can form the basis of the system to possibly reduce the accidents
related to driver’s drowsiness. In a year there were 824 fatalities recorded in NHTSA’s
FARS database that were drowsydriving-related. According to the results of the study
presented at the International Symposium on Sleep Disorders, fatigue of drivers is
responsible for30% of road accidents[1] Different techniques are used in driver-fatigue
monitoring systems. These techniques are divided into four categories. The first
categorys includes intrusive techniques, which are mostly based on monitoring
biomedical signals, and therefore require physical contact with the driver. The second
category includes non-intrusive techniques based on visual assessment of driver’s
bio-behavior from face images. The third category includes methods basedon driver’s
performance, which monitor vehicle behavior such as moving course, steering angle, speed,
braking,etc.Finally,thefourth category combines techniques from the

Dept.of CSE,NHCE 3
DRIVER DROWSINESS DETECTION SYSTEM

abovementionedthree categories.The computer vision basedtechniques from the second

category are particularly effective, because the drowsiness can be detected by
observing the facial features and visual bio-behavior such as head position, gaze, eye
openness, eyelid movements, and mouth openness. Proposed algorithm is based on
computer vision method. The main focus is on the detection of blinks by estimating the
EAR(Eye aspect Ratio). This is achieved by monitoring the eyes of the driver throughout
the entire video sequence. An IR camera will be used for capturing live video of driver
eyes in all light conditions and frames will extracted for image processing scheme of
video capturing .

1.4 PROJECT FEATURES

Driver fatigue their time requirement is a fraction of the previously used methods. The
algorithm starts with the detection of heads on colour pictures using deviations in colour
and structure of the human face and that of the background. By normalizing the distance
and position of the reference points, all faces should be transformed into the same size and
position. For normalization, eyes serve as points of reference. Other OpenCV algorithm
finds the eyes on any grayscale image by searching characteristic features of the eyes and
eye sockets. Tests made on a standard database show that the algorithm works very fast
and it is reliable.Here we will work with face detection. Initially, the algorithm needs a lot
of positive images (images of faces) and negative images (images without faces) to
train the classifier. Then we need to extract features from it. For this, OpenCV features
shown in the below image are used. They are just like our convolutional kernel.

1.5 MODULES DESCRIPTION

1.5.1 DATA COLLECTION

Data plays the most important role in this project. The data has to be genuine and
trustworthy. Data has to be collected from the right sources and each record must have all
the necessary fields to perform the analysis. It can either be procured from online sources
or collected manually to attain more accuracy, but is a very tedious task. Once data is
collected, it needs to be verified for relevance before finalizing as valid data for the project.
The accuracy of the outcome is directly proportional to the size of the dataset. i.e Bigger

Dept.of CSE,NHCE 4
DRIVER DROWSINESS DETECTION SYSTEM

the size of the dataset, more is the accuracy. This is because the machine learning
algorithms need a lot of data to work with, to identify patterns and learn on itsown.

1.5.2 APPLYING ALGORITHMS

This module has two parts to it- identifying the algorithms and applying the algorithms.
Identification plays a key role in this process and in the project as a whole. Identifying the
right algorithm may be done through research and mostly trial and error. We pick the
algorithms which yield the most accurate results and zero in on them. These algorithms are
then applied to the data set using Spyder IDE. The identification, application and
verification phase of the system go hand in hand, all three steps need to be executed for each
algorithm to decide on which one is the most efficient.

1.5.3 VERIFICATION/TESTING PHASE

The former technique, while more accurate, is not realistic since highly sensitive electrodes
would have to be attached directly on the driver’ body and hence which can be annoying
and distracting to the driver. In addition, long time working would result in perspiration on
the sensors, diminishing their ability to monitor accurately. The second technique is to
measure physical changes (i.e. open/closed eyes to detect fatigue) is well suited for real
world conditions since it is non-intrusive by using a video camera to detect changes. In
addition, micro sleeps that are short period of sleeps lasting 2 to 3 minutes are good
indicators of fatigue. Thus, by continuously monitoring the eyes of the driver one can
detect the sleepy state of driver and a timely warning is issued.

Dept.of CSE,NHCE 5
DRIVER DROWSINESS DETECTION SYSTEM

CHAPTER 2
Literature survey
2.1 Data mining
Data mining is the process of sorting the large data sets to establish relationships and
identification of patterns to solve real world problems through data analysis. Companies use
data mining to turn the raw data into the information which can be useful for them. It
mainly depends on computer processing, warehousing, and collection of effective data.
Data mining is used in various different ways such as fraud detection, identify spam email,
and database marketing.
The process of data mining is broken down into five steps. First it collects the data loads
them into their own data warehouses. Second, they manage the data and store them either
on the cloud or on in-house severs. Then, software applications sorts the data based on the
user requirements and then, finally the user on the end displays the data in an easy manner
like table or graph.
Various researches on this process has led to formation of different data mining algorithms.
We can use these algorithms directly on a dataset to create some models and get the
inferences from the dataset. Popular data mining algorithms are Nearest Neighbor (KNN),
Naive Bayes, Random forest, Logistic regression, Decision tree, and Support vector
machines.
1. Nearest neighbor (KNN) is a managed machine calculation which can be utilized to
unravel a wide range of relapse and arrangement issues.
2. Naive Bayes calculation is a group of probabilistic calculations that exploits the
likelihood hypothesis and Bayes hypothesis to anticipate the tag of a book.
3. Logistic relapse is a characterization calculation used to allot perception to set of
discrete classes. A portion of the characterization issues are email spam, online
misrepresentation identification, tumor harmful or kind. Calculated relapse
essentially changes its yield utilizing the strategic sigmoid capacity to restore
likelihood esteem.

Dept.of CSE,NHCE 6
DRIVER DROWSINESS DETECTION SYSTEM

4. Decision tree calculation goes under the class of directed learning. This calculation
can be utilized to explain both the relapse and grouping issues. The choice tree
utilizes tree portrayal to take care of the issues.
5. A support vector machine is an administered AI calculation which can be utilized for
both relapse and grouping issues. In help vector machine, we plot every datum thing
as a point in N-dimensional space (where n is the quantity of highlights) with the
estimation of each component being a similar estimation of specific organizations.
6. Random timberland calculation is a technique that works by building numerous
choice trees during preparing stage. The choice of greater part of the trees is picked
by the arbitrary backwoods as an official conclusion. It utilizes include haphazardness
and stowing when assembling an individual tree to attempt to make a woods of
trees whose forecast is more exact and precise than that of any individual tree.

2.2 Existing System

Images or the real time video is captured from the camera installed in front of the driver's
face. This video is converted into number of frames. OpenCV face OpenCV-classifier is loaded.
Each frame is compared with the pre-defined features of the OpenCV-classifiers.when the
features are matched the face is detected and a rectangle is drawn around the face. Using
feature extraction we estimate the position of the eyes. By comparing with the OpenCV
eye-OpenCV classifier, the eyes are detected and rectangles are drawn around left and right
eye.
Manu B.N in 2016, has proposed a method that detect the face using OpenCV
feature-based cascade classifiers. Initially, the algorithm needs a lot of positive images (images
of faces) and negative images (images without faces) to train the classifier that will detect the
object. So along using the fea with the OpenCV feature-based classifiers, cascaded Adaboost
classifier is exploited to recognize the face region then the compensated image is segmented
into numbers of rectangle areas, at any position and scale within the original image. Due to
the difference of facial feature, OpenCVlike feature is efficient for real-time face detection.
These can be calculated according to the difference of sum of pixel values within rectangle
area and during the process the Adaboost algorithm will allow all the face samples and
it will discard the non-face samples of images.

Dept.of CSE,NHCE 7
DRIVER DROWSINESS DETECTION SYSTEM

Amna Rahman in 2015, has proposed a method to detect the drowsiness by using Eye state
detection with Eye blinking strategy. In this method first, the image is converted to gray
scale and the corners are detected using Harris corner detection algorithm which will
detect the corner at both side and at down curve of eye lid. After tracing the points then it
will make a straight line between the upper two points and locates the mid-point by
calculation of the line, and it connects the mid-point with the lower point. Now for each
image it will perform the same procedure and it calculates the distance ‘d’ from the
mid-point to the lower point to determine the eye state. Finally, the decision for the eye
state is made based on distance ’d’ calculated. If the distance is zero or is close to zero, the
eye state is classified as “closed” otherwise the eye state is identified as “open”. They have
also invoked intervals or time to know that the person is feeling drowsy or not. This is done
by the average blink duration of a person is 100-400 milliseconds (i.e. 0.1-0.4 of a second).
In-vehicle camera is commonly installed to realize the possible reasons of car accidents.
Such a camera can also be used to detect the fatigue of the driver. Several studies related
to the fatigue detection are described as follows. Sharma et al. [3] utilized the number of
pixels in the eye image to determine the eye state, open or close. Hornget at. [4]
established an edge map to locate the eyes locations and the eye state is determined based
on the HSL color space of the eye image. Its accuracy is dependent on the location of the
eyes. Sharma and Banga [3] converted the face image to YCbCr color space. The average
and standard deviation of the pixel number in thebinarization image is computed. Then,
fuzzy rules [5] are used to determine the eye state. Liu et al. [6] and Tabrizi et al.

Dept.of CSE,NHCE 8
DRIVER DROWSINESS DETECTION SYSTEM

2.3 PROPOSED SYSTEMS

2.3.1 Face and Eye Detection by OpenCV Algorithms

In this paper a novel approach to critical parts of face detection problems is given, based on
analogic cellular neural network (OpenCV) algorithms. The proposed OpenCValgorithms find
and help to normalize human faces is, effectively while cause for most accident related to the
vehicles crashes. Driver fatigue their time requirement is a fraction of the previously used
methods.The algorithm starts with the detection of heads on colour pictures using deviations in
colour and structure of the human face and that of the background.By normalizing the distance
and position of the reference points, all faces should be transformed into the same size and
position.For normalization, eyes serve as point reference. Other OpenCV algorithm finds the
eyes on any grayscale image by searching characteristic is features of the eyes and eye sockets.
Tests made on a standard database show that the algorithm works very fast and it is reliable.In
proposed method, first the image is acquired by the webcam for processing. The images of the
driver are captured from the camera which is installed in front of the driver on the car
dashboard. It will be passed to preprocessing which prepares the image for further processing
by the system. Its main operations are to eliminate noises caused by the image acquisition
subsystem and image enhancement using Histogram Equalization. Then we search and detect
the faces in each individual frame. If no face is detected then another frame is acquired. If a
face is detected, then a region of interest in marked within the face. This region of interest
contains the eyes. Defining a region of interest significantly reduces the computational
requirements of the system. After that the eyes are detected from the region of interest. If an
eye is detected then there is no blink and the blink counter is set to „20‟. If the eyes are closed
in a particular frame, then the blink counter is decremented and a blink is detected. When the
eyes are closed for more than 4 frames then it is deducible that the driver is feeling drowsy.
Hence drowsiness is detected and an alarm sounded. After that the whole process is repeated
as long as the driver is driving the car. The overall flowchart for drowsiness detection system is
shown in Figure 2.1

Dept.of CSE,NHCE 9
DRIVER DROWSINESS DETECTION SYSTEM

Figure 2.1. Flowchart of Driver Drowsiness Detection System

2.3.2 Face Detection

Face detection is accomplished by the OpenCV algorithm proposed by Paul Viola and
Michael Jones in 2001. Due to the complex background, it is not a good choice to locate or
detect both the eyes in the original image, for this we will take much more time on
searching the whole window with poor results. So firstly, we will locate the face, and
reduce the range in which we will detect both the eyes. After doing this we can improve
the tracking speed and correct rate, reduce the affect of the complex background. Besides,
we propose a very simple but powerful method to reduce the computing complexity.

Dept.of CSE,NHCE 10
DRIVER DROWSINESS DETECTION SYSTEM

i) OpenCV-like features: The simple features used are suggestive approaches of OpenCV
basis functions which have been used by Papageorgiouetal.A OpenCV-like feature considers
affixed rectangular regions at a specific part in a detection window; each OpenCV like
feature expressed by two or three jointed black and white rectangles shown in figure 2. The
value of a OpenCV like feature is the difference between the sums of the pixel values within
the black and white rectangular regions. These sums are used to find the difference
between regions. Then the differences can be used to classify the sub region of an image.
These differences are compared against learned threshold values to determine whether or
not the object appears in the region.

Figure 2.2: Rectangle features example: (A) and (B) shows two-rectangle features,
(C) Shows three-rectangle feature, and (D) shows four-rectangle feature

ii) Integral image The simple rectangular features of an image are calculated using an
intermediate representation of an image, called the integral image as in (1) .The integral
images are an array which consists of sums of the pixels‟ intensity values located directly
to the left of a pixel and directly above the pixel at location (x,y) inclusive. Here, A[x,y] is
the original image and Ai[x,y] is the integral image. Ai[x, y] = Σ A[ x', y'] x'≤ x, y'≤ y (1) iii)
AdaBoost Adaboost, nothing but "Adaptive Boosting ", It can be used with many other
types of learning algorithms to improve their performance. Adaboost takes a number of
positive and negative images features and training sets, The machine creates a set of weak
classifiers of OpenCV-like features. It selects a set of weak classifiers to combine and that
assigns lesser weights to good features whereas larger weights to poor features. This
weighted combination gives strong classifier.

iii) Cascaded classifier

Dept.of CSE,NHCE 11
DRIVER DROWSINESS DETECTION SYSTEM

The cascade classifier consists of number of stages, where each stage is a collection of weak
learners. The weak learners are simple classifiers known as decision stumps. Boosting is used to
train the classifiers. It provides the ability to train a highly accurate classifier by taking a weighted
average of the decisions made by the weak learners.

Figure 2.3: Cascade of Classifiers

Each stage of the classifier shows the region defined by the current location of the sliding
window as either positive or negative. Positive indicates an object was found and negative
indicates no object. If the label is negative, the classification of this region is complete, and
the detector shifts the window to the next location. If the label is positive, the classifier
passes the region to the next stage. The detector reports an object found at the current
window location when the final stage classifies the region as positive. It is used to eliminate
less likely regions quickly so that no more processing is needed. Hence, the speed of overall
algorithm is increased.

2.4 System Description

2.4.1 Eye Detection
Images or the real time video is captured from the camera installed in front of the driver's
face. This video is converted into number of frames. OpenCV face OpenCV-classifier is loaded.
Each frame is compared with the pre-defined features of the OpenCV-classifiers. When the
features are matched the face is detected and a rectangle is drawn around the face. Using
feature extraction we estimate the position of the eyes. By comparing with the OpenCV
eye-OpenCV classifier, the eyes are detected and rectangles are drawn around left and right

Dept.of CSE,NHCE 12
DRIVER DROWSINESS DETECTION SYSTEM

eye. In the system we have used facial landmark prediction for eye detection Facial
landmarks are used to localize and represent salient regions of the face, such as:
• Eyes

• Eyebrows

• Nose

• Mouth

• Jawline

Facial landmarks have been successfully applied to face alignment, head pose estimation,
face swapping, blink detection and much more. In the context of facial landmarks, our goal is
detecting important facial structures on the face using shape prediction methods. Detecting
facial landmarks is therefore a twostep process:
• Localize the face in the image.
• Detect the key facial structures on the face ROI.
Localize the face in the image: The face image is localized by Haar feature-based cascade
classifiers which was discussed in the first step of our algorithm i.e. face detection. Detect the
key facial structures on the face ROI: There are a variety of facial landmark detectors, but all
methods essentially try to localize and label the following facial regions:
• Mouth

• Right eyebrow

• Left eyebrow

• Right eye

• Left eye

• Nose7

The facial landmark detector included in the dlib library is an implementation of the One
Millisecond Face Alignment with an Ensemble of Regression Trees paper by Kazemi and
Sullivn.
This method starts by using:
1. A training set of labeled facial landmarks on an image. These images are manually
labeled, specifying specific (x, y)-coordinates of regions surrounding each facial structure.
2. Priors, of more specifically, the probability on distance between pairs of input pixels.
The pre-trained facial landmark detector inside the dlib library is used to estimate the
location of 68 (x, y)-coordinates that map to facial structures on the face.

Dept.of CSE,NHCE 13
DRIVER DROWSINESS DETECTION SYSTEM

The indexes of the 68 coordinates can be visualized on the image below:

Fig.2.4: Visualizing the 68 facial landmark coordinates

We can detect and access both the eye region by the following facial landmark index show
below
• The right eye using [36, 42].
• The left eye with [42, 48].
These annotations are part of the 68 point iBUG 300-W dataset which the dlibfacial
landmark predictor was trained on. It’s important to note that other flavors offacial landmark
detectors exist, including the 194 point model that can be trained on the HELEN dataset.
Regardless of which in dataset is used, the same dlib framework can be leveraged to train a
shape predictor on the input training data.

2.4.2 Recognition of Eye's State:

The eye area can be estimated from optical flow, by sparse tracking or by frame-to-frame
intensity differencing and adaptive thresholding. And Finally, a decision is made whether
the eyes are or are not covered by eyelids. A different approach is to infer the state of the
eye opening from a single image, as e.g. by correlation matching with open and closed eye
templates, a heuristic horizontal or vertical image intensity projection over the eye region,
a parametric model fitting to find the eyelids, or active shape models. A major drawback of
the previous approaches is that they usually implicitly impose too strong requirements on
the setup, in the sense of a relative face-camera pose (head orientation), image resolution,

Dept.of CSE,NHCE 14
DRIVER DROWSINESS DETECTION SYSTEM

illumination, motion dynamics, etc. Especially the heuristic methods that use raw image
intensity are likely to be very sensitive despite their real-time performance.
Therefore, we propose a simple but efficient algorithm to detect eye blinks by using a
recent facial landmark detector. A single scalar quantity that reflects a level of the eye
opening is derived from the landmarks. Finally, having a per-frame sequence of the
eye-opening estimates, the eye blinks are found by an SVM classifier that is trained on
examples of blinking and nonblinking patterns.

2.4.3 Eye Aspected Ratio Calculation:

For every video frame, the eye landmarks are detected. The eye aspect ratio (EAR) between
height and width of the eye is computed.
EAR = ||p2 − p6|| + ||p3 − p5|| (1)
2||p1 − p4||
where p1, . . ., p6 are the 2D landmark locations, depicted in Fig. 1. The EAR is mostly
constant when an eye is open and is getting close to zero while closing an eye. It is partially
person and head pose insensitive. Aspect ratio of the open eye has a small variance among
individuals, and it is fully invariant to a uniform scaling of the image and in-plane rotation of
the face. Since eye blinking is performed by both eyes synchronously, the EAR of both eyes
is averaged.

2.4.4 Eye State Determination:

Finally, the decision for the eye state is made based on EAR calculated in the previous step.
If the distance is zero or is close to zero, the eye state is classified as “closed” otherwise the
eye state is identified as “open”. The eye area can be estimated from optical flow, by
sparse tracking or by frame-to-frame intensity differencing and adaptive thresholding. And
Finally, a decision is made whether the eyes are or are not covered by eyelids. A different
approach is to infer the state of the eye opening from a single image, as e.g. by correlation
matching with open and closed eye templates.

2.4.5 Drowsiness Detection:

The last step of the algorithm is to determine the person’s condition based on a pre-set
condition for drowsiness. The average blink duration of a person is 100-400 milliseconds

Dept.of CSE,NHCE 15
DRIVER DROWSINESS DETECTION SYSTEM

(i.e. 0.1-0.4 of a second). Hence if a person is drowsy his eye closure must be beyond this
interval. We set a time frame of 5 seconds. If the eyes remain closed for five or more
seconds, drowsiness is detected and alert pop regarding this is triggered.

 Template Matching
Template matching is basically the two-dimensional cross correlation of a grayscale image
with a grayscale template; hence it can be used to estimate the degree of similarity between the
two images. Template matching is sensitive to variation of poses. Template matching is necessary
for the desired accuracy in analyzing the user‟s blinking since it allows the user some freedom to
move around slightly. Template matching is a technique in digital image processing for finding small
parts of an image which match a template image. The normalized correlation coefficient is used to
accomplish the tracking

 Physiological level approach:

This technique is an intrusive method where electrodes are used to obtain pulse rate,
heart rate and brain activity with information. ECG is used to calculate the variations in
heart rate and detect different conditions for drowsiness.The correlation between different
signal such as ecg (electrocardiogram), EEG(electroencephalogram), VPES(validps), and aslo an
EMG(electromyogram) are made and then the output is generated
whether the person is drowsy or not.

 Behavioral based approach:

In this technique eye blinking frequency, head pose,etc.of a person is monitored through a
camera and the person is also any of these drowsiness symptoms are detected.

2.4.6 Face Detection

For the face Detection it uses OpenCV feature-based cascade classifiers is an effective
object detection method proposed by Paul Viola and Michael Jones in their paper, "Rapid
Object Detection using a Boosted Cascade of Simple Features" in 2001. It is a machine
learning based approach where a cascade function is trained from a lot of positive and

Dept.of CSE,NHCE 16
DRIVER DROWSINESS DETECTION SYSTEM

negative images. It is then used to detect objects in other images. Here we will work with
face detection. Initially, the algorithm needs a lot of positive images (images of faces) and
negative images (images without faces) to train the classifier. Then we need to extract
features from it. For this, OpenCV features shown in the below image are used. They are
just like our convolutional kernel. Each feature is a single value obtained by subtracting sum
of pixels under the white rectangle from sum of pixels under the black rectangle. Fig. 3.2
represents five OpenCV like features shown in Fig 3.3.

Fig 3.2 Fig 3.3

2.5 Technology Used

2.5.1 TensorFlow: IT is an open-source software library for dataflow programming across a range
of tasks. It is a symbolic math library, and is also used for the machine learning applications such
as neural networks. It is used for both research and production.TensorFlow computations are
expressed as tensor.

2.5.2 Machine learning: Machine learning is the kind of programming which gives computers
the capability to automatically learn from data without being explicitly programmed. This means in
other words that these programs change their behavior by learning from data. Python is clearly one
of the best languages for machine learning. Python does contain special libraries for machine
learning namely scipy, pandas and numpy which great for linear algebra and getting to know kernel
methods of machine learning. The language is great to use when working with machine learning
algorithms and has easy syntax relatively.

Dept.of CSE,NHCE 17
DRIVER DROWSINESS DETECTION SYSTEM

2.5.3 OpenCV: OpenCV stands for Open Source Computer Vision. It's an Open Source BSD
licensed library that includes hundreds of advanced Computer Vision algorithms that are
optimized to use hardware acceleration. OpenCV is commonly used for machine learning,
4 image processing, image proc. manipulation, and much more. OpenCV has a modular
structure. There are shared and static libraries and a CV Namespace. In short, OpenCV is
used in our application to easily load bitmap files that contain landscaping pictures and
perform a blend operation between two pictures so that one picture can be seen in the
background of another picture. This image manipulation is easily performed in a few lines
of code using OpenCV versus other meths.

2.5.4 TOOLS AND IMAGE PROCESSING LIBRARIES

Following optimized tools and image processing libraries are used by author for
implementation of presented algorithm. Open CV: OpenCV (Open-source Computer Vision) is the
Swiss Army knife of computer vision.
It has a wide range of modules that can help us with a lot of computer vision problems. But perhaps
the most useful part of OpenCV is its architecture and memory management. It provides you with
a framework in which you can work with images and video in any way you want, using OpenCV’s
algorithms or your own, without worrying about allocating and reallocating memory for your
images. Open CV libraries and functions are highly optimized and can be used for real time
image and video processing. OPENCV’s highly optimized image processing function are used by
author for real time image processing of live video feed from camera.

DLib: Dlib is a modern C++ toolkit containing machine learning algorithms and tools for
creating complex software in C++ to solve real world problems. It is used in both industry and
academia in a wide range of domains including robotics, embedded devices, mobile phones, and
large high performance computing environments. Dlib's open source licensing allows you to use it
in any application, free of charge. Open Source Dib library is used by author for implementation
of CNN(Neural Networks). Highly optimized Pre-learned facial shape predictor and detectors
functions are used by author for detectionof facial landmarks. Facial landmarks were further
used for extracting eye coordinates.

Dept.of CSE,NHCE 18
DRIVER DROWSINESS DETECTION SYSTEM

Python: Python is an object-oriented programming language created by Guido Rossum in 1989. It

is ideally designed for rapid prototyping inthepres fcomplex applications. It has interfaces to many
OS system calls and libraries and is extensible to C or C++. Many large companies use the Python
programming language include NASA, Google, YouTube, BitTorrent, etc. Python is widely used in
Artificial Intelligence, Natural Language Generation, Neural Networks and other advanced fields of
Computer Science. Python had deep focus on code readability. Python language is used by author
due to his cross platform compatibility as main coding language for algorithm. Open CV and Dlib
libraries are integrated in python interpreter for using readymade optimized functions.

PyQT5: PyQt5 is a Python binding of the cross-platform GUI toolkit Qt, implemented as a
Python plug-in. PyQt is free software developed by the British firm Riverbank Computing. PyQt
supports Microsoft Windows as well as various flavors of UNIX, including Linux and Mac OS.
highly optimized open source libraries of Python is used by author for GUI design and graphical
data plotting.PyQt5 PyQtgraph function is used by author for live graphical monitoring of EAR
values.

2.6 SOFTWARE DESCRIPTION

2.6.1 PYTHON
Python is a significant level programming language used on a very basic level for generally
helpful programming. It was made in the year 1991 by Guido van Rossum. It has a structure
sanity that underlines generally on points of view, for instance, code intelligibility and
strikingly uses significant whitespace. It is a viable programming language and is commonly
used over the globe for both little and tremendous scope programming.
Python has motorized memory the board. It gives different programming perfect models
some of which are object-arranged, fundamental, utilitarian and procedural. It in like manner
has a broad and sweeping standard library. Favorable circumstances of Python which will help
us in this task are:
 Python can run on any Operating System as it is stage free.
 Python grammar is straightforward and simple to recollect.
 Size and intricacy of the code is decreased in python when contrasted with

Dept.of CSE,NHCE 19
DRIVER DROWSINESS DETECTION SYSTEM

other programming dialects.

 The prototyping process is accelerated because of the utilization of a compiler
over a mediator.
 Python underpins every one of the 3-procedural programming,
object-situated programming just as useful programming and is in this
manner considered extremely proficient.
Some of the striking utilizations of Python are:
 Helps in server-side programming in Web improvement
 Used tremendously in programming advancement where various complex
issues should be settled
 Used in insights
 Used for System scripting
 Python bolsters database network and can likewise peruse from, write to and
annex documents.
 It can be utilized in taking care of huge information and furthermore to
determine complex scientific issues.
 It is additionally utilized for programming prototyping in the product
advancement process
2.6.2 SPYDER/ANACONDA
Spyder is a Python Integrated Development Environment(IDE) for programming in the Python
programming language. It is an open source, cross-stage IDE accessible on Anaconda. Spyder
has a voluminous arrangement of inbuilt libraries that can be utilized. Not many of them are:
 NumPy
 SciPy
 Matplotlib
 Pandas
 IPython
 SymPy
 Cython
Spyder was made by Pierre Raybaut in 2009 and post 2012 Spyder has been kept up and
unendingly improved by a gathering of intelligent Python engineers and a submitted network

Dept.of CSE,NHCE 20
DRIVER DROWSINESS DETECTION SYSTEM

for the equal.

 PyFlakes
 Pylint
 Rope
For GUI, Spyder utilizes Qt. It is intended to utilize both of the PyQt or PySide Python ties. A
slim reflection layer created by the Spyder venture and later embraced by various different
bundles and gives the opportunity to utilize either backend is called QtPy. It is a python system
used to create basic frontend GUIs for python activities and projects created in Python.
As a cross-stage IDE it is accessible through:
 Anaconda
 Windows,
 macOS through MacPorts,
and on significant Linux conveyances, for example,
 Debian
 Arch Linux,
 Gentoo Linux
 Fedora,
 openSUSE
 Ubuntu
Spyder can be extensible with first-gathering and outsider modules and has the game plan
for instinctive gadgets for data audit and embeds Python-unequivocal code quality
confirmation and thoughtfulness instruments.

Dept.of CSE,NHCE 21
DRIVER DROWSINESS DETECTION SYSTEM

Fig 2.5 Anaconda Architecture

 The salient features of Spyder IDE are:

 Comprises of an editor with efficient syntax highlighting, introspection and code

completion on typing a part of it

 It facilitates use of parallel Python consoles on the IDE

 It has an illustrious Help Pane which facilitates in retrieving and rendering rich text
documentation on functions, classes and methods either manually or

automatically.

 It has the ability to explore and edit the variables from a GUI

Dept.of CSE,NHCE 22
DRIVER DROWSINESS DETECTION SYSTEM

 For the purpose of step-by-step execution, a debugger linked to IPdb is used.

 It enables static code analysis with the use of Pylint

 It has a run-time Profiler in order to mark the code

 It has an efficient project support that allows easy and parallel work on multiple
development projects simultaneously

 It has an in-built file explorer which can be used for browsing and interacting with the
filesystem and managing projects

 It has a search option which can be used to search for anything over a vast filesystem
with a single regular expression

 It has a provision for online help that allows users to search and view Python and its
various packages documentation inside the IDE

 It has an internal console which allows for introspection and control over it’s own
operation

 And last but not the least, it consists of a history log, which can come in handy whenwe
havetorecordeveryusercommandwhichwasenteredineachconsole
 of the Spyder IDE.

Available pluginsinclude:

Spyder-Unit test, which faicilitates in integrating the popular unit testing frameworks
Pytest, Unittest and Nose with the support of Spyder from

Anaconda.
Spyder-Notebook, which allows to view and edit Jupyter Notebooks within the IDE

Spyder-Reports, which enables the use of literate programming techniques in Python

Spyder-Terminal, provides the ability to open, control and manage cross- platform
system shells within Spyder

Spyder-Vim, which has commands and shortcuts for emulating the Vim text editor

Spyder-AutoPEP8, facilitating in automatically conforming code to the standard PEP 8

Dept.of CSE,NHCE 23
DRIVER DROWSINESS DETECTION SYSTEM

codestyle

Spyder-Line-Profiler and Spyder-Memory-Profiler, which extends the built-in profiling

functionality to include testing an individual line, and measuring

memory usage

2.6.3 PANDAS

The dataset which is gotten will be in csv design. i.e they will be comma- separated
qualities. This dataset must be sustained into Pandas to perform prescient investigation.
Pandas is utilized alongside python programming language and is accordingly an ideal fit for
this task. It gives data structures and activities to recovering and controlling numerical tables
and time arrangement. It is an open source programming, made allowed to use by people in
general. The name Pandas is gotten from the econometrics term "panel data" which is utilized
for data sets that incorporate readings over numerous timespans for a similar entity.

The prominent features of Pandas are:

It provides support for Integrated indexing of Data Frame objects.

It has numerous tools that facilitate in reading and writing of data between different
file formats.

Data alignment and missing data, if any is taken care of.

Reduction in the size of datasets based on certain criteria is simplified.

Indexingandsubsetting oflargedata setstohaveareferenceismadeavailable.

Insertion and deletion of columns is made easy.

Has a “group by” engine which provides split-apply-combine operations on the data
sets.

Merging and joining of datasets is supported.

It facilitates hierarchical axis indexing in order to work with high-dimensional data in a

lower-dimensional data structure.

Dept.of CSE,NHCE 24
DRIVER DROWSINESS DETECTION SYSTEM

Time series-functionality: It includes a myriad of functionalities such as frequency

conversion, moving window statistics, moving window linear regressions, date

shifting and lagging.

It provides data filtration for the datasets.

2.6.4 SKLEARN
Scikit-learn is an open source software machine learning library for the Python programming
language. It encompasses numerous classification, regression and clustering algorithms
including support vector machines, random forests, gradient boosting, k-means and DBSCAN,
and is designed to collaborate with the Python numerical and scientific libraries NumPy and
SciPy.

The scikit-learn which was initially called as scikits.learn was a “Google Summer of Code” project
by David Cournapeau. The name originates from the notion that it is a "SciKit"

(SciPy Toolkit), a separately-developed and distributed third-party extension to SciPy. The

original codebase was later rewritten by other developers.

2.6.5 HTML and CSS

HTML stands for Hyper Text Markup Language whereas CSS stands for Cascading Style Sheets.

HTML is the standard language used in web development. CSS is used for styling
HTMLdocuments. Javascript documents can be embedded in HTML documentsforincluding
certain functionalities.Advantages and main reasons for choosing HTML for the front end of this
project are:
It easy to learn and use. Since the main objective of this project is the system that predicts
malnourishment, and not the GUI, HTML is the ideal choice for developing the front
end of this project.
HTML is supported by all browsers. So the adaptability of the system will be considerably
good.HTML syntax is very similar to XML syntax. Owing to this similarity, it will be very
convenient to use both the languages for developing the webpage.HTML is free to use and
that is another reason for using it in this project front end implementation. Another
reason is that HTML is supported by majorityof the web developmenttools.

Dept.of CSE,NHCE 25
DRIVER DROWSINESS DETECTION SYSTEM

2.6.6 SQL-SERVER
SQL database server is used to store the userid and password of the users who log on to
the website. They can log on to the web page using these credentials and access the data that
they had initially entered, the medical details of their children that is fed toPandas.

Some of the advantages of SQL Server-

 It is simple to store, manipulate and retrieve data.

 Security features arebetter.

 Lower cost ofownership.

 Enhanced performance.

Dept.of CSE,NHCE 26
DRIVER DROWSINESS DETECTION SYSTEM

Chapter 3
REQUIREMENT ANALYSIS
3.1 Functional Requirements
A Functional prerequisite is described as one portion or an element of a product , in the entire
methodology of programming building. A capacity or part is also depicted as the lead of a section
and its yields, given a great deal of data sources. A useful prerequisite may be the figuring identified
with specialized and subtleties or data control and getting ready or whatever other express
usefulness that describes the target of a particular structure uses the useful necessities are found
being utilized cases.
a. Recording the driver's steering behavior the moment the trip begins.
b. Then recognizes changes over the course of long trips
c. Determines the driver's level of fatigue.

3.2 Non- Functional Requirements

a. Image processing is done using the captured video.
b. Image is stored in a library called OpenCV.
c. Stored image undergo various algorithm and detects if the driver is fatigue
and if fatigue raises an alarm.

3.3 Hardware Requirements

Processor Intel or high level
RAM 2 GB
Space on disk Min 2 GB
Device Any device that has internet access
Execution space Min 100mb

Dept.of CSE,NHCE 27
DRIVER DROWSINESS DETECTION SYSTEM

The system basically consists of webcam, microcontroller ARM7, buzzer, vibrator, LED‟s, relay.
Microcontroller is used to perform various function and operations is ARM7 i.e. LPC2148. This
paper presents a non-intrusive approach for drowsiness detection, based on computer vision. The
webcamera is installed in a car and it is able to work under real operation conditions.

3.4 Software Requirements

The implementation is done by using EmguCV which is an open-source image processing library for
C#.NET. EmguCV is a cross platform .Net wrapper to the OpenCV image processing library. It permit
OpenCV functions to be called from .NET compatible languages such as C#, VB, VC++, Iron, Python
etc..To detect human facial feature, Intel developed an Open source library used for the
implementation of computer vision related programs called OpenCV(Open source computer
vision). . This creates a classifier given a set of positive and negative samples. OpenCV is an open
source computer vision library. It is designed for computational efficiency and with a strong focus
on real time applications. It helps to build vision applications quickly and easily. OpenCV satisfies
the low processing power and high speed requirements of our application.

Dept.of CSE,NHCE 28
DRIVER DROWSINESS DETECTION SYSTEM

CHAPTER 4

DESIGN

Drivers face is continuously monitored using a video camera. In order to detect the drowsiness the
first step is to detect the face using the series of frame shots taken by the camera. Then the
location of the eyes is detected and retina of the eye is continuously monitored. The captured
image is sent to the Raspberry Pi board for image processing. The raspberry Pi converts the
received image to digital signal using Open CV. The digital signal is transmitted from transmitter to
the receiver. Both the transmitter and the receiver are paired up. The signal is then passed to the
LPC2148, the microcontroller. If the signal crosses the threshold of two seconds, then the alarm
beeps and the speed of the vehicle is automatically reduced.The drowsiness detection and
correction system developed is capable of detecting drowsiness in a rapid manner. The system
which can differentiate normal eye blink and drowsiness which can prevent the driver from
entering the state of sleepiness while driving. The system works well even in case of drivers wearing
spectacles and under low light conditions also. During the monitoring, the system is able to decide
if the eyes are opened or closed. When the eyes have been closed for about two seconds, the alarm
beeps to alert the driver and the speed of the vehicle is reduced. By doing this many accidents will
reduced and provides safe life to the driver and vehicle safety. A system for driver safety and car
security is presented only in the luxurious costly cars. Using drowsiness detection system, driver
safety can be implemented in normal cars also. The raspberry Pi converts the received image to
digital signal using Open CV. The digital signal is transmitted from transmitter to the receiver. Both
the transmitter and the receiver are paired up. The signal is then passed to the LPC2148, the
microcontroller. If the signal crosses the threshold of two seconds, then the alarm beeps and the
speed of the vehicle is automatically reduced.

Dept.of CSE,NHCE 29
DRIVER DROWSINESS DETECTION SYSTEM

This projected is designed to achieve certain goals which are:

 Driver drowsiness detection is a car safety technology which helps to save the
life of the driver by preventing accidents when the driver is getting drowsy.
 The main objective is to first design a system to detect driver’s drowsiness by
continuously monitoring retina of the eye.
 The system works in spite of driver wearing spectacles and in various lighting
conditions.
 To alert the driver on the detection of drowsiness by using buzzer or alarm.
 Speed of the vehicle can be reduced.
 Traffic management can be maintained by reducing the accidents.

The drowsiness detection and correction system developed is capable of detecting

drowsiness in a rapid manner. The system which can differentiate normal eye blink
and drowsiness which can prevent the driver from entering the state of sleepiness
while driving. The system works well even in case of drivers wearing spectacles and
under low light conditions also. During the monitoring, the system is able to decide if
the eyes are opened or closed. When the eyes have been closed for about two
seconds, the alarm beeps to alert the driver and the speed of the vehicle is reduced.
By doing this many accidents will reduced and provides safe life to the driver and
vehicle safety. A system for driver safety and car security is presented only in the
luxurious costly cars. Using drowsiness detection system, driver safety can be
implemented in normal cars also.Once the eye aspect ratio calculated, algorithm can
threshold it to determine if a person is blinking the eye aspect ratio will remain
approximately constant when the eyes are open and then will rapidly approach zero
during a blink, then increase again as the eye opens. The duration of blink further
provide estimation of microsleep. The proposed algorithm has been tested on
personal car driver for testing purposes. For authentic results, the camera position
was focused on the driver’s face. Further, the algorithm has been tested in day time
driving and Night time using IR camera. The results are discussed in Result section
and found satisfactory. The proposed algorithm focused solely on using the eye
aspect ratio as a quantitative metric to determine if a person has blinked in a video.

Dept.of CSE,NHCE 30
DRIVER DROWSINESS DETECTION SYSTEM

4.1 System Architecture

4.1.1 Class Diagram

Fig 4.1 Class Diagram

This viewpoint contains the static structure of system with entities, interaction
among entities and software patterns that applied to the static structure of the
system. Figure shows associated class diagram which contains information.

1. Main Activity : This class is the starting point of our project. It starts all the
services. Methods getBatteryPercentage(): Gets the percentage of battery level.
SpeakOut(): Triggers Text-To-Speech.

2. STTService : This class is responsible for Speech-To-Text. It is supposed to work

at background all the time and receive voice commands. Attributes serviceState:
Stores the current state of service`: WAITING,LISTENING, PROCESSING Methods
getNumberWithName(String name): Gets the phone number of according to the
name from contacts. calling(String phoneNumber):Calls the number.

Dept.of CSE,NHCE 31
DRIVER DROWSINESS DETECTION SYSTEM

3. ShakeEventListener : This class implemented for detecting system crashes.

4. RecognitionListener : This class implemented for speech recognition as

background.

5. NLServic : This class implemented for notification handling.

6. DrowsinessDetection : This class implements driver drowsiness detection.

4.1.2 State Diagram

Fig 4.2 State Diagram

1. Idle

When the application starts, it waits for starter command. After it receives the
starter command, it goes the listening state.

2. Listening

In this state, the application waits for a valid command. If one of the supported
command is received, it goes to one of the states to perform a task according to

Dept.of CSE,NHCE 32
DRIVER DROWSINESS DETECTION SYSTEM

the received command.

3. Message

If the user says “message” in the listening state, the application enters the
message state. When an existing contact name is provided, the application enters
the feedback state. If the contact name is not found, the application goes to idle
state.

4. Record

Video Whenever user gives the command “record” in the listening state, the
application enters the Video Record state. After the recording starts, the
application goes to idle state. If the user stops video recording, video is saved to
the hard drive.

5. Eye Detection

When the users says “detect” in the listening state, the application enters the Eye
Detection state and starts detection process. If the user closes eye detection, the
application goes to the idle state.

Dept.of CSE,NHCE 33
DRIVER DROWSINESS DETECTION SYSTEM

4.1.3 Sequence Diagram

Fig 4.3 Sequence Diagram

1. Camera : Captures the video of the driver when the vehical is in motion and sends
the captured video to the storage unit.

2. Facial Landmark Detector : Detects the drivers facial changes and reports then.

3. Driver : The person who is driving the vehicle.

4. Drowsiness Detection Sysytem : The Image capturing mechanism where sequence

of images are processed using various algorithms to detect whether the driver is
fatigue or not.

Dept.of CSE,NHCE 34
DRIVER DROWSINESS DETECTION SYSTEM

4.1.3 Data Flow Diagram

Fig 4.4 Data Flow Diagram

The first step in building a blink detector is to perform facial landmark detection to
localize the eyes in a given frame from a video stream. The eye aspect ratio for each
eye can be calculated using Euclidian distance functions of OPEN CV , which is a
singular value, relating the distances between the vertical eye landmark points to the
distances between the horizontal landmark points. Once the eye aspect ratio
calculated, algorithm can threshold it to determine if a person is blinking the eye
aspect ratio will remain approximately constant when the eyes are open and then
will rapidly approach zero during a blink, then increase again as the eye opens. The
duration of blink further provide estimation of microsleep. The proposed algorithm
has been tested on personal car driver for testing pu

Dept.of CSE,NHCE 35
DRIVER DROWSINESS DETECTION SYSTEM

4.2 EXPECTED OUTCOME

We have used Open CV as a platform to develop a code for eye detection in real time.
The code is then implemented on system installed with Open CV software. To detect
human eyes, face has to be detected initially. This is done by OpenCV face haar
cascade classifier. Once the face is detected, the location of the eyes is estimated
and eye detection is done using eye Haar-cascade classifier. Hence using the open CV,
face and eyes are detected accurately and displayed on the monitor as shown in the.
The larger yellow square indicates the face while smallerred squares indicate the
eyes. Once face and eyes are detected, it is checking status of eyes i.e. open or
closed state of the eyes. If both eyes remain closed for successive frames, it indicates
that the driver is drowsy and gives the warning signal.A webcam is a video camera
that feeds its image in real time to a computer or computer network. Unlike an IP
camera (which uses a direct connection using Ethernet or Wi-Fi), a webcam is
generally connected by a USB cable, FireWire cable, or similar cable. Their most
popular use is the establishment of video links, permitting computers to act as
videophones. The common use as a video camera for the World Wide Web gave the
webcam its name. Other popular uses include security surveillance, computer vision,
video broadcasting, and for recording social videos. A fatigue detection system based
on the above method was implemented by using Visual C++. At first, we fix a camera
on a car in front of the driver. Then we capture some videos from 8 drivers in normal
conditions. The whole input image format is 320×240 and they are in RGB color
space. We have also found that the optimum distance from camera which obtained
about 30cm-50cm that is very suitable for our method.The average accuracy of our
combination method is 90.873%. Thus our eye detection method is robust and
irrelevant with different sizes and more accurate. According to obtained results, our
system can determine the eye states with a high rate of correct decision.

Dept.of CSE,NHCE 36
DRIVER DROWSINESS DETECTION SYSTEM

Chapter 5
Testing
Software testing is defined as an activity to check whether the actual results match
the expected results and to ensure that the software system is Defect free. It
involves execution of a software component or system component to evaluate one
or more properties of interest.Software testing also helps to identify errors, gaps or
missing requirements incontrary to the actual requirements. It can be either done
manually or using automated tools.

5.1 Importance of Testing

1. Software testing is really required to point out the defects and errors that were
made during the development phases.Example: Programmers may make a mistake
during the implementation of the software. There could be many reasons for this like
lack of experience of the programmer, lack of knowledge of the programming
language, insufficient experience in the domain, incorrect implementation of the
algorithm due to complex logic or simply human error.

2. It’s essential since it makes sure that the customer finds the organization reliable
and their satisfaction in the application is maintained.
If the customer does not find the testing organization reliable or is not satisfied with
the quality of the deliverable, then they may switch to a competitor organization.
Sometimes contracts may also include monetary penalties with respect to the
timeline and quality of the product. In such cases, if proper software testing may also
prevent monetary losses.

3. It is very important to ensure the Quality of the product. Quality product delivered
to the customers helps in gaining their confidence. (Know more about Software
Quality) As explained in the previous point, delivering good quality product on time
builds the customers confidence in the team and the organization.

Dept.of CSE,NHCE 37
DRIVER DROWSINESS DETECTION SYSTEM

4. Testing is necessary in order to provide the facilities to the customers like the
delivery of the high quality product or software application which requires lower
maintenance cost and hence results into more accurate, consistent and reliable
results. High quality product typically has fewer defects and requires lesser
maintenance effort, which in turn means reduced costs.

5. Testing is required for an effective performance of software application or

product.

6. It’s important to ensure that the application should not result into any failures
because it can be very expensive in the future or in the later stages of the
development. Proper testing ensures that bugs and issues are detected early in the
life cycle of the product or application. If defects related to requirements or design
are detected late in the life cyle, it can be very expensive to fix them since this might
require redesign, re-implementation and retesting of the application.

7. It’s required to stay in the business.Users are not inclined to use software that has
bugs. They may not adopt a software if they are not happy with the stability of the
application. In case of a product organization or startup which has only one product,
poor quality of software may result in lack of adoption of the product and this may
result in losses which the business may not recover from.

5.2 Types of Testing

5.2.1. Functional Testing:

Functional Testing is a type of software testing whereby the system is tested against
the functional requirements/specifications. Functions (or features) are tested by
feeding them input and examining the output. Functional testing ensures that the
requirements are properly satisfied by the application. This type of testing is not
concerned with how processing occurs, but rather, with the results of processing. It
simulates actual system usage but does not make any system structure assumptions.
During functional testing, Black Box Testing technique is used in which the internal

Dept.of CSE,NHCE 38
DRIVER DROWSINESS DETECTION SYSTEM

logic of the system being tested is not known to the tester.Functional testing is
normally performed during the levels of System Testing and Acceptance Testing.
Typically, functional testing involves the following steps:

● Identify functions that the software is expected to perform.

● Create input data based on the function’s specifications.

● Determine the output based on the function’s specifications.

● Execute the test case.

● Compare the actual and expected outputs.

Functional testing is more effective when the test conditions are created directly
from
user/business requirements. When test conditions are created from the system
documentation (system requirements/ design documents), the defects in that
documentation will not be detected through testing and this may be the cause of
end-users’ wrath when they finally use the software.

5.2.2. Non-Functional Testing

NON-FUNCTIONAL TESTING is defined as a type of Software testing to check

non-functional aspects (performance, usability, reliability, etc) of a software
application. It is designed to test the readiness of a system as per nonfunctional
parameters which are never addressed by functional testing. An excellent example
of a non-functional test would be to check how many people can simultaneously
login into a software. Non-functional testing is equally important as functional
testing and affects client satisfaction. The Non Functional requirements were also
not given proper attention in the earlier test cycles. However, this has changed now.
Non-functional tests are now most important as they consider all the application
performance and security issues these days. This testing has a greater impact on
applications when it comes to the performance of the application under high user
traffic. This testing ensures that your application is stable and is
able to handle load under extreme conditions.

Dept.of CSE,NHCE 39
DRIVER DROWSINESS DETECTION SYSTEM

5.2.3. Regression Testing

REGRESSION TESTING is a type of software testing that intends to ensure that

changes (enhancements or defect fixes) to the software have not adversely affected
it. The likelihood of any code change impacting functionalities that are not directly
associated with the code is always there and it is essential that regression testing is
conducted to make sure that fixing one thing has not broken another thing.
During regression testing, new test cases are not created but previously created test
cases are re-executed. Regression [noun] literally means the act of going back to a
previous place or state; return or reversion. In an ideal case, a full regression test is
desirable but oftentimes there are time/resource constraints. In such cases, it is
essential to do an impact analysis of the changes to identify areas of the software
that have the highest probability of being affected by the change and that have the
highest impact to users in case of malfunction and focus testing around those areas.
Due to the scale and importance of regression testing, more and more companies
and projects are adopting regression test automation tools.

5.2.4. Performance Testing

Performance testing is the process of determining the speed, responsiveness and

stability of a computer, network, software program or device under a workload.
Performance testing can involve quantitative tests done in a lab, or occur in the
production environment in limited scenarios. Typical parameters include processing
speed, data transfer rate, network bandwidth and throughput, workload efficiency
and reliability. For example, an organization can measure the response time of a
program when a user requests an action or the number of millions of instructions
per second (MIPS) at which a mainframe functions.

Dept.of CSE,NHCE 40
 DRIVER DROWSINESS DETECTION SYSTEM

CHAPTER 6
OUTPUT SCREENSHOTS

Fig 6.1. Open eyes detection

Fig 6.2. Closed eyes detection and alerting

Dept.of CSE,NHCE 41
DRIVER DROWSINESS DETECTION SYSTEM

CHAPTER 7
CONCLUSION

This project tells how to build a drowsiness detector using OpenCV, Python, and Dlib
opensource Libraries. The first step in building a blink detector is to perform facial
landmark detection to localize the eyes in a given frame from a video stream. The
eye aspect ratio for each eye can be calculated using Euclidian distance functions of
OPEN CV , which is a singular value, relating the distances between the vertical eye
landmark points to the distances between the horizontal landmark points. Once the
eye aspect ratio calculated, algorithm can threshold it to determine if a person is
blinking the eye aspect ratio will remain approximately constant when the eyes are
open and then will rapidly approach zero during a blink, then increase again as the
eye opens. The duration of blink further provide estimation of microsleep. The
proposed algorithm has been tested on personal car driver for testing purposes. For
authentic results, the camera position was focused on the driver’s face. Further, the
algorithm has been tested in day time driving and Night time using IR camera. The
results are discussed in Result section and found satisfactory. The proposed
algorithm focused solely on using the eye aspect ratio as a quantitative metric to
determine if a person has blinked in a video stream. However, due to noise in a
video stream, subpar facial landmark detections, or fast changes in viewing angle, a
simple threshold on the eye aspect ratio could produce a false-positive detection,
reporting that a blink had taken place when in reality the person had not blinked. To
make our blink detector more robust to these challenges further following
improvements can be implemented Computing the eye aspect ratio for the N th
frame, along with the eye aspect ratios for N – 6 and N + 6 frames, then
concatenating these eye aspect ratios to form a 13 dimensional feature vector.

Dept.of CSE,NHCE 42
DRIVER DROWSINESS DETECTION SYSTEM

REFERENCES

[1] Association for Safe International Road Travel (ASIRT), Road Crash
Statistics.http://asirt.org/initiatives/informingroadusers/road-safety-facts/road-cras
h-statistics, 2016

[2] https://en.wikipedia.org/wiki/Microslee

[3] Journal of VLSI Signal Processing 23, 497–511 (1999) c °1999 Kluer Acadmic
Publishers.Manufactured in The Netherlands.

[4] https://docs.opencv.org/trunk/d7/d8b/tutorial_py_face_de tection.html

[5] Eye Detection Using Morphological and Color Image Procesing Tanmay
Rajpathaka, Ratnesh Kumar and Eric Schwartzb

[6] A Robust Algorithm for Eye Deteuction on Grey Intensity Face without
Spectacles- JCS&T Vol. 5 No. 3

[7] Froba Kebbuck: Audio- and Video-Based Biometric Person Authentication, 3rd
International Conference, AVBPA 2001, Halmstad, Sweden, June 2001. Proceedings,
Springer. ISBN 3-540-42216-1.

Dept.of CSE,NHCE 43