1st KEC Conference Proceedings| Volume I ISBN 978-9937-0-4872-9

KEC Conference September 27, 2018

SMART GLOVE FOR SIGN LANGUAGE

TRANSLATION USING ARDUINO
Bijay Sapkota Mayank K. Gurung Prabhat Mali Rabin Gupta
Department of Department of Department of Department of
Computer and Computer and Computer and Computer and
Electronics Electronics Electronics Electronics
Engineering Engineering Engineering Engineering
Kantipur Engineering Kantipur Engineering Kantipur Engineering Kantipur Engineering
College College College College
Lalitpur, Nepal Lalitpur, Nepal Lalitpur, Nepal Lalitpur, Nepal
bijaysdsp1057@hotmai mayankgurung95@gm [email protected] guptarabin96@hotmail.
l.com ail.com m com

ABSTRACT- The easiest way for The number of speech impaired

communication in the world is speech. and hearing impaired people are increasing
Whereas it becomes difficult for speech day by day. About 70 million people in the
impaired and hearing impaired people world are speech impaired and hearing
to communicate as they use sign impaired [1]. We often come across these
language for the communication. It people communicating with the normal
becomes difficult for normal people to world. These people communicate with the
understand. Smart Glove for Sign help of sign language. When a speech
Language Translation is a work that impaired person tries to communicate with
aims to present an easy way of normal person and vice-versa, they feel
communication for speech impaired and difficult to understand. To bridge the gap
hearing impaired people. This work between speech impaired, hearing
consists of a glove equipped with sensors impaired and normal masses, gesture
which senses different sign language recognition system is being used. Gesture
gesture, these senses data are fed to recognition is a widely explored field. A
arduino and transfer data to android lot of work has been done in the past few
phone via Bluetooth module, a common years. An electronic device has been used
android phone is used in this work for as a language interpreter and provides
sign to voice translation and voice to convenient way for communication
sign language translation. between speech impaired, hearing
impaired and normal people. Gesture for
In real life, the sign language American standard language is shown in
users mostly use both hands. Thus, this figure 1. Gesture recognition is classified
is a prototype work presenting an ease into two main categories i.e. image
in communication for the speech processing based and sensor based. The
impaired and hearing impaired people. main disadvantage of image processing
based techniques includes complex
Keywords- Sign Language Translation, algorithms for data processing. Another
Sensor Based, Communication, challenge in image and video processing
Bluetooth module includes variant lighting conditions,
backgrounds and field of view constraints.
The sensor based technique offers greater
mobility.

I. INTRODUCTION

KECConference2018, Kantipur Engineering College, Dhapakhel, Lalitpur 5

 KEC Conference

and voice to sign language in order to

ensure effective and easy communication
between different communities.

Many methods for hand gesture

recognition using image processing have
been proposed. “Hand Gesture
Recognition System using Image
Processing” uses digital image processing
techniques using modified SIFT algorithm.
Figure 1: American Standard Sign With the help of the algorithm the sign
Language language is decoded successfully. The
advantage using this algorithm is high
processing speed which can produce result
This work uses sensor based in real time. Although the proposed
technique instead of image processing system is fast requires expensive materials
based technique. First of all, sign language also [2].
is converted to analog voltage signal using
flex sensor and accelerometer. Using ADC “Sign Language Recognition Using
from the microcontroller board analog Image Processing” uses image processing
signal is converted to digital signal. Now, with the help of SURF (Speed up Robust
the microcontroller processes this digital Features) algorithm. Video camera is used
signal, detects respective characters and to record hand movements, and the input
transmits through Bluetooth module to video is partitioned into frames, for each
Android phone. An application on android frame, a set of features are extracted. The
is used to display the characters received system is implemented in MATLAB [3].
from Bluetooth module in the form of text
and text is transformed to voice using “Deaf-Mute Communication
Google-text-to-speech. Also, voice is Interpreter” uses sensor based technique
transformed to display text and convert it comprising of flex sensor, tactile sensors
into sign language using speech and accelerometer to translate American
recognizer. Sign Language gestures to both text and
auditory voice. Although, they were only
II. LITERATURE able to translate thirteen sign into their
REVIEW respective alphabets namely letters ‘A’ ‘B’
‘C’ ‘D’ ‘F’ ‘I’ ‘L’ ‘O’ ‘M’ ‘N’ ‘T’ ‘S’ ‘W’
Sign language is the only and tactile sensor were used to improve the
communication tool used by speech accuracy of three letters M, N and T [4].
impaired and hearing impaired people to
communicate to each other. However, “Sign Language to Speech
normal people do not understand sign Translation System Using PIC
language and this will create a large Microcontroller” consists of flex sensors
communication barrier between speech that is used to detect finger gestures and
impaired, hearing impaired and normal gyro sensors for providing a signal
people. In addition, the sign language is corresponding to the orientation of the
also not easy to learn due to its natural motion of the hand [5].
differences in sentence structure and
grammar. Therefore, there is a need to “Design of Smart Gloves” uses pair
develop a system which can help in of gloves with flex sensors along each
translating the sign language into voice finger, thumb and arm is used to capture

KECConference2018, Kantipur Engineering College, Dhapakhel, Lalitpur 6

 KEC Conference

the movement of user. The problem with order to match the resultant output with
this work is it can detect only few letters pre-stored values of different signs
[6]. regarding the alphabets. For this,
appropriate ranges are set for each
“A Review Paper on Smart Glove - alphabet and the words that can be
Converts Gestures into Speech and Text” recognized with single hand based on the
uses five flex sensors and accelerometer measured data obtained from repeated
attached on the back of the glove to measurements. A Bluetooth module is
measure the bending and motion of the connected to Arduino UNO. The
hand. The problem with this work is to Processed data’s are then transferred to the
recognize some letters. Besides that, letters Bluetooth module (transmitter) obtained in
M, N, O, R, S, T, V and X cannot be string format. The Android mobile also
displayed due similar in gesture with have an inbuilt Bluetooth capability. These
another letters [7]. two Bluetooth devices are then paired, and
string is transmitted to Android mobile.
“Digital Text and Speech Android mobile receives data via
Synthesizer Using Smart Glove for Deaf Bluetooth in bytes format, convert them
and Dumb” uses five flex sensor to detect into string. Finally, the string is converted
gesture and accelerometer used for sensing into voice using the text to speech
axis x, y and z, find the angle of the glove application of Android mobile. This
tilted with respect to the. The glove is overall system is mounted over a normal
capable of translating their sign language glove for easy handling and recognizes the
gestures into speech through android hand gestures accurately.
phone. This smart glove focuses the
translation of gestures of words only [8]. The work also transforms voice to
respective text and sign. Speech recognizer
, “Smart Glove: Gesture Vocalizer detects the speech and converts into
for Deaf and Dumb People” uses glove at respective text. The image of used letters
the transmitter side which has to be worn and words are stored in database. The text
by the user. This glove is mounted with 4 from speech recognizer is compared with
flex sensors each on the 4 fingers of the name of image and required image is
glove namely thumb, index, middle and displayed.
ring. This work also focuses the translation
of gestures of words only [9]. The work also transforms text to
respective voice and sign. Google-text-to-
III. WORKING speech converts the typed text into
PRINCIPLE respective speech. The text from speech
recognizer is compared with name of
The flex sensor measures the image and required image is displayed.
bending of fingers according to gesture
and outputs change in resistances
corresponding to the amount of bending.
Accelerometer sensor measures the linear
movements of hand in X-axis and outputs
different values of X corresponding to the
movement in X-axis. Touch sensor
measures if there is any contact between
two fingers. All the data’s from sensors are
then processed on Arduino UNO involves
combination of all the sensor outputs in

KECConference2018, Kantipur Engineering College, Dhapakhel, Lalitpur 7

 KEC Conference

Flex VII. FLOW CHART FOR

Sensors ANDROID
APPLICATION
Accelero-
Arduino Bluetooth
UNO Module Start Smart Glove Application
meter

Touch Turn on
Sensors Bluetooth If voice

Connect
to Smart Yes
Sound & Glove
Gesture Convert
Mobile Android to text
Bluetooth Mobile and
Receiver Micro-
Waiting gesture
phone
for data

No
Android Text If text
Application If data
received

Figure 2: Block diagram for smart glove Yes

Yes

Convert
VI. FLOW CHART OF SMART Convert
to voice
to text
GLOVE and
gesture
Turn on Glove
Translate
to voice
Read sensor data

Figure 4: Flowchart of Receiving Section

No
If predefined
value Wait VIII. RESULT AND DISCUSSION

The outcome of this work was

Yes
estimated where all the American standard
Outputs
result
sign language with their respective 26
alphabets and some words were
successfully displayed in to text and voice
Send output to
Bluetooth Module which can help speech impaired and
hearing impaired people to communicate
No with normal people easily. There is
If mobile another feature also of this system which
connected makes normal people to communicate with
Yes them also through an android application
Transmit Data
which can convert people voice in to text
and sign language. So, this project is quite
Figure 3: Flowchart of Transmitting feasible to develop two way
Section

KECConference2018, Kantipur Engineering College, Dhapakhel, Lalitpur 8

 KEC Conference

communication between speech impaired Table 2: Output of hand gesture ‘C’ in

hearing impaired and normal people. android application in text and in graph

All of the works mentioned, deals only Sign Output

with one way communication from
impaired people to normal mass. This
work allows two way communications
between these groups. Also, this works
converts all of the letters and some words
unlike others. The only work that converts
all letters and some word is “Digital Text
and Speech Synthesizer Using Smart
Glove for Deaf and Dumb” but this work
deals with converting gestures into text
and speech only.

For convenience, four alphabets ‘A’ ‘C’

‘U’ and ‘V’ with sign language, output and
graph have been shown below.

Table 1: Output of hand gesture ‘A’ in

android application in text and in graph
Table 3: Output of hand gesture ‘U’ in
Sign Output android application in text and in graph

Sign Output

KECConference2018, Kantipur Engineering College, Dhapakhel, Lalitpur 9

 KEC Conference

Table 4: Output of hand gesture ‘V’ in Figure 6 and 7 depicts another

android application in text and in graph feature of system that can help people
disability due to any diseases and
Sign Output accidental causes etc. It can help them to
learn gesture through android application
easily.

Figure 6: Text to Sign language

Figure 5 depicts the feature of

system which helps normal people to
communicate with speech impaired and
hearing impaired people by voice to sign
language.

Speak
“Hello”

Figure 7: Word to Sign language

Figure 5: Voice to Text and Sign language

KECConference2018, Kantipur Engineering College, Dhapakhel, Lalitpur 10

 KEC Conference

IX. CONCLUSION International Journal of Engineering

Research & Technology, Vol. 3, pp. 210-
The project proposes a system for 214, November 2014.
speech impaired and hearing impaired [7] K. Rastogi and P. Bhardwaj, “A
people using glove technology and enable Review Paper on Smart Glove - Converts
normal people to communicate with them Gestures into Speech and Text”,
too. The use of five flex sensor, touch International Journal on Recent and
sensors and an accelerometer on to a glove Innovation Trends in Computing and
demonstrate that it is helpful to beak the Communication, Vol. 4, pp. 92-94, May
gap between speech impaired hearing 2016.
impaired and normal people. This device [8] K. Kashyap, A. Saxena, H. Kaur,
will be an apt tool not only for the people A. Tandon and K. Mehrotra, “Digital Text
got such disability naturally rather it also and Speech Synthesizer Using Smart
helps disability due to oral diseases and Glove for Deaf and Dumb”, International
accidental cause make them to learn Journal of Advanced Research in
gesture through application easily. The Electronics and Communication
project can be enhanced further by Engineering, Vol. 6, pp. 421-428, May
including more words and different 2017.
standard sign language. [9] K.V. Fale, A. Phalke, P. Chaudhari
and P. Jadhav, “Smart Glove: Gesture
X. REFERENCES Vocalizer for Deaf and Dumb People”,
International Journal of Innovative
[1] https://www.quora.com/What- Research in Computer and
percentage-of-people-in-the-world-are- Communication Engineering, Vol. 4, pp.
mute 6800-6806, April 2016.
[2] S.P. More and A. Sattar, “Hand .
Gesture Recognition System using Image
Processing “, International Conference on
Electrical, Electronics and Optimization
Techniques, Vol. 3, pp. 1-4, April 2015.
[3] K.P. Kour and L. Mathew, “Sign
Language Recognition Using Image
Processing”, International Journals of
Advanced Research in Computer Science
and Software Engineering, Vol. 7, pp. 142-
145, August 2017.
[4] A. Rajamohan, R. Hemavathy and
M. Dhanalakshmi, “Deaf-Mute
Communication Interpreter”, International
Journal of Scientific Engineering and
Technology, Vol. 2, pp. 336-341, May
2013.
[5] K. Gunasekaran and R.
Maniknandan, “Sign Language to Speech
Translation System Using PIC
Microcontroller” International Journal of
Engineering and Technology, Vol. 5, pp.
1024-1028, May 2013.
[6] P. Verma, S.L. Shimi and S.
Chatterji, “Design of Smart Gloves”,

KECConference2018, Kantipur Engineering College, Dhapakhel, Lalitpur 11