International Journal of Recent Technology and Engineering (IJRTE)

ISSN: 2277-3878, Volume-8, Issue-2S11, September 2019

Artificial Intelligence Based Person

Identification Virtual Assistant
P.Praddeep, P.Balaji, S.Bhanumathi

x stant. Generally, these collaborators have these normal

Abstract: In future all the electronic gadgets are operated by highlights like programmed automatic speech recognition,
usingvirtualassistantwhichisanythingbutdifficulttogettoyetit needs
in security. Project aims to provide security for virtual Assistant Text to speech, manufactured talking face and dialog
(VA) through facial recognition. The framework enables just
management [4,14].
approved users to access voice commands. By this we can get
protection and security for virtual assistant (VA). Users can ask The virtual assistant is improved by giving Facial
their help addresses like time, date and climate and find solution Recognition framework. The facial recognition framework
to the inquiries. This virtual assistant causes us to send email forvirtualassistantusingAIstrategiestodetectandrecognize
through voice commands and it also takes notes from voice faces. They are two kinds of users like approved clients and
commands with security. It gives access to the unapproved user to unapproved users [2]. The user stands before the camera
enlist with the required consent from the administrator. It is can
which takes different pictures of the user. The captured
exchange the pictures and documents just by using voice
commands. It will take photographs using camera when we use pictures experience the face detection process [12]. In this
the fitting voice commands. Various users in a family can get procedureitidentifiesfacesinthepicturesandenablesaccess to
access to the virtual Assistant by facial recognition module. the virtual assistant for the approvedusers.

Index Terms: Virtual Assistant, Facial Recognition and II. RELATEDWORK

Security.
The author in [1] presented about the attacks to voice
I. INTRODUCTION assistance, that can be followed in a several availabilities
which are given in several operating systems attackers can
In Today's advanced crisp market for artificialintelligence principally control these apparatuses to perform unapproved
could be a key unlocking the users of tomorrow. Man-made commands. The assailants could release the delicate datalike
brainpowerisasofnowallaroundtheuser.Manydependon it user’s area visit web pages which contain malware to gain
every day as per Gartner about 38% of consumers have unapprovedaccesstocertaingadgetsthispromptsspillageof
utilized virtual assistant benefits on their smart devices as of touchydata.
late numerous enterprises receiving AI to convey the logical Voice assistance can be utilized without getting any
conversationalconsistent and customized home permission from the user. Voice assistance attacks are
administrations now is the ideal time to be natural and generally static attacks, attackers broadly utilize recorded
proactiveabouthowuserswillencounteritandcrossoverany sound documents which are played by applications or by
barrier between the innovation and how it helps users are staticmediumthismaycausesendingmailsandsesamevoice
devouring it then AI gets advanced with power huge calls
information and investigation it can offer a one of a kind and Even send post via web-based networking media
separate clientexperience. aggressors can utilize voice help for long range interpersonal
Users like to associate with the machines using voice communication applications to post destructive things from
commands which are conceivable through the virtual user account.
assistant,gadgetsforexample,smartphones,smartTV’ s and car At times the voice recognition may fail by getting great
navigation systemsandsoforth.Thevirtualassistantsare the exampleofvictim’svoiceandutilizepreparedvoiceengineto
astute operators that can enable userstocomplete [13]. copy his voice. The usage of a voice order framework as an
Undertakingmoresuccessfullyandadvantageouslybymeans Intelligent Personal Assistant (IPA) can play out various
ofspokencollaborationsamonguserandthevirtualassistant. errands or administrationsfora person. In this framework one
Thevirtualassistantscansupportawidescopeforallusersin can make inquiries to the
business enterprises, education, government, medicinal framework,conjureitsAIgenerallygetitfromWikipedia[2].
services and diversion. The main organizations planned their Theauthorsinthe[4]proposedthatinsight full projectswith
own virtual assistants, for example, Microsoft's Cortina, natural language processing that are as of nowaccessible,with
Apple'sSiri,Amazon'sAlexaSamsungSVoice,andGoogleAssi various classifications of support, and look at the helpfulness
of one explicit bit of programming as a Virtual Personal
Revised Version Manuscript Received on 10 September, 2019.
Penumarthi Pradeep, Department of Computer Science and Assistant. Which are analyzed by fundamental regular
Engineering, Sathyabama Institute of Science and Technology, Chennai, language handling and the capacity to work without the
[email protected]. requirement for other sort of
Pallapothu Balaji, Department of Computer Science and Engineering,
Sathyabama Institute of Science and Technology, Chennai, human info (or
[email protected].
S.Bhanumathi,School of Computing, Sathyabama Institute Of Science
and Technology, Chennai,[email protected].

Published By:
Retrieval Number: B12590982S1119/2019©BEIESP Blue Eyes Intelligence Engineering
DOI: 10.35940/ijrte.B1259.0982S1119 2315 & Sciences Publication
 ARTIFICIAL INTELLIGENCE BASED PERSON IDENTIFICATION VIRTUAL ASSISTANT

programming) may as of now be reasonable in the menial FACEDETECTION:

helpers however at some point it might fail because of Createadatabaseinwhichallthetrainingimagesarestored and
prepared voice engine to mirror his voice. all the images are captured by the camera are stored in
database. The face recognition is finished by using two
III. EXISTINGSYSTEM algorithmstobespecificHaarCascadeAlgorithmandFisher
Virtualassistancecandoanythingwithoutanindividualfor Face Algorithm. In the Haar Cascade algorithm is based on
help. VA can do administrative assistance like sending email Cascade classifiers which consists of haar features. The
and taking notes.VA can access the information form online cascade classifiers are the concatenation of a set of weak
website and gives information to the users. This information classifiers used to create a strong classifier. These classifiers
can be extracted in Jason and converted into text format this identify the human face dependent on the most significant
text format again converted into speech. This system will highlights like eyes, eyebrows and lips. Haar features are
interact with the user and fetched information form online detected based on the concrete computation, in which we
cloud according to the question asked from the user. voice allocate a pixel intensity to every single pixel related to
activateddigitaldeviceslike amazon choandpersonalAlexa grayscale values inside the scope of 0 to 255 where 0 speaks
assistant may also raise personal concerns like amazon is to the white shading and 255 speaks to the darkshading.
considering giving trance scripts Alexa's audio recordings to
the third-party. When they hear key word Alexa, they start
recording the voice which is not necessary. This recording
was stored in the developer's database. As artificial
intelligence is based on data and data is based on the
recordings of theusers.

IV. PROPOSEDSYSTEM
TheFacialRecognitiontothevirtualassistantgivesgreater
security to the system by the Haar Cascade algorithm. The Fig 4.1: original values detected on an image.
face is recognized by the camera and it is handled through
OpenCV in which it detects the faces and articles placed put
before the the cameraandthepicturestakeniscontrastedwith
the pictures in the in the system by Haar Cascade Algorithm.
After recognizing the face it permits to interact with the
virtualsystem.
The virtual assistant hangs tight for our voice commands.
Heretothevirtualassistanttheinputisgiveninspeechformat
throughBluetoothconnecteddevicethensystemconvertsthe
speech in to the text format and understands the input and
undergoes processing and gives the appropriate output in the
text format later it is converted to speech format as outputby Fig 4.2: Ideal Haar features and Pixel intensities 0 for
theuseofeSpeak(speechsynthesizer)andtheoutputisinthe white and 1 for black.
audio format.
FACERECOGNITION:

Fig 4.3 Person Identification VirtualAssistant

Published By:
Retrieval Number: B12590982S1119/2019©BEIESP Blue Eyes Intelligence Engineering
DOI: 10.35940/ijrte.B1259.0982S1119 2316 & Sciences Publication
 International Journal of Recent Technology and Engineering (IJRTE)
ISSN: 2277-3878, Volume-8, Issue-2S11, September 2019

Inthefig5.3cameracapturesthephotooftheuserandthen 1. Make an informational index envelope with the end goal

ittrainsthemodelsfortheimagesstoredinthedatabasewith that every one of the countenances to be perceived results in
respective to the features of the person. In the detection these present circumstancesorganizer.
sectionitdetectsthefacebasedonthehaarxmlfilewhichhas some 2. Instate the Hight and width of the face that is taken
pre-defined features of face. After detecting the face, it will while catching thepicture.
start extracting the features of the face. By considering the 3. Import the haar record utilizing openCV open source
features the images are classified accordingly. These application which has some pre-characterizedcapacities.
classified images are compared with the trained models. If it 4. OpenCv helps us to utilize webcam with the assistance
matches the user can access the virtualassistance. of VideoCapturefunction.Ontheoffchancethat0ispassedas
thecontentioninthiscapacityitwillutilizewebcamofthePC else
4.1.2 RECONIZING THE DETECTEDFSCE:
1 is passed which initiates different cameras which are
InFisherfaceAlgorithmthepictureswhichareavailablein c associated with thegadget.
number of classes with n samples for every individual dataset 5. To take n quantity of pictures loop the functions
and it is given as the example set of K's the place K ranges readfrom webcam and cvt.color which changes over shading
from 1 to c and every individual sample of T has the scope of to high
1 ton. contrastofcourseuseloopcommandinsidetheloopforeach and
K={K1,K2,… ,Kc}[11] every coordinate offace.
Ki={k1,k2,… ,kn}[11] 6. Inside this nested loop actualize factions of the OpenCV
To acquire the global mean of the considerable number of rectangle shape in which picture caught by the camera,
pictures and all the class this characterized by μ and μi is the arranges and shading range is given asarguments.
meanofpictureswithineachclass.Toacquirethechangeand 7. To detect the face use function
Ascertain the variance matrix which helps us to gives the face_cascde.detectMultiScale with gray and cascade values
scatter inside the class. For each class discover the contrast are given as arguments and use resize and image the write
between the picture and the mean esteem which is related to faction for resizing theimage.
class where the picture adjusted.
Sw=∑Cj=1∑nji=1(xij−μj)(xij−μj)T [11] FaceDetection
Wherexijistheithtestofclassj,μjisthemeanofclassj , and nj For Face detection we use Haar Cascade algorithm which
the quantity of tests in class j ,Sw is the scatter within theclass involves in the following steps:
The scattering between the classes is determinedutilizing 1.
Sb=∑Cj=1(μj−μ)(μj−μ)T[11] In the Haar Cascade algorithm is based on Cascade
Where Sb is the scatter between the classes. classifiers which consists of Haar features which are in Haar
We now need to discover those premise vectors V where file helps for the detection of theface.
Sw is limited and Sb is augmented, where V is a network 2. Thecascadeclassifiersarethecombinationofasetofweak
whose sections vi are the premise vectors characterizing the classifiers used to create a strongclassifier.
subspace. These are given by, 3. This combination forms a triangle shape which consists
|VTSbV||VTSwV| [11] of black and white identification lines onthe
The answer for this issue is given by the summed up eigen
Face is recognized using the fisher facealgorithm:
value decay.
This involves in two parts. They are Creating Fisher
SbV=SwVΛ, [11] Recognizer and to use Fisher Recognizer on the camera
stream.
V. IMPLEMENTATION Part1:Thisinvolvesintrainingtheimagesinwhicharestored
in the datasets. Create two lists which consist of images and
Necessary packages their corresponding names. Loop the image and their id for
● Bluetooth each and every subdirectory in the dataset. Then create a
● Time numpyarraycommonforboththelists.OpenCvhelpstotrain
● Httplib models for images with respective their id using train
function.
● urllib2
Part 2: In this the Fisher Recognizer helps to recognize the
● json
face on camera stream. First it will detect the face in front of
● subprocess thecamerausinghaarfile.Thedetectedfaceisconvertedinto
● cv2 black and white image. Use gray and resize functions for
● Sys modification of the
● numpy capturedimage.Thetrainedmodelgivesthepredictionvaluefort
● os hecaptured image the minimum limit of the prediction value
● subpocess is five
hundred.Ifthepredictionvalueislessthanfivehundredthen
CreateDatasets thepersonisauthenticatedelsethepersonisunauthenticated.
The accompanying advances will make the datasets this is
finished utilizing haar cascade frontal face default xml file
trained features of faces.

Published By:
Retrieval Number: B12590982S1119/2019©BEIESP Blue Eyes Intelligence Engineering
DOI: 10.35940/ijrte.B1259.0982S1119 2317 & Sciences Publication
 ARTIFICIAL INTELLIGENCE BASED PERSON IDENTIFICATION VIRTUAL ASSISTANT

Get Access to VoiceCommands authenticated and unauthenticated. If the person is

Ifthepersonisauthenticatedthenheorshewillgetaccessto unauthenticated it stops the process and get back to its initial
voicecommands. position where as in case of authenticated person it follows
the procedure flow. The speech given by the person is
converted into the text format which would be getting
processed in the next step. Then finally the processed text
isfollowedwiththeresponsegivenbythemoduletothespeech
form.Thisishowthefacerecognitionvirtualassistancetakes
place.

VI. RESULTS

Fig 5.1: Flow of Face Recognition Virtual Assistance. Fig 6.1 Face Input for Authentication The Virtual
Assistance.
This voice commands are revived via Bluetooth In the fig 5.3 cameras captures the photo of the user and
connection through a socket bind with port 1. The socket then it trains the models for the images stored in thedatabase
connection is automatically connected to the virtual assistant with respective to the features of the person. In the detection
if we add the device in the Bluetooth module of the virtual sectionitdetectsthefacebasedonthehaarxmlfilewhichhas some
assistant. The received data through attachment was pre-defined features of face. After detecting the face, it will
examined through conditions given and offer response to the start extracting the features of the face. By considering the
regarded individual. features the images are classified accordingly. These
For the appropriate responses that ought to be replayed is classified images are compared with the trained models. If it
finished by bringing in Wikipedia and wolfram alpha. matches the user can access the virtualassistance.
Wikipedia will give the responses for the general questions
and wolfram alpha will give responds to for the intelligent The input is taken in the form of picture as shown in the
inquiries. Utilizing voice commands, the virtual assistantcan figure 6.1. This input is processed with the trained models.
write notes and send email through the SMTP server. It There are usually three scenarios when it is processed with
likewise gives answers for climate, date, time and takes trained models namely:
photographs of thepeople. 1. authenticated person.
Flow Chart for face Recognition VirtualAssistance 2. Unauthenticated person.
3. thereisnofaceinfrontofthecamera’commentforthe third
case of not having input face at the cameravision.

Fig 6.2: Output for the detected Face and Replay for
Questions.

Incaseofauthenticatedpersonitwouldhavetheaccesstothe
virtualassistancethroughwhichwegivetheinputintheform
ofmethodologyusedinfig.5.1.Itaswellasgivetheresponse to
the questions asked by theperson.
Incaseofunauthenticatedpersonitleavesamessageandgoes
backtoitsrecognitionpartofitsloopinanykindofscenario.
Fig 5.2 Flow of Face Recognition Virtual Assistance.

In the flow chart of fig.5.1 the recognition part takes place

first which leaves two possibilities of a person getting

Published By:
Retrieval Number: B12590982S1119/2019©BEIESP Blue Eyes Intelligence Engineering
DOI: 10.35940/ijrte.B1259.0982S1119 2318 & Sciences Publication
 International Journal of Recent Technology and Engineering (IJRTE)
ISSN: 2277-3878, Volume-8, Issue-2S11, September 2019

3. Emad S. Othman in November 2017, "Voice Controlled

PersonalAssistantUsingRaspberryPi",InternationalJourna
l of Scientific & Engineering Research Volume 8, Issue
11, 1611, ISSN2229-5518.
4. Anurag Mishra, Pooja Makula, AkshayKumar,Krit Karan
and V. K. Mittal, May 28-30, 2015, "A Voice-Controlled
Personal Assistant Robot", IEEE journal, Page: 8,
INSPEC accession number 15291099.
Fig 6.3 Privacy and Security Issues Faced ByVirtual 5. Tatiana Ekeinhor-Komi, Jean-Léon Bouraoui, Romain
Assistants. Laroche, Fabrice Lefèvre, 09 February 2017, "Towards a
virtual personal assistant based on a user-defined
Thebarchatsinfig6.2frepresentstheprivacyandsecurity portfolioof multi-domain vocal applications", IEEE
issues faced by the virtual assistants. In the year 2017 forty Xplore, INSPEC 16657478.
6. O.Portillo-Rodriguez,C.A.Avizzano,A.Chavez-AguilarM
one percentage of the virtual assistance was used by third . Raspolli, S. Marcheschi and M. Bergamasco, "Haptic
party. The virtual assistance is always active results in Desktop: The Virtual Assistant Designer", IEEE/ASME
recordingsensitiveinformation.Someasthesecuritypurpose International Conference on Mechatronics and Embedded
government will listen to our private conversation all this Systems and Applications.
problems can be resolved using face recognition. This may 7. Bisma Shakeel, Tabasum and Mir Shahnawaz Ahmad in
solve by voice recognition but the attackers uses recoded July- 2017, "Siri - Apple's Personal Assistant: A Review",
International Journal of Computer Science and Mobile
mimic voice and get access for the virtual assistance. This Computing ol.6 Issue.7, ISSN2320-08.
show the need of security in the virtualassistant. 8. Sanchez, Carlos, Mun˜oz de la Pen˜a, David,
Gomez-EsternandFabio,"Virtualassistantforindividualize
Table 6.1 Represents Several Voice Assistants and d practical training on controller desig", ScienceDirect,
their features and How it differs from Proposed Model IFAC 48 29 2015 205210.
9. Prajyot Mane, Shubham Sonone, Nachiket Gaikwad and
Jyoti Ramteke, "Smart Personal Assistant using Machine
Learning", International Conference on Energy,
Communication, Data Analytics and
SoftComputing,ICECDS-2017.
10. Hyung-Ji Lee, Wan-Su Lee, and Jae-No Chung, 2001
published their paper "Face Recognition Using Fisherface
Algorithm And Elastic GRAPH Matching", IEEE
withISBN 0-7803-6725-1.
11. Sushma Jaiswal, Sarita Singh Bhadauria and Rakesh
In the table 6.1 represents the several voice assistants Singh Jadon, July 2011, "Comparison Between Face
Recognition Algorithm-Eigenfaces, Fisherfaces And
including proposed assistant accessibilities in the various
Elastic
fields. All the assistant’s response to the audio input whichis BunchGraphMatching",Volume2,No.7,JournalofGlobal
known as voice response, The Google Assistant and Siri can Research in Computer Science,ISSN-2229-371X.
be activated when the screen is locked whereas other 12. Dr.R.Subhashini, E.Nagarajan and Niveditha.P.R,
assistants lacks this feature. Proposed Assistant has Facial "Detection Of An Incognitos Intruder in Industries and
recognition that enables more security. Proposed system can Semantic Mapping Of Emotions", International Journal
Of Applied Engineering Research, ISSN-0973-4562,
sendmailinmoresecuredwaythantheothervoiceassistants.
VOLUME 9, Number 20(2014) pp.6727-6734.
13. R.Subhashini and V. Jawahar Senthil Kumar, "Shallow
VII. CONCLUSION NLP Techniques for Noun Phrase Extraction", Presented
This paper will help to demonstrate that virtual assistant is in the International Conference on Trendz in Information
attackedbythirdpartyandgivesasolutiontotheproblem.In this Sciences & Computing (TISC - 2010) in association with
Cognizant Technology Solutions and IEEE from 17th to
the virtual assistant is improvised with Facial Recognition. 19th of December, 2010, Sathyabama University,
The facial recognition system for virtual assistant using Chennai.
Machine learning strategies to detect and recognize faces.
After recognizing the face, the users can access the virtual AUTHORS PROFILE
assistant. If the person is third party ten it will say Penumarthi Pradeep, Department of Computer Science and
unauthenticated person. The unauthenticated person can get Engineering, Sathyabama Institute of Science and Technology, Chennai,
access to virtual assistance with administer permission. This [email protected].
Assistance can send email and take notesof some personal Pallapothu Balaji, Department of Computer Science and Engineering,
Sathyabama Institute of Science and Technology, Chennai,
information only for the authorized person it also says some [email protected].
general information like time, date and weather. S.Bhanumathi,School of Computing, Sathyabama Institute Of Science
and Technology, Chennai,[email protected].
REFERENCES
1. Efthimios and Constantino in 2017, "Monkey says,
Monkey Does-Security and Privacy on VoiceAssistants",
IEEE journal, ISSN: 2169-3536.
2. Peter Imrie and Peter M.Bednar, 2013, "VirtualPersonal
Assistant" in Research gate with ISBN:
978-88-6685-007-6 4.

Published By:
Retrieval Number: B12590982S1119/2019©BEIESP Blue Eyes Intelligence Engineering
DOI: 10.35940/ijrte.B1259.0982S1119 2319 & Sciences Publication