Deep Features Creation For Smile Classification in Biometric Systems

DEEP FEATURES CREATION FOR SMILE
CLASSIFICATION IN BIOMETRIC SYSTEMS
Boris Assanovich, Yury Veretilo

YK State University of Grodno
National Anti-Doping Laboratory
Grodno, Minsk, Republic of Belarus
E-mail: [email protected], [email protected]
A new biometric imprint was proposed, which is obtained from smile video using stacked autoencoder that allows
to build a biometric cryptosystem. The performance of the proposed system based on deep face features has been
verified with the use of posed smiles dataset from UvA-NEMO database.
Introduction in ML sparse autoencoders are used. It is done

usually by adding a regularizer to the cost function.
Recently, the analysis of human emotions
This regularizer is a function of the average
has attracted more and more researchers. Often
output activation value of a neuron and encourages
a person’s smile is a key factor not only in
the autoencoder to learn a representation, where
determining the person’s psychological state, but
each neuron in the hidden layer <<fires>> to a
also one of the main elements in the implementation
small number of training examples. That makes to
of a human-machine interface (HMI). In recent
respond of neurons to some feature that is only
years, facial biometry has become one of the
present in a small subset of the training examples.
most preferred biometric methods both in video
In our system we applied so-called stacked
surveillance and in digital banking [1].
autoencoder (SAE) that is a neural network
To ensure high accuracy of recognition and
including several layers of sparse autoencoders
resistance to various attacks (cropping, spoofing,
where output of each hidden layer is connected to
etc.), a number of authors switched to the analysis
the input of the successive hidden layer. In this case
of the dynamic characteristics of a person’s face or
the hidden layers are trained in an unsupervised
to soft biometrics. Recently Murat Taskiran et all
way and then fine-tuned by a supervised method.
[2] have applied dynamic face features extraction
The use of SAE will allow the use of the
from videos and used them for face recognition.
trained data of the output layer of the neural
Authors performed the the segmentation of smile
network as features for biometric separation of users
phases from video and extraction of the main
into genuine and imposters (persons who pretends
face landmarks allowed to recognize face with
to be somebody else). The effectiveness of user
the equal error rate EER = 30% (threshold
comparisons depends on similarity rates, which are
2.8) for the spontaneous smile and with EER =
often determined by the distribution of root mean
21 : 25% (threshold 2:39) for the posed one
square (RMS) distances of their characteristics.
with the use of 648 feature vector. Different
The more the two distributions are separated
techniques use facial dynamics to identify a
and the smaller the standard deviation for each
person with the use of various spatio-temporal
distribution, the better the separation of the
parameters of face. Classical linear methods of
classified classes. This property of distributions is
image processing and feature extraction based on
estimated by such a parameter as decidability index
principal components analysis (PСА) are replaced
by non-linear transformations. Compared with |µg − µi |
PCA, the use of an autoencoder significantly DI = q , (1)
increases the classification accuracy. (σg2 + σi2 )/2
In this paper, we will consider the use of
where µg , µi and σg , σi are the means and
auto-encoders to extract soft biometric data from a
standard deviations of genuine and imposter
person’s smile and use them for authentication and
distributions.
identification to provide access to digital services.
In addition to the decidability index, an
The article provides the analysis of
equal error rate (ERR), which is the error rate
classification efficiency of soft deep face deep
at which a false accept rate (FAR) is equal to a
features and demonstrates the creation of a digital
false rejection rate (FRR), is normally used as a
smiling face imprint to provide a secure biometric
measure of biometric system verification accuracy.
interface.
In biometrics FAR is the rate at which an imposter
print is incorrectly accepted as genuine and FRR
I. AUTOENCODERS
is the rate at which a genuine print is incorrectly
Actually an autoencoder is the network that is rejected as imposter.
forced to learn to select the most important features Then, a comparison of users by their feature
from data [3]. To exclude the process of overfitting representation can be performed by the evaluation
1
of a receiver operator characteristic (ROC) curve (511,28,111). The results showed a superposition of
that demonstrates how a genuine acceptance rate intraclass and intraclass distributions on each other
(GAR) determining a correctly received genuine and significant classification errors.
fingerprint depends on a change in FAR. A series of experiments were performed with
The use of autoencoders will make it possible SAE to get good compact biometric features. To
to compactly present biometric features and reduce time spent, in these experiments the subsets
then apply the scheme to organize HCI, where of 40 subjects randomly selected from the entire
instead of tokens and passwords, a biometric key UvA-NEMO Database were used, reproducing a
can be exploited for authentication. Biometric posed smile. Then normalized grayscale images
authentication is the process of establishing from corresponding video of 112x112 pixels in size,
user identity by measurements of his biological scaled to 50%, creating a vector length of the
characteristics. To handle the variability inherent in input layer of 6272 elements have been used for
biometric authentication, it is necessary to create unsupervised learning of SAE.
and store a template for each user. To create it, The training of the SAE layers consisted of 2
one can use the fuzzy commitment scheme with stages:
application of error correcting codes (ECC) [4]. 1. Unsupervised training on a 6x40 = 240 frames
from training dataset, containing 3 phases of
II. PROPOSED SYSTEM a smile and a neutral state;
As soon as autoencoders are widely used in 2. Supervised training, when 4 additional video
many applications many researches have tested frames were added to SAE input from the
their efficiency. The experiments of face recognition control dataset for tuning testing.
based on SVM and softmax classifiers have shown Soft-Max classifier was used for test
that in recognition of 51 classes autoencoder identification.
achieved the error rate 9.53% while PCA 13.5% [5]. To assess the quality of training, on the basis
Inspired by a possible improvement in the of latent layer data y, such values as FRR, FAR,
classification characteristics, we used an auto- GAR, ERR and DI were calculated and the ROC-
encoder to obtain biometric data on a person’s smile characteristic was monitored, as well as the values
and bind them to a secure user key. According to of MSE for controlling the intraclass and interclass
our knowledge, the use of auto-encoders to create a distribution. The results of SAE training and tuning
biometric imprint from face dynamics the context through the corresponding number of iterations are
of implementing BC and linking the secret key to a shown in Fig.1, and the histograms of the obtained
human smile have not been considered before. distributions in Fig.2.
Thus, the user smile imprint captured from
ones video can serve as the biometric key to organize
the access to different external digital services.
III. RESULTS AND CONCLUSIONS

In order to perform capturing images and
video, features extraction and processing, we
developed the FaceAnalyzer App (FAA) that has
been applied to select and encode face HOG
features, extract and normalize frames related to
Fig. 1 – ROCs
three main phases of video smile, calculate the
statistical characteristic of biometric features.
The UvA-NEMO Smile Database has been
used for experiments. It contained videos of
spontaneous and deliberate smiles collected from
400 subjects.
Using FAA, video was analyzed with a
person’s smile and its three phases:
1. onset (neutral to expressive) phase;
2. apex;
3. offset phase (expressive to neutral), describing
Fig. 2 – DIs
the state of a smiling person’s face.
To compare the opportunities of binary and The experiments based on of UvaNemo
non-binary ECC by analogy with our earlier and Caltech datasets performed have shown the
work [4], we extracted 4464-dimensional vectors improvement in the accuracy of classification
from selected video smile frames, calculated their genuine smile from posed one by 10%, as well as a
reliability and applied a 511-element mask for reduction in the complexity of biometric templates
subsequent encoding with BCH codes (511,58,91) design.
2
In this work, an analysis of the autoencoders Intelligent SysTems and Applications (INISTA), Sofia,
application to obtain trained biometric features to 3-5 July 2019. 1–6.
construct the smile imprint for user authentication 3. Assanovich B. AutoEncoders for Denoising and
Classication Applications. OSTIS 2020, М., 2020. –
has been performed. The main difference between Вып. 4. – С. 309-312
the previous paper [6] is in the analysis of the 4. Assanovich, B., Veretilo Yu. Biometric Database Based
learning parameters of SAE and the assessment on HOG Structures and BCH Codes. In Proc.
of deep features for solving the problems of face Information Techn. and Syst. BSUIR, –2017 – P. 286–
classification based on the smile imprint. 287.
5. Siwek, K., Osowski, S. (2017) Autoencoder versus PCA
in face recognition. In: 18th International Conference
1. Cook, S. (2017) Selfie banking: is it a reality? Biometric on Computational Problems of Electrical Engineering
Technology Today, 3 (March). 9–11. (CPEE). Kutna Hora, 11-13 September 2017. 1–4.
2. Taskiran, M. et al. (2019) Face Recognition Using 6. Assanovich B., Bich N., Pronevich. A. Smile Biometric
Dynamic Features Extracted from Smile Videos. In: Imprint Creation with the Use of Autoencoder. ТСЗИ.
IEEE International Symposium on INnovations in – М., 2020. – P. 8.

Deep Features Creation For Smile Classification in Biometric Systems

Uploaded by

Copyright:

Available Formats

Deep Features Creation For Smile Classification in Biometric Systems

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Deep Features Creation For Smile Classification in Biometric Systems

Uploaded by

Copyright:

Available Formats

DEEP FEATURES CREATION FOR SMILE

CLASSIFICATION IN BIOMETRIC SYSTEMS

Boris Assanovich, Yury Veretilo

Introduction in ML sparse autoencoders are used. It is done

III. RESULTS AND CONCLUSIONS

You might also like