Project Title: Bachelor of Technology IN Computer Science and Engineering
Project Title: Bachelor of Technology IN Computer Science and Engineering
Project Title: Bachelor of Technology IN Computer Science and Engineering
Page | 1
Mini Project submitted in partial fulfillment of the requirement for the award of the
degree of
BACHELOR OF TECHNOLOGY
IN
By
P.Mahesh Reddy(15R11A05M9)
S.Kishore Kumar(15R111A05P0)
T.Subhashith Reddy(15R11A05P2)
CERTIFICATE
This is to certify that the B.Tech Main Project report entitled “SECURITY
EVALUATION AND DESIGN PATTERNS UNDER OCCURENCE” is a bonafide
work done by P.MAHESH REDDY(15R11A05M9), S.KISHORE KUMAR
(15R11A05P0), T.SUBASHITH REDDY(15R11A05P2), in partial fulfillment of the
requirement of the award for the degree of Bachelor of Technology in “Computer Science
and Engineering” from Jawaharlal Nehru Technological University, Hyderabad during the
year 2016-2017.
Internal Guide
Mr.ShivaNarayana Reddy
Geethanjali College of Engineering & Technology
(UGC Autonomous)
(Affiliated to JNTUH Approved by AICTE,New Delhi)
Cheeryal (V), Keesara(M), R.R.Dist.-501 301.
Page | 3
DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING
Accredited by NBA
and Engineering, Geethanjali College of Engineering and Technology, is submitted in partial fulfillment
of the requirements for the award of the degree of Bachelor of Technology in Computer Science and
Engineering.
This is a record of bonafide work carried out by me/us in [cyberaegis It solutions] and the results
embodied in this project have not been reproduced or copied from any source. The results embodied in
this project report have not been submitted to any other University or Institute for the award of any other
degree or diploma.
P.Mahesh,15R11A05M9
S.kishore,15R11A05P0
T.Subhashith,15R11A05P2
Department of CSE
Cheeryal.
Page | 4
ABSTRACT
Pattern classification systems are commonly used in adversarial applications, like
biometric authentication, network intrusion detection, and spam filtering, in which
data can be purposely manipulated by humans to undermine their operation. As
Page | 5
this adversarial scenario is not taken into account by classical design methods,
pattern classification systems may exhibit vulnerabilities, whose exploitation may
severely affect their performance, and consequently limit their practical utility.
Extending pattern classification theory and design methods to adversarial settings
is thus a novel and very relevant research direction, which has not yet been pursued
in a systematic way. In this paper, we address one of the main open issues:
evaluating at design phase the security of pattern classifiers, namely, the
performance degradation under potential attacks they may incur during operation.
We propose a framework for empirical evaluation of classifier security that
formalizes and generalizes the main ideas proposed in the literature, and give
examples of its use in three real applications. Reported results show that security
evaluation can provide a more complete understanding of the classifier’s behaviour
in adversarial environments, and lead to better design choices
Page | 6
TABLE OF CONTENTS
1.2 Objective
2. System Analysis………………………………………………………….
2.2.1 Details
2.2.2 Ethics
2.2.3 Cost
3. Literature Overview……………………………………………….
It should include the info of the technology and its tools on which your
project is run.
4. System Design…………………………………………………………
6. Testing………………………………………………………………………………..
6.1 Testing
7. Output Screens………………………………………………………………………
8. Conclusion…………………………………………………………………………….
8.1 Conclusion
9. Bibliography………………………………………………………………………….
Page | 8
10 Appendices
A. SW used
B. Methodologies used
Etc.,
11 Plagiarism Report
INSTRUCTIONS TO BE FOLLOWED
The project report should be submitted in A4 size. Number of copies to be submitted:
Three (One for Library, One for department, One for Internal Guide). The
certificate should consist of names and roll numbers of all batch members for the above
three copies.
One copy for each batch member. The certificate should consist of batch member name
and his/her roll number.
Bond paper (A4 size) should be used for the preparation of the project report. Typing
should be done on one side of the paper with character font in size 12 of Times New
Roman.
The layout should provide a margin of 4 cm on the left, 3 cm on the top and bottom and 2
cm on the right.
i.e. Left margin space 4 cm
Page | 11
biometric authentication, network intrusion detection, and spam filtering, in which
data can be purposely manipulated by humans to undermine their operation. As
this adversarial scenario is not considered by classical design methods, pattern
classification systems may exhibit vulnerabilities, whose exploitation may severely
affect their performance, and consequently limit their practical utility. Extending
pattern classification theory and design methods to adversarial settings is thus a
novel and very relevant research direction, which has not yet been pursued in a
systematic way. In this paper, we address one of the main open issues: evaluating
at design phase the security of pattern classifiers, namely, the performance
degradation under potential attacks they may incur during operation. We propose a
Page | 12
framework for empirical evaluation of classifier security that formalizes and
generalizes the main ideas proposed in the literature, and give examples of its use
in three real applications. Reported results show that security evaluation can
Page | 13 provide a more complete understanding of the classifier’s behaviour in adversarial
environments, and lead to better design choices
SYSTEM ANALYSIS :
EXISTING SYSTEM:
Page | 14 Pattern classification systems based on classical theory and design methods do not
consider adversarial settings; they exhibit vulnerabilities to several potential
attacks, allowing adversaries to undermine their effectiveness. A systematic and
unified treatment of this issue is thus needed to allow the trusted adoption of
pattern classifiers in adversarial environments, starting from the theoretical
foundations up to novel design methods, extending the classical design cycle of. In
particular, three main open issues can be identified: (i) analyze the vulnerabilities
of classification algorithms, and the corresponding attacks. (ii) Developing novel
methods to assess classifier security against these attacks, which are not possible
using classical performance evaluation methods. (iii) Developing novel design
methods to guarantee classifier security in adversarial environments.
PROPOSED SYSTEM:
In this work we address issues above by developing a framework for the empirical
evaluation of classifier security at design phase that extends the model selection
and performance evaluation steps of the classical design cycle .We summarize
previous work, and point out three main ideas that emerge from it. We then
formalize and generalize them in our framework. First, to pursue security in the
context of an arms race it is not sufficient to react to observed attacks, but it is also
necessary to proactively anticipate the adversary by predicting the most relevant,
potential attacks through a what-if analysis; this allows one to develop suitable
countermeasures before the attack actually occurs, according to the principle of
security by design. Second, to provide practical guidelines for simulating realistic
attack scenarios, we define a general model of the adversary, in terms of her goal,
knowledge, and capability, which encompass and generalize models proposed in
Page | 15 previous work. Third, since the presence of carefully targeted attacks may affect
the distribution of training and testing data separately, we propose a model of the
data distribution that can formally characterize this behaviour, and that allows us to
take into account a large number of potential attacks; we also propose an algorithm
for the generation of training and testing sets to be used for security evaluation,
which can naturally accommodate application-specific and heuristic techniques for
simulating attacks.
ETHICS:
The aspect of study is to check the level of acceptance of the system by the
user. This includes the process of training the user to use the system efficiently.
The user must not feel threatened by the system, instead must accept it as a
necessity. The level of acceptance by the users solely depends on the methods that
are employed to educate the user about the system and to make him familiar with
it. His level of confidence must be raised so that he is also able to make some
Page | 16 constructive criticism, which is welcomed, as he is the final user of the system.
COST:
This study is carried out to check the economic impact that the system will have
on the organization. The amount of fund that the company can pour into the
research and development of the system is limited. The expenditures must be
justified. Thus the developed system as well within the budget and this was
achieved because most of the technologies used are freely available. Only the
customized products had to be purchased.
MODULES:
MODULES DESCRIPTION:
Attack Scenario and Model of the Adversary:
Pattern Classification:
Multimodal biometric systems for personal identity recognition have received great
interest in the past few years. It has been shown that combining information
coming from different biometric traits can overcome the limits and the weaknesses
inherent in every individual biometric, resulting in a higher accuracy. Moreover, it
is commonly believed that multimodal systems also improve security against
Spoofing attacks, which consist of claiming a false identity and submitting at least
one fake biometric trait to the system (e.g., a “gummy” fingerprint or a photograph
of a user’s face). The reason is that, to evade multimodal system, one expects that
the adversary should spoof all the corresponding biometric traits. In this
application example, we show how the designer of a multimodal system can verify
if this hypothesis holds, before deploying the system, by simulating spoofing
attacks against each of the matchers.
Adversarial classification:
Assume that a classifier has to discriminate between legitimate and spam emails on
the basis of their textual content, and that the bag-of-words feature representation
has been chosen, with binary features denoting the occurrence of a given set of
Page | 18 words
Security modules:
Intrusion detection systems analyze network traffic to prevent and detect malicious
activities like intrusion attempts, ROC curves of the considered multimodal
biometric system under a simulated spoof attack against the fingerprint or the face
matcher. Port scans, and denial-of-service attacks. When suspected malicious
traffic is detected, an alarm is raised by the IDS and subsequently handled by the
system administrator. Two main kinds of IDSs exist: misuse detectors and
anomaly-based ones. Misuse detectors match the analyzed network traffic against a
database of signatures of known malicious activities. The main drawback is that
they are not able to detect never-before-seen malicious activities, or even variants
of known ones. To overcome this issue, anomaly-based detectors have been
proposed. They build a statistical model of the normal traffic using machine
learning techniques, usually one-class classifiers, and raise an alarm when
anomalous traffic is detected. Their training set is constructed, and periodically
updated to follow the changes of normal traffic, by collecting unsupervised
network traffic during operation, assuming that it is normal (it can be filtered by a
misuse detector, and should)
SYSTEM CONFIGURATION:
HARDWARE REQUIREMENTS:
SOFTWARE REQUIREMENTS:
Java Technology
Page | 20
Simple
Architecture neutral
Object oriented
Portable
Distributed
High performance
Interpreted
Multithreaded
Robust
Dynamic
Secure
You can think of Java byte codes as the machine code instructions for the
Java Virtual Machine (Java VM). Every Java interpreter, whether it’s a
development tool or a Web browser that can run applets, is an implementation of
the Java VM. Java byte codes help make “write once, run anywhere” possible. You
can compile your program into byte codes on any platform that has a Java
compiler. The byte codes can then be run on any implementation of the Java VM.
That means that as long as a computer has a Java VM, the same program written in
the Java programming language can run on Windows 2000, a Solaris workstation,
or on an iMac.
Page | 22
Native code is code that after you compile it, the compiled code runs
on a specific hardware platform. As a platform-independent environment,
the Java platform can be a bit slower than native code. However, smart
compilers, well-tuned interpreters, and just-in-time byte code compilers can
bring performance close to that of native code without threatening
portability.
ODBC
Microsoft Open Database Connectivity (ODBC) is a standard programming
interface for application developers and database systems providers. Before ODBC
Page | 24 became a de facto standard for Windows programs to interface with database
systems, programmers had to use proprietary languages for each database they
wanted to connect to. Now, ODBC has made the choice of the database system
almost irrelevant from a coding perspective, which is as it should be. Application
developers have much more important things to worry about than the syntax that is
needed to port their program from one database to another when business needs
suddenly change.
The ODBC system files are not installed on your system by Windows
95. Rather, they are installed when you setup a separate database application, such
as SQL Server Client or Visual Basic 4.0. When the ODBC icon is installed in
Control Panel, it uses a file called ODBCINST.DLL. It is also possible to
administer your ODBC data sources through a stand-alone program called
ODBCADM.EXE. There is a 16-bit and a 32-bit version of this program and each
maintains a separate list of ODBC data sources.
From a programming perspective, the beauty of ODBC is that the
application can be written to use the same set of function calls to interface with any
data source, regardless of the database vendor. The source code of the application
Page | 25 doesn’t change whether it talks to Oracle or SQL Server. We only mention these
two as an example. There are ODBC drivers available for several dozen popular
database systems. Even Excel spreadsheets and plain text files can be turned into
data sources. The operating system uses the Registry information written by
ODBC Administrator to determine which low-level ODBC drivers are needed to
talk to the data source (such as the interface to Oracle or SQL Server). The loading
of the ODBC drivers is transparent to the ODBC application program. In a
client/server environment, the ODBC API even handles many of the network
issues for the application programmer.
The advantages of this scheme are so numerous that you are probably
thinking there must be some catch. The only disadvantage of ODBC is that it isn’t
as efficient as talking directly to the native database interface. ODBC has had
many detractors make the charge that it is too slow. Microsoft has always claimed
that the critical factor in performance is the quality of the driver software that is
used. In our humble opinion, this is true. The availability of good ODBC drivers
has improved a great deal recently. And anyway, the criticism about performance
is somewhat analogous to those who said that compilers would never match the
speed of pure assembly language. Maybe not, but the compiler (or ODBC) gives
you the opportunity to write cleaner programs, which means you finish sooner.
Meanwhile, computers get faster every year.
JDBC
In an effort to set an independent database standard API for Java; Sun
Microsystems developed Java Database Connectivity, or JDBC. JDBC offers a
generic SQL database access mechanism that provides a consistent interface to a
variety of RDBMSs. This consistent interface is achieved through the use of “plug-
in” database connectivity modules, or drivers. If a database vendor wishes to have
Page | 26 JDBC support, he or she must provide the driver for each platform that the
database and Java run on.
To gain a wider acceptance of JDBC, Sun based JDBC’s framework on
ODBC. As you discovered earlier in this chapter, ODBC has widespread support
on a variety of platforms. Basing JDBC on ODBC will allow vendors to bring
JDBC drivers to market much faster than developing a completely new
connectivity solution.
JDBC was announced in March of 1996. It was released for a 90 day public
review that ended June 8, 1996. Because of user input, the final JDBC v1.0
specification was released soon after.
The remainder of this section will cover enough information about JDBC for you
to know what it is about and how to use it effectively. This is by no means a
complete overview of JDBC. That would fill an entire book.
Web Container
Tomcat is a web container, when a request is made from Client to web server, it
passes the request to web container and it’s web container job to find the correct
resource to handle the request (servlet or JSP) and then use the response from the
resource to generate the response and provide it to web server. Then web server
sends the response back to the client.
When web container gets the request and if it’s for servlet then container creates
two Objects HTTPServletRequest and HTTPServletResponse. Then it finds the
correct servlet based on the URL and creates a thread for the request. Then it
invokes the servlet service() method and based on the HTTP method service()
method invokes doGet() or doPost() methods. Servlet methods generate the
dynamic page and write it to response. Once servlet thread is complete, container
converts the response to HTTP response and send it back to client.
Page | 29 Some of the important work done by web container are:
Communication Support – Container provides easy way of communication
between web server and the servlets and JSPs. Because of container, we
don’t need to build a server socket to listen for any request from web server,
parse the request and generate response. All these important and complex
tasks are done by container and all we need to focus is on our business logic
for our applications.
Lifecycle and Resource Management – Container takes care of managing the life
cycle of servlet. Container takes care of loading the servlets into memory,
initializing servlets, invoking servlet methods and destroying them.
Container also provides utility like JNDI for resource pooling and
management.
Multithreading Support – Container creates new thread for every request to the
servlet and when it’s processed the thread dies. So servlets are not initialized
for each request and saves time and memory.
JSP Support – JSPs doesn’t look like normal java classes and web container
provides support for JSP. Every JSP in the application is compiled by
container and converted to Servlet and then container manages them like
other servlets.
Miscellaneous Task – Web container manages the resource pool, does memory
optimizations, run garbage collector, provides security configurations,
support for multiple applications, hot deployment and several other tasks
behind the scene that makes our life easier.
MySQL:
Page | 30 MySQL, the most popular Open Source SQL database management system, is
developed, distributed, and supported by Oracle Corporation.
The MySQL Web site (http://www.mysql.com/) provides the latest information
about MySQL software.
A relational database stores data in separate tables rather than putting all the
data in one big storeroom. The database structures are organized into
physical files optimized for speed. The logical model, with objects such as
databases, tables, views, rows, and columns, offers a flexible programming
environment. You set up rules governing the relationships between different
data fields, such as one-to-one, one-to-many, unique, required or optional,
and “pointers” between different tables. The database enforces these rules,
so that with a well-designed database, your application never sees
inconsistent, duplicate, orphan, out-of-date, or missing data.
The SQL part of “MySQL” stands for “Structured Query Language”. SQL is
the most common standardized language used to access databases.
Depending on your programming environment, you might enter SQL
Page | 31 directly (for example, to generate reports), embed SQL statements into code
written in another language, or use a language-specific API that hides the
SQL syntax.
SQL is defined by the ANSI/ISO SQL Standard. The SQL standard has been
evolving since 1986 and several versions exist. In this manual, “SQL-92”
refers to the standard released in 1992, “SQL:1999” refers to the standard
released in 1999, and “SQL:2003” refers to the current version of the
standard. We use the phrase “the SQL standard” to mean the current version
of the SQL Standard at any time.
Open Source means that it is possible for anyone to use and modify the
software. Anybody can download the MySQL software from the Internet
and use it without paying anything. If you wish, you may study the source
code and change it to suit your needs. The MySQL software uses the GPL
(GNU General Public License), http://www.fsf.org/licenses/, to define what
you may and may not do with the software in different situations. If you feel
uncomfortable with the GPL or need to embed MySQL code into a
commercial application, you can buy a commercially licensed version from
us. See the MySQL Licensing Overview for more information
(http://www.mysql.com/company/legal/licensing/).
The MySQL Database Server is very fast, reliable, scalable, and easy to use.
If that is what you are looking for, you should give it a try. MySQL Server
can run comfortably on a desktop or laptop, alongside your other
applications, web servers, and so on, requiring little or no attention. If you
Page | 32 dedicate an entire machine to MySQL, you can adjust the settings to take
advantage of all the memory, CPU power, and I/O capacity available.
MySQL can also scale up to clusters of machines, networked together.
You can find a performance comparison of MySQL Server with other
database managers on our benchmark page.
The official way to pronounce “MySQL” is “My Ess Que Ell” (not “my sequel”),
but we do not mind if you pronounce it as “my sequel” or in some other localized
way.
SYSTEM DESIGN
SYSTEM ARCHITECTURE:
Page | 34
DATA FLOW DIAGRAM:
Page | 35
1. The DFD is also called as bubble chart. It is a simple graphical formalism
that can be used to represent a system in terms of input data to the system,
various processing carried out on this data, and the output data is generated
by this system.
2. The data flow diagram (DFD) is one of the most important modeling tools. It
is used to model the system components. These components are the system
process, the data used by the process, an external entity that interacts with
the system and the information flows in the system.
3. DFD shows how the information moves through the system and how it is
modified by a series of transformations. It is a graphical technique that
depicts information flow and the transformations that are applied as data
moves from input to output.
4. DFD is also known as bubble chart. A DFD may be used to represent a
system at any level of abstraction. DFD may be partitioned into levels that
represent increasing information flow and functional detail.
Login
Page | 36
Check
Biometric Authendication
Admin
user
view attackers
download file
UML DIAGRAMS
GOALS:
The Primary goals in the design of the UML are as follows:
1. Provide users a ready-to-use, expressive visual modeling Language so that
they can develop and exchange meaningful models.
2. Provide extendibility and specialization mechanisms to extend the core
concepts.
3. Be independent of particular programming languages and development
process.
4. Provide a formal basis for understanding the modeling language.
Page | 38 5. Encourage the growth of OO tools market.
6. Support higher level development concepts such as collaborations,
frameworks, patterns and components.
7. Integrate best practices.
USE CASE DIAGRAM:
Login
user
upload file
download file
view attackers
CLASS DIAGRAM:
In software engineering, a class diagram in the Unified Modeling Language
(UML) is a type of static structure diagram that describes the structure of a system
by showing the system's classes, their attributes, operations (or methods), and the
Page | 41 relationships among the classes. It explains which class contains information.
Admin
User
Attacker;
Register
newusers
Login
blockeduser
File
Filetype
Spamfileter
uploadfile()
activate();
download()
activities();
spamfilter()
viewattacker():
SEQUENCE DIAGRAM:
A sequence diagram in Unified Modeling Language (UML) is a kind of interaction
diagram that shows how processes operate with one another and in what order. It is
a construct of a Message Sequence Chart. Sequence diagrams are sometimes called
Page | 42 event diagrams, event scenarios, and timing diagrams.
Data base
Page | 43 User Admin
Registration
activate user
upload file
download file
find attacker
ACTIVITY DIAGRAM:
Activity diagrams are graphical representations of workflows of stepwise activities
and actions with support for choice, iteration and concurrency. In the Unified
Modeling Language, activity diagrams can be used to describe the business and
Page | 44 operational step-by-step workflows of components in a system. An activity
diagram shows the overall flow of control.
Login
Page | 45
Activate users
user
yes
apply spam filter
Download file
A
A
CONCLUSION
In this paper we focused on empirical security evaluation of pattern classifiers that
have to be deployed in adversarial environments, and proposed how to revise the
Page | 46
classical performance evaluation design step, which is not suitable for this purpose.
Page | 47
REFERENCES
[1] R.N. Rodrigues, L.L. Ling, and V. Govindaraju, “Robustness of Multimodal
Page | 48 Biometric Fusion Methods against Spoof Attacks,” J. Visual Languages and
Computing, vol. 20, no. 3, pp. 169-179, 2009.
[4] G.L. Wittel and S.F. Wu, “On Attacking Statistical Spam Filters,” Proc. First
Conf. Email and Anti-Spam, 2004.
[5] D. Lowd and C. Meek, “Good Word Attacks on Statistical Spam Filters,” Proc.
Second Conf. Email and Anti-Spam, 2005.
[6] A. Kolcz and C.H. Teo, “Feature Weighting for Improved Classifier
Robustness,” Proc. Sixth Conf. Email and Anti-Spam, 2009.
[11] M. Barreno, B. Nelson, R. Sears, A.D. Joseph, and J.D. Tygar, “Can Machine
Learning be Secure?” Proc. ACM Symp. Information, Computer and Comm.
Security (ASIACCS), pp. 16-25, 2006.
Page | 50 [17] P. Laskov and M. Kloft, “A Framework for Quantitative Security Analysis of
Machine Learning,” Proc. Second ACM Workshop Security and Artificial
Intelligence, pp. 1-4, 2009.
[23] A. Globerson and S.T. Roweis, “Nightmare at Test Time: Robust Learning by
Feature Deletion,” Proc. 23rd Int’l Conf. Machine Learning, pp. 353-360, 2006.
[24] R. Perdisci, G. Gu, and W. Lee, “Using an Ensemble of One-Class SVM
Classifiers to Harden Payload-Based Anomaly Detection Systems,” Proc. Int’l
Page | 51 Conf. Data Mining, pp. 488-498, 2006.
[25] S.P. Chung and A.K. Mok, “Advanced Allergy attacks: Does a Corpus Really
Help,” Proc. 10th Int’l Conf. Recent Advances in Intrusion Detection (RAID ’07),
pp. 236-255, 2007.
[27] G.F. Cretu, A. Stavrou, M.E. Locasto, S.J. Stolfo, and A.D. Keromytis,
“Casting out Demons: Sanitizing Training Data for Anomaly Sensors,” Proc. IEEE
Symp. Security and Privacy, pp. 81-95, 2008.
[28] B. Nelson, M. Barreno, F.J. Chi, A.D. Joseph, B.I.P. Rubinstein, U. Saini, C.
Sutton, J.D. Tygar, and K. Xia, “Exploiting Machine Learning to Subvert Your
Spam Filter,” Proc. First Workshop Large- Scale Exploits and Emergent Threats,
pp. 1-9, 2008.
[29] B.I. Rubinstein, B. Nelson, L. Huang, A.D. Joseph, S.-h. Lau, S. Rao, N. Taft,
and J.D. Tygar, “Antidote: Understanding and Defending against Poisoning of
Anomaly Detectors,” Proc. Ninth ACM SIGCOMM Internet Measurement Conf.
(IMC ’09), pp. 1-14, 2009.
[30] M. Kloft and P. Laskov, “Online Anomaly Detection under Adversarial
Impact,” Proc. 13th Int’l Conf. Artificial Intelligence and Statistics, pp. 405-412,
2010.
Page | 52
Page | 53
ABSTRACT
Pattern classification systems are commonly used in adversarial applications, like
biometric authentication, network intrusion detection, and spam filtering, in which
data can be purposely manipulated by humans to undermine their operation. As
Page | 54
this adversarial scenario is not taken into account by classical design methods,
pattern classification systems may exhibit vulnerabilities, whose exploitation may
severely affect their performance, and consequently limit their practical utility.
Extending pattern classification theory and design methods to adversarial settings
is thus a novel and very relevant research direction, which has not yet been pursued
in a systematic way. In this paper, we address one of the main open issues:
evaluating at design phase the security of pattern classifiers, namely, the
performance degradation under potential attacks they may incur during operation.
We propose a framework for empirical evaluation of classifier security that
formalizes and generalizes the main ideas proposed in the literature, and give
examples of its use in three real applications. Reported results show that security
evaluation can provide a more complete understanding of the classifier’s behaviour
in adversarial environments, and lead to better design choices
SCREENSHOTS:
Page | 55
Page | 56
Page | 57
Page | 58