MVPS’s KBT College of Engineering

Name of Faculty: Mr. Bhushan S. Gholap

Department Computer Engineering

Maratha Vidya Prasarak Samaj’s

Karmaveer Adv. Baburao
Ganpatrao
Thakare College of Engineering
Nashik Maharashtra, India Enging
03-09-2020 MVPS’s KBT College of Engineering 1
 MVPS’s KBT College of Engineering

Subject Name:
410245(B)- Software Testing and Quality Assurance
Teaching Scheme: Examination Scheme:
TH: 03 Hours/Week In-Sem (Paper): 30 Marks
End-Sem (Paper): 70 Marks

Unit-III: Software Test Automation

03-09-2020 MVPS’s KBT College of Engineering 2
 MVPS’s KBT College of Engineering

Unit-III : Software Test Automation

CO3: Understand fundamental concepts of software automation

Syllabus:

What is Test Automation, Terms used in automation, Skills needed for automation, What to
automate, scope of automation, Design and Architecture of automation, Generic requirement for Test
Tool, Process Model for Automation, Selecting Test Tool, Automation for XP/Agile model, Challenges
in Automation, Data-driven Testing. Automation Tools like JUnit, Jmeter.
26-08-2020 MVPS’s KBT College of Engineering 3
 Software Test Automation..
Content
• What is Test Automation?
• Terms used in automation
• Skills needed for automation
• What to automate?
• Scope of automation
• Design and Architecture of automation
• Generic requirement for Test Tool
• Process Model for Automation
03-09-2020 MVPS’s KBT College of Engineering 4
 Software Test Automation
Content
• Selecting Test Tool
• Automation for XP/Agile model
• Challenges in Automation
• Data-driven Testing
• Automation Tools like JUnit, Jmeter

03-09-2020 MVPS’s KBT College of Engineering 5

 What is Test Automation?[1]..

• “Developing software to test the software is called Test

Automation”. Test Automation can help address several problems.
• Automation testing is a Software testing technique to test and
compare the actual outcome with the expected outcome. This can be
achieved by writing test scripts or using any automation testing tool.
• Test automation is used to automate repetitive tasks and other testing
tasks which are difficult to perform manually.
03-09-2020 MVPS’s KBT College of Engineering 6
 What is Test Automation?[1]..
• Automation saves time as software can execute test cases faster than
human do.
• The time thus saved can be used effectively for test engineers to,
1. Develop additional test cases to achieve better coverage;
2. Perform some esoteric or specialized test like ad hoc testing;
3. Perform some extra manual testing;

• Test automation can free the test engineers from mundane tasks and
make them focus on more creative tasks.

03-09-2020 MVPS’s KBT College of Engineering 7

 What is Test Automation?[1]
• Automation tests can be more reliable.
• Automation helps in immediate testing.
• Automation can protect an organization against attrition of test
engineers.
• Test automation opens up opportunities for better utilization of global
resources.
• Certain types of testing cannot be executed without automation.
• Automation means end-to-end, not test execution alone.
03-09-2020 MVPS’s KBT College of Engineering 8
 Which Test Cases to Automate?[3]
• Test cases to be automated can be selected using the following criterion to increase the
automation ROI
1. High Risk - Business Critical test cases
2. Test cases that are repeatedly executed
3. Test Cases that are very tedious or difficult to perform manually
4. Test Cases which are time-consuming
• The following category of test cases are not suitable for automation:
1. Test Cases that are newly designed and not executed manually at least once
2. Test Cases for which the requirements are frequently changing
3. Test cases which are executed on an ad-hoc basis.
03-09-2020 MVPS’s KBT College of Engineering 9
 Terms Used in Testing[1]..
Belongs to what type of
S No. Test Cases for Testing
testing

1 Check whether log in works Functionality

2 Repeat log in operation in a loop for 48 hours Reliability

3 Perform log in from 10000 clients Load/Stress Testing

Measure time taken for log in operations in

4 Performance
different conditions

Run log in operation from a machine running

5 Internationalization
Japanese language

03-09-2020 MVPS’s KBT College of Engineering 10

 Terms Used in Testing[1]

03-09-2020 MVPS’s KBT College of Engineering 11

 Automated Testing Process[3]

03-09-2020 MVPS’s KBT College of Engineering 12

 Skills Needed for Automation[1]..
• There are different “Generations of Automation”.
• The skills required for automation depends on what generation of
automation the company is in or desires to be in the nature.
• The automation of testing is broadly classified into three generations.
1. First Generation: Record and Playback
2. Second Generation: Data Driven
3. Third Generation: Action Driven

03-09-2020 MVPS’s KBT College of Engineering 13

 Skills Needed for Automation[1]
First Generation Second Generation Third Generation

Skills for test case Skills for test case Skills for test case Skills for framework
automation automation automation
Scripting Languages Scripting Languages Scripting Languages Programming Languages

Record-playback tools Programming Languages Programming Languages Design & Architecture Skills
usage for framework creation

Knowledge of data Design & Architecture of Generic test requirements

generation techniques the product under test for multiple products

Usage of the product under Usage of the framework

test

03-09-2020 MVPS’s KBT College of Engineering 14

 What to Automate[1]
• Some generic tips for identifying the scope for automation.
1. Identifying the Types of Testing Amenable to Automation
2. Automating Areas Less Prone to Change
3. Automate Tests that Pertain to Standards
4. Management Aspects in Automation

03-09-2020 MVPS’s KBT College of Engineering 15

 Design and Architecture for Automation[1]..

Components of Test Automation[1]

03-09-2020 MVPS’s KBT College of Engineering 16

 Design and Architecture for Automation[1]
1. External Modules
a. TCDB
b. Defect DB

2. Scenario and Configuration File Modules

3. Test Cases and Test Framework Modules
4. Tools and Results Modules
5. Report Generator and Reports/Metrics Modules

03-09-2020 MVPS’s KBT College of Engineering 17

 Generic Requirement for Test Tool / Framework[1]..
1. No hard coding in the test suite.
2. Test Case/Suite expandability.
3. Reuse of code for different types of testing, test cases.
4. Automate setup and cleanup.
5. Independent test cases.
6. Test case dependency.
7. Insulating test cases during execution.
8. Coding standards and directory structure.
9. Selective execution of test cases.
10. Random execution of test cases.
03-09-2020 MVPS’s KBT College of Engineering 18
 Generic Requirement for Test Tool / Framework[1]
11. Parallel execution of test cases.
12. Looping the test cases.
13. Grouping of test scenarios.
14. Test case execution based on previous results.
15. Remote execution of test cases.
16. Automatic archival of test data.
17. Reporting scheme.
18. Independent of languages.
19. Portability to different platforms.
03-09-2020 MVPS’s KBT College of Engineering 19
 Process Model for Automation[1]..

Fig: Similarities between product development and automation

03-09-2020 MVPS’s KBT College of Engineering 20
 Process Model for Automation[1]

Fig: W Model- Phases Involved in automation

03-09-2020 MVPS’s KBT College of Engineering 21
 Selecting a Test Tool
1. Free tools are not well supported and get phased out soon.
2. Developing in-house tools takes time.
3. Test tools sold by vendors are expensive.
4. Test tools require strong training.
5. Test tools generally do not meet all the requirements for
automation.
6. Not all test tools run on all platforms.
03-09-2020 MVPS’s KBT College of Engineering 22
 Criteria for Selecting Test Tools..
1. Meeting Requirements
2. Technology Expectations
3. Training/Skills
4. Management Aspects

03-09-2020 MVPS’s KBT College of Engineering 23

 Criteria for Selecting Test Tools
Meeting Requirements Technology Expectations Training Skills Management Aspects

Checking whether the tools Extending the test tool is Lack of trainers for test Test tools requires system
meet requirements, difficult tools upgrades
involves effort and money

Test tools are not fully Requires instrumented Test tools requires people Migration to other test
compatible with products. code to be removed for to learn new language / tools difficult.
certain tests. scripts.
Test tools are not tested Test tools are not cross- Deploying tool requires
with the same seriousness platform. huge planning and effort
as product for new
requirements.
Difficult to isolate problems
of product and test suite;
change in product causes
test suite to be changed.
03-09-2020 MVPS’s KBT College of Engineering 24
 Steps for Tool Selection and Deployment
● Steps to select and deploy a test tool,
a. Identify your test suite requirements among the generic requirements
discussed. Add other requirements.
b. Make sure experiences discussed in previous sections are taken care of.
c. Collect the experiences of other organizations which used similar test tools.
d. Keep a checklist of questions to be asked to the vendors on cost/effort/support.
e. Identify list of tools that meet the above requirements.
f. Evaluate and shortlist one/set of tools and train all test developers on the tool.
g. Deploy the tool across test teams after training all potential users of the tool.
03-09-2020 MVPS’s KBT College of Engineering 25
 Automation for XP (Extreme Programming) Model..
● One famous type of agile models is eXtreme Programming model;
it is called as XP.
● XP approach emphasizes on the development of the tests for the
product in the beginning itself.
● XP doesn’t expect detailed requirements documentation to begin
with.
● XP empowers programmers to easily respond to the fast changing
requirements from the customer.
03-09-2020 MVPS’s KBT College of Engineering 26
 Automation for XP (Extreme Programming) Model..
● In XP the customer, managers, programmers and testers act as
partners of a cohesive and collaborative team.
● XP involves iterative process expecting frequent re-prioritizing, re-
estimating and re-scheduling of activities.

03-09-2020 MVPS’s KBT College of Engineering 27

 Extreme Programming[3]

03-09-2020 MVPS’s KBT College of Engineering 28

 Automation for XP (Extreme Programming) Model
● Advantages of deploying XP in Testing:
a. Customer Satisfaction
b. Encourages early and frequent Testing
c. To make the process as efficient and effective
d. Tests be re-executed
e. Requires to design to be completed before coding has started

03-09-2020 MVPS’s KBT College of Engineering 29

 Challenges in Automation
1. Effective Communication and Collaboration
2. Selecting the Right Tool
3. Demanding Skilled Resources
4. Selecting a Proper Testing Approach
5. High Investment Cost

03-09-2020 MVPS’s KBT College of Engineering 30

 Data-Driven Testing[3]..
● Data Driven Testing is a software testing method in which test
data is stored in table or spreadsheet format. Data driven testing
allows testers to input a single test script that can execute tests
for all test data from a table and expect the test output in the
same table. It is also called table-driven testing or parameterized
● Data Driven Testing is important because testers frequently have
multiple data sets for a single test and creating individual tests for
each data set can be time-consuming.
03-09-2020 MVPS’s KBT College of Engineering 31
 Data-Driven Testing[3]

Fig: Flow Diagram Data Driven Testing

03-09-2020 MVPS’s KBT College of Engineering 32
 Junit[2]..
● Erich Gamma and Kent Beck initially develop it.
● JUnit is a unit testing framework for Java programming language
● It plays a crucial role test-driven development, and is a family of
unit testing frameworks collectively known as xUnit.
● It is useful for Java Developers to write and run repeatable tests.
● JUnit promotes the idea of "first testing then coding", which
emphasizes on setting up the test data for a piece of code that can
be tested first and then implemented.
03-09-2020 MVPS’s KBT College of Engineering 33
 Junit[2]
● This approach is like "test a little, code a little, test a little, code a
little." It increases the productivity of the programmer and the
stability of program code, which in turn reduces the stress on the
programmer and the time spent on debugging.

03-09-2020 MVPS’s KBT College of Engineering 34

 Features of Junit[2]..
● JUnit is an open source framework, which is used for writing and
running tests.
● Provides annotations to identify test methods.
● Provides assertions for testing expected results.
● Provides test runners for running tests.
● JUnit tests allow you to write codes faster, which increases
quality.
● JUnit is elegantly simple. It is less complex and takes less time.
03-09-2020 MVPS’s KBT College of Engineering 35
 Features of Junit[2]
● JUnit tests can be run automatically and they check their own
results and provide immediate feedback. There's no need to
manually comb through a report of test results.
● JUnit tests can be organized into test suites containing test cases
and even other test suites.
● JUnit shows test progress in a bar that is green if the test is
running smoothly, and it turns red when a test fails.

03-09-2020 MVPS’s KBT College of Engineering 36

 Jmeter[2]..
● JMeter is a software that can perform load test, performance-
oriented business (functional) test, regression test, etc., on
different protocols or technologies.
● Stefano Mazzocchi of the Apache Software Foundation was the
original developer of JMeter. He wrote it primarily to test the
performance of Apache JServ (now called as Apache Tomcat
project). Apache later redesigned JMeter to enhance the GUI and
to add functional testing capabilities.
03-09-2020 MVPS’s KBT College of Engineering 37
 Jmeter[2]..
● JMeter is a Java desktop application with a graphical interface that
uses the Swing graphical API. It can therefore run on any
environment / workstation that accepts a Java virtual machine, for
example − Windows, Linux, Mac, etc.

03-09-2020 MVPS’s KBT College of Engineering 38

 Jmeter[2]
● The protocols supported by JMeter are −
○ Web − HTTP, HTTPS sites 'web 1.0' web 2.0 (ajax, flex and flex-ws-amf)
○ Web Services − SOAP / XML-RPC
○ Database via JDBC drivers
○ Directory − LDAP
○ Messaging Oriented service via JMS
○ Service − POP3, IMAP, SMTP
○ FTP Service

03-09-2020 MVPS’s KBT College of Engineering 39

 Jmeter Features[2]..
● Being an open source software, it is freely available.
● It has a simple and intuitive GUI.
● JMeter can conduct load and performance test for many different
server types − Web - HTTP, HTTPS, SOAP, Database via JDBC,
LDAP, JMS, Mail - POP3, etc.
● It is a platform-independent tool. On Linux/Unix, JMeter can be
invoked by clicking on JMeter shell script. On Windows, it can be
invoked by starting the jmeter.bat file.
03-09-2020 MVPS’s KBT College of Engineering 40
 Jmeter Features[2]..
● It has full Swing and lightweight component support (precompiled
JAR uses packages javax.swing.* ).
● JMeter store its test plans in XML format. This means you can
generate a test plan using a text editor.
● Its full multi-threading framework allows concurrent sampling by
many threads and simultaneous sampling of different functions by
separate thread groups.

03-09-2020 MVPS’s KBT College of Engineering 41

 Jmeter Features[2]
● It is highly extensible.
● It can also be used to perform automated and functional testing of
the applications.

03-09-2020 MVPS’s KBT College of Engineering 42

 How JMeter Works?[2]

03-09-2020 MVPS’s KBT College of Engineering 43

 References
1. Srinivasan Desikan, Gopalswamy Ramesh, “Software Testing Principles and
Practices”, Pearson, ISBN-10: 817758121X
2. https://www.tutorialspoint.com/
3. https://www.guru99.com/

03-09-2020 MVPS’s KBT College of Engineering 44

 THANK
YOU
Maratha Vidya Prasarak Samaj’s Email id: [email protected]
Phone Number: 0253-2571439/ 0253-2582891
KBT College of Engineering Nashik College website: http://kbtcoe.org
Fax number: 0253-2317016
Udoji Maratha Boarding Campus, Gangapur Road,
Nashik-422013. Maharashtra India