Lect 11 12 Testing
Lect 11 12 Testing
Lect 11 12 Testing
2
Real life examples
First U.S. space mission to Venus failed. (reason:
missing comma in a Fortran do loop)
3
Testing Strategies
SW testing is one element of a broader topic referred to as
V&V (many SQA activities).
Verification (correctly implement specific function)
Defect detection and correction
Comparison between implementation and the
corresponding specification
Are we building the product right?
Validation (SW is traceable to customer requirements)
Defect prevention
Provision of sound basis for specific design
decisions
Are we building the right product?
Example: Checking the printing of receipts is verification and
correct printing (info) is validation.
4
Types/Strategies of testing
Code Inspections
Software (Module) Testing
Unit Testing
Functional Testing
Integration Testing
Compliance
Interoperability Testing
System Testing
Recovery
Security
Load / Stress
Acceptance Testing
Stress, Security, performance, Load Testing etc
5
Software Testing
Objectives (different for each type)
(1) it does what it should ---- requirements conformance
(2) it does it well and efficiently ------ performance
(3) it does so without error ------ errors
errors--correctness
(4) And so on ------ an indication of quality
Carefully designed and coded program - less chances of
errors but some always exist.
To some people: ”A process of checking a program to show
there are no errors”- implies that a successful testing is one
that shows no error.
This a wrong approach. The aim of testing is to find errors
so that they can be corrected.
6
Software Testing
A better definition: ”the process of running software with
the intent of finding an error”.
7
Three Levels of Correctness
Possible correctness
Obtaining correct output for some arbitrary input (single
set).
If the outcome is wrong, the program cannot possibly be
correct.
For example a multiply program: for input 2 and 3, if
result is 6; is possibly correct.
Probable correctness
Obtaining correct output for a number of carefully selected inputs.
If all potential problematic areas are checked in this way,
the program is probably correct.
Try several values, including the obvious “problem” values
such as zero, largest negative, 1, -1 and so on.
8
Absolute correctness
can be demonstrated only by a test that involves every
possible combination of inputs.
Requires huge amount of time; it is therefore not practical
{However for some programs we can prove correctness
mathematically}.
Imagine the same above test for a 32 bit machine (how
many combinations are possible? (Hundreds of Millions).
9
Testability
Which S/W is testable? Characteristics of a testable S/W.
Operability —it operates cleanly
Observability —the results of each test case are
readily observed
Controllability —the degree to which testing can
be automated and optimized
Decomposability —testing can be targeted
Simplicity —reduce complex architecture and
logic to simplify tests
Stability —few changes are requested during
testing
Understandability —of the design
10
Test case "Bugs lurk in corners
and congregate at
design boundaries ..."
Boris Beizer
OBJECTIVE ---- to uncover errors
CRITERIA ----- In a complete manner
CONSTRAINT ------ with a minimum of effort and time
FURTHER
Logical errors and incorrect assumptions are inversely
proportional to the probability that a program path will
be executed.
Sometime a path executes counterintuitive.
Typographical errors are random.
Also: if black box testing finds error, locating it is easier
with additional white box testing!
12
Exhaustive Testing
loop 20
loop 20
13
White-Box Testing
16
Basis Path Testing
Sequence If
While (for)
Case
Until (do-while)
17
Nodes, Edges, And Regions
18
Flowchart And Flow Graph
Node
1
1 Edge
2,3
2
6 R2 4,5
3 7 R3 8
R1
6 4 9
7 8 5
10 Region
9
10
11 R4
11
19
Cyclomatic Complexity
Cyclomatic complexity provides a quantitative
measure of the logical complexity of a program
– It is the number of tests that must be conducted
to assure that all statements have been executed
at least once
Cyclomatic complexity, V(G) is given by: 1 Node
Edge
(1) V(G) = E - N + 2 2,3
6 R2 4,5
where E is the number of edges and N
7 R3 8
number of nodes (11-9 +2 = 4) 9 R1
(2) V(G) = P + 1, where P is the number
of predicate nodes (3+1 = 4) 10 Region
(3) V(G) = number of regions (4)
11 R4
20
Independent Paths
The value for cyclomatic complexity defines the
number of independent paths in the basis set of a
program
An independent path is any path through the
program that introduces at least one new set of
processing statements or a new condition
An independent path must move along at least one
edge that has not been traversed before the path is
defined
A set of independent paths for a flow graph
composes a basis set.
21
Deriving Test Cases
Using the design or code, draw a
corresponding flow graph
you don't need a flow chart,
1 Node
Determine the cyclomatic complexity (even Edge
without developing flow graph: count each 2,3
simple logical test (compound tests as 2 or
6 4,5
more) + 1 R
7 R3 8 2
9 R1
Determine a basis set of paths
Prepare test cases that will force execution of 10 Region
paths in the basis set
11 R4
basis path testing should be applied to
critical modules
Exercise: Read and understand the example (Figure 17.4 Pressman5th –450, 6th – Fig14.4
22
Basis Path Testing
1
derive the
independent paths:
2 Since V(G) = 4,
there are four paths
3
4
5 6 Path 1: 1,2,3,6,7,8
Path 2: 1,2,3,5,7,8
Path 3: 1,2,4,7,8
7
Path 4: 1,2,4,7,2,4,...7,8
8
derive test cases to exercise
these paths.
paths.
23
Control Structure Testing: Loop
Nested
Loops
Simple Concatenated
loop Unstructured
Loops Loops
24
Loop Testing: Simple Loops
Minimum conditions—
conditions—Simple Loops
25
Loop Testing: Nested Loops
Nested Loops
(1) Start at the innermost loop. Set all outer loops to their minimum
iteration parameter values.
(2) Test the min+1, typical, max-1 and max for the innermost loop (while
holding the outer loops at their minimum values).
(3) Move out one loop and set it up as in step 2, holding all other loops at
typical values.
Continue this step until the outermost loop has been tested.
Concatenated Loops
If the loops are independent of one another
then treat each as a simple loop
else
treat as nested* loops
endif
26
Theoretical Foundation of Testing
• Mathematical model: Let P be a program, and let D and R denote its
input (Data) and output (Result) ranges. That is, D is a set of all data that
can correctly be supplied to P, and the results of P’s execution, if any, are
elements of R.
TEST EXAMPLE : Statement Coverage Criteria: We can use above model
to define test as: Select a test set T such that, by executing P for each d (subset
of D) in T, each elementary statement of P is executed at least once.
Test Case Design Examples
Number of test cases in a test does not necessarily contribute: To test this
incorrect program
If x > y Then
max = x;
else
max = x;
endif;
The test case set [x =3, y= 2; x= 2, y= 3] is able to detect the error. Whereas
Test case set [x=3, y = 2; x =4, y = 3, x = 5, y = 1] is not, although it
contains more test cases.
27
Black-Box Testing
requirements
output
input events
28
Black Box Testing
Types of errors regarding functional requirements of
software:
-- Incorrect or missing functions
-- Interface errors
-- Error in data structure & external data base access
-- Performance errors
-- Initialization & termination errors
No functional requirements NO Black Box Testing.
Demonstrates that each function is fully operational.
Uncovers different kind of errors than white box testing.
Performed later in the testing process.
- 29
Black box techniques derive a set of test cases
that satisfy the following criteria:
(1) Test cases reduce the number of
additional test cases that must be designed to
achieve reasonable testing
(2) Test cases that tell us something about the
presence or absence of classes of errors,
rather than errors associated only with the
specific test at hand.
Black box techniques can supplement the test
cases generated by white box.
30
How to Design Test Cases?
31
Equivalence Class Partitioning
Two considerations:
1. each test case should invoke as many different input
conditions as possible in order to minimize the total
number of test cases necessary.
2. one should try to partition the input domain of a program
into a finite number of equivalence classes.
33
Identifying Equivalence classes
A key concept in the identification of classes is
negation, i.e. If a characteristic is identified as an
equivalence class, then one should immediately
negate the characteristic in order to find
examples of classes which should cause the
module to do something different such as
“generate an error message”.
Partitioning each input condition into two or
more groups.
Two types of equivalence classes are identified:
1. Valid Equivalence Classes
Invalid data
data outside bounds of the program
physically impossible data
proper value supplied in wrong place
35
Defining equivalence classes
37
Characteristic Valid equivalence Invalid equivalence
class class
First char must be alphabetic Letter Non-letter
40
Testing Phase
What content should be included in a software test plan?
41
Test Execution
using manual approach
using an automatic approach
using a semi-automatic approach
Basic activities in test execution:
Select a test case
Set up the pre-conditions for a test case
Set up test data
Run a test case following its procedure
Track the test operations and results
Monitor the post-conditions of a test case & expected results
Verify the test results and report the problems if there is any
Record each test execution
- 42
Example: Windows Calculator
R-001:
The users should be able to add two numbers and view their result on the
display.
Use Case: UC01 Add Two Numbers
Actors: User
Purpose: Add two numbers and view their result
Overview: The user inputs two numbers and (then adds them and)
checks the result, displayed on the screen.
Type: Primary, Real
Cross References: R-001
2. The actor input the first number by clicking on the 3. The digit is displayed on the screen.
keypad or using keyboard.
4. The actor clicks or presses the “+” key.
5.The actor then adds the second number as (2). 6. The pressed digit is displayed on the screen.
7. The actor clicks the “=” key. 8. The sum of the two digits is displayed on the screen.
43
Test Case
Test Case ID: T-101 Test Item: Add Numbers
Wrote By: (tester name) Junaid Documented Date: 26th April 2005
Test Type: Manual Test Suite#: NA
Product Name: Windows Calculator Release and Version No.: V 1.0
Test case description:
Add any two Numbers
Operation procedure:
Open Calculator
Press “1”
Press “+”
Press “2”
Press “=”
Pre-conditions: Post-conditions:
Calculator Opened Result Displayed
Inputs data and/or events: Expected output data and/or events:
1+2= 3
Required test scripts (for auto): NA
Cross References: (Requirements or Use Cases) R-001, UC-01
44
Bug Report
Problem ID: B-101 current software name: Windows Calculator
Release and Version No.: V 1.0
Test type: Manual Reported by: Junaid
Reported date: 26th April 2005 Test case ID: T-101
Subsystem (or module name): Calculation Feature Name (or
Subject): Add Numbers
Problem type (REQ, Design, Coding,): Coding
Problem severity (Fatal, Major, Minor,): Major
Problem summary and detailed description:
On adding two numbers the result is not correct.
Cause analysis: NA
How to reproduce? Why Required:
Attachments: Nil
45