Module 3

INTERNAL AND EXTERNAL VIEWS OF TESTING

Any engineered product can be tested in one of two ways:

(1) Knowing the specified function that a product has been designed to perform, tests can be conducted that
demonstrate each function is fully operational while at the same time searching for errors in each function.
The first test approach takes an external view and is called black-box testing.

(2) Knowing the internal workings of a product, tests can be conducted to ensure that “all gears mesh,” that
is, internal operations are performed according to specifications and all internal components have been
adequately exercised. The second requires an internal view and is termed white-box testing.

A: WHITE-BOX TESTING

White-box testing, sometimes called glass-box testing or structural testing, is a test-case design philosophy
that uses the control structure described as part of component-level design to derive test cases. Using white-
box testing methods, we can derive test cases that (1) guarantee that all independent paths within a module
have been exercised at least once, (2) exercise all logical decisions on their true and false sides, (3) execute
all loops at their boundaries and within their operational bounds, and (4) exercise internal data structures
to ensure their validity.

A1: BASIS PATH TESTING

Basis path testing is a white-box testing technique which enables the test-case designer to derive a logical
complexity measure of a procedural design and use this measure as a guide for defining a basis set of
execution paths. Test cases derived to exercise the basis set are guaranteed to execute every statement in
the program at least one time during testing.

A.1.1 Flow Graph Notation

The flow graph depicts logical control flow using the notation illustrated in Figure
To illustrate the use of a flow graph, consider the procedural design representation in Figure 23.2a . Here, a
flowchart is used to depict program control structure.

 Figure 23.2bmaps the flowchart into a corresponding flow

 Referring to Figure 23.2b, each circle, called a flow graph node, represents one or more procedural
statements.
 A sequence of process boxes and a decision diamond can map into a single node.The arrows on the
w graph, called edges or links, represent flow of control and are analogous to flowchart arrows.
 An edge must terminate at a node, even if the node does not represent any procedural statements
(e.g., see the flow graph symbol for the if-then-else construct)
 Areas bounded by edges and nodes are called regions. When counting regions, we include the area
outside the graph as a region.

When compound conditions are encountered in a procedural design, the generation of a flow graph
becomes slightly more complicated. A compound condition occurs when one or more Boolean operators
(logical OR, AND, NAND, NOR) is present in a conditional statement. Referring to Figure, the program
design language (PDL) segment translates into the flow graph shown. Note that a separate node is
created for each of the conditions a and b in the statement IF a OR b. Each node that contains a condition
is called a predicate node and is characterized by two or more edges emanating from it.

A.1.2 Independent Program Paths

An independent path is any path through the program that introduces at least one new set of processing
statements or a new condition. When stated in terms of a flow graph, an independent path must move
along at least one edge that has not been traversed before the path is defined. For example, a set of
independent paths for the flow graph illustrated in Figure 23.2 b is not considered to be an independent
path because it is simply a combination of already specified paths and does not traverse any new edges.
Path 1: 1-11
Path 2: 1-2-3-4-5-10-1-11
Path 3: 1-2-3-6-8-9-10-1-11
Path 4: 1-2-3-6-7-9-10-1-11
Note that each new path introduces a new edge.
The path 1-2-3-4-5-10-1-2-3-6-8-9-10-1-11
 Paths 1 through 4 constitute a basis set for the flow graph in Figure 23.2b . That is, if you can design tests
to force execution of these paths (a basis set), every statement in the program will have been guaranteed
to be executed at least one time and every condition will have been executed on its true and false sides.
It should be noted that the basis set is not unique. In fact, a number of different basis sets can be derived
for a given procedural design.
Cyclomatic complexity is a software metric that provides a quantitative measure of the logical complexity
of a program. When used in the context of the basis path testing method, the value computed for
cyclomatic complexity defines the number of independent paths in the basis set of a program and
provides you with an upper bound for the number of tests that must be conducted to ensure that all
statements have been executed at least once.
Cyclomatic complexity has a foundation in graph theory and provides you with an extremely useful
software metric. Complexity is computed in one of three ways:
1. The number of regions of the flow graph corresponds to the cyclomatic complexity.
2. Cyclomatic complexity V( G) for a flow graph G is defined as V( G) =E-N+ 2 where E is the number of
flow graph edges and N is the number of flow graph nodes.
3. Cyclomatic complexity V( G) for a flow graph G is also defined as V( G) =P+1
where P is the number of predicate nodes contained in the flow graph G.
Referring once more to the flow graph in Figure 23.2b , the cyclomatic complexity can be computed
using each of the algorithms just noted:
1. The flow graph has four regions.
2. V( G) = 11 edges - 9 nodes + 2 = 4.
3. V( G) = 3 predicate nodes + 1 = 4.
Therefore, the cyclomatic complexity of the flow graph in Figure 23.2b is 4.
More important, the value for V( G) provides you with an upper bound for the number of independent paths
that form the basis set and, by implication, an upper bound on the number of tests that must be designed
and executed to guarantee coverage of all program statements
A.1.3 Graph Matrices
A data structure, called a graph matrix, can be quite useful for developing a software tool that assists in basis
path testing.
A graph matrix is a square matrix whose size (i.e., number of rows and columns) is equal to the number of
nodes on the flow graph. Each row and column corresponds to an identified node, and matrix entries
correspond to connections (an edge) between nodes. A simple example of a flow graph and its corresponding
graph matrix Figure below.
Referring to the figure, each node on the flow graph is identified by numbers, while each edge is identified
by letters. A letter entry is made in the matrix to correspond to a connection between two nodes. For
example, node 3 is connected to node 4 by edge b.
To this point, the graph matrix is nothing more than a tabular representation of a flow graph. However, by
adding a link weight to each matrix entry, the graph matrix can become a powerful tool for evaluating
program control structure during testing. The link weight provides additional information about control flow.
In its simplest form, the link weight is 1 (a connection exists) or 0 (a connection does not exist). But link
weights can be assigned other, more interesting
properties:
• The probability that a link (edge) will be executed.
• The processing time expended during traversal of a link
• The memory required during traversal of a link
• The resources required during traversal of a link.

B : BLACK-BOX TESTING
Black-box testing, also called behavioral testing or functional testing focuses on the functional requirements
of the software. That is, black-box testing techniques enable you to derive sets of input conditions that will
fully exercise all functional requirements for a program

 Black-box testing attempts to find errors in the following categories: (1) incorrect or missing
functions, (2) interface errors, (3) errors in data structures or external database access, (4) behavior
or performance errors, and (5) initialization and termination errors.
 Black-box testing is focused on the information domain. Black-box tests are designed to validate
functional requirements without regard to the internal workings of a program.
 Black-box testing techniques focus on the information domain of the software,deriving test cases by
partitioning the input and output domain of a program in a manner that provides thorough test
coverage.

 Equivalence partitioning divides the input domain into classes of data that are likely to exercise a
specific software function.
 Boundary value analysis probes the program’s ability to handle data at the limits of acceptability.
 Orthogonal array testing provides an efficient, systematic method for testing systems with small
numbers of input parameters.
 Model-based testing uses elements of the requirements model to test the behavior of an application.

1 Graph-Based Testing Methods

The first step in black-box testing is to understand the objects 5 that are modeled in software and the
relationships that connect these objects. Once this has been accomplished, the next step is to define a series
of tests that verify “all objects have the expected relationship to one another” . Stated in another way,
software testing begins by creating a graph of important objects and their relationships and then devising a
series of tests that will cover the graph so that each object and relationship is exercised and errors are
uncovered.
To accomplish these steps, you begin by creating a graph—a collection of nodes that represent objects, links
that represent the relationships between objects, node weights that describe the properties of a node (e.g.,
a specific data value or state behavior), and link weights that describe some characteristic of a link.
The symbolic representation of a graph is shown in Figure a . Nodes are represented as circles connected by
links that take a number of different forms. A directed link (represented by an arrow) indicates that a
relationship moves in only one direction. A bidirectional link, also called a symmetric link, implies that the
relationship applies in both directions. Parallel links are used when a number of different relationships are
established between graph nodes.
As a simple example, consider a portion of a graph for a word-processing application (Figure 23.8b ) where
Object #1 = newFile (menu selection)
Object #2 = documentWindow
Object #3 = documentText
Referring to the figure, a menu select on newFile generates a document window. The node weight of
documentWindow provides a list of the window attributes that are to be expected when the window is
generated. The link weight indicates that the window must be generated in less than 1.0 second. An
undirected link establishes a symmetric relationship between the newFile menu selection and
documentText, and parallel links indicate relationships between documentWindow and documentText.
we can then derive test cases by traversing the graph and covering each of the relationships shown. These
test cases are designed in an attempt to find errors in any of the relationships. Beizer describes a number of
behavioral testing methods that can make use of graphs:
Transaction flow modeling. The nodes represent steps in some transaction (e.g., the steps required to make
an airline reservation using an online service), and the links represent the logical connection between steps
Finite state modeling. The nodes represent different user-observable states of the software (e.g., each of
the “screens” that appear as an order entry clerk takes a phone order), and the links represent the transitions
that occur to move from state to state input).
Data flow modeling. The nodes are data objects, and the links are the transformations that occur to translate
one data object into another
Timing modeling. The nodes are program objects, and the links are the sequential connections between
those objects. Link weights are used to specify the required execution times as the program executes.
2 Equivalence Partitioning
Equivalence partitioning is a black-box testing method that divides the input domain of a program into classes
of data from which test cases can be derived.
Test-case design for equivalence partitioning is based on an evaluation of equivalence classes for an input
condition. Using if a set of objects can be linked by relationships that are symmetric, transitive, and reflexive,
an equivalence class is present .
An equivalence class represents a set of valid or invalid states for input conditions. Typically, an input
condition is either a specific numeric value, a range of values, a set of related values, or a Boolean condition.
Equivalence classes may be defined according to the following guidelines:
1. If an input condition specifices a range, one valid and two invalid equivalence classes are defined.
2. If an input condition requires a specific value, one valid and two invalid equivalence classes are defined.
3. If an input condition specifies a member of a set, one valid and one invalid equivalence class are defined.
4. If an input condition is Boolean, one valid and one invalid class are defined.
By applying the guidelines for the derivation of equivalence classes, test cases for each input domain data
item can be developed and executed. Test cases are selected so that the largest number of attributes of an
equivalence class are exercised at once.
3 Boundary Value Analysis
A greater number of errors occurs at the boundaries of the input domain rather than in the “center.” It is for
this reason that boundary value analysis (BVA) has been developed as a testing technique.

 Boundary value analysis leads to a selection of test cases that exercise bounding values.
 Boundary value analysis is a test-case design technique that complements equivalence partitioning.
Rather than selecting any element of an equivalence class, BVA leads to the selection of test cases
at the “edges” of the class.
Guidelines for BVA are similar in many respects to those provided for equivalence partitioning:
1. If an input condition specifies a range bounded by values a and b, test cases should be designed with values
a and b and just above and just below a and b.
2. If an input condition specifies a number of values, test cases should be developed that exercise the
minimum and maximum numbers. Values just above and below minimum and maximum are also tested.
3. Apply guidelines 1 and 2 to output conditions. For example, assume that a temperature versus pressure
table is required as output from an engineering analysis program. Test cases should be designed to create
an output report that produces the maximum (and minimum) allowable number of table entries.
4. If internal program data structures have prescribed boundaries (e.g., a table has a defined limit of 100
entries), be certain to design a test case to exercise the data structure at its boundary.
4 Orthogonal Array Testing
Orthogonal array testing can be applied to problems in which the input domain is relativel small but too large
to accommodate exhaustive testing.
The orthogonal array testing method is particularly useful in finding region faults—an error category
associated with faulty logic within a software component.
To illustrate the difference between orthogonal array testing and more conventional “one input item at a
time” approaches, consider a system that has three input items, X, Y, and Z.
Each of these input items has three discrete values associated with it. There are 33 = 27 possible test cases.
Phadke suggests a geometric view of the possible test cases associated with X, Y, and Z illustrated in Figure .
Referring to the figure, one input item at a time may be varied in sequence along each input axis. This results
in relatively limited coverage of the input domain (represented by the left-hand cube in the figure).

When orthogonal array testing occurs, an L9 orthogonal array of test cases is created. The L9 orthogonal
array has a “balancing property”. That is, test cases (represented by dark dots in the figure) are “dispersed
uniformly throughout the test domain,” as illustrated in the right-hand cube in Figure . Test coverage across
the input domain is more complete.
To illustrate the use of the L9 orthogonal array, consider the send function for a fax application. Four
parameters, P1, P2, P3, and P4, are passed to the send function. Each takes on three discrete values. For
example, P1 takes on values:
P1 = 1, send it now
P1 = 2, send it one hour later
P1 = 3, send it after midnight
P2, P3, and P4 would also take on values of 1, 2 and 3, signifying other send functions.
If a “one input item at a time” testing strategy were chosen, the following sequence of tests
(P1, P2, P3, P4) would be specified: (1, 1, 1, 1), (2, 1, 1, 1), (3, 1, 1, 1), (1, 2, 1, 1), (1, 3, 1, 1), (1, 1, 2, 1), (1, 1,
3, 1), (1, 1, 1, 2), and (1, 1, 1, 3). But these would uncover only single mode faults
[Pha97], that is, faults that are triggered by a single parameter.
Given the relatively small number of input parameters and discrete values, exhaustive testing is possible. The
number of tests required is 3 4 5 81, large but manageable. All faults associated with data item permutation
would be found, but the effort required is relatively high.
The orthogonal array testing approach enables you to provide good test coverage with far fewer test cases
than the exhaustive strategy. An L9 orthogonal array for the fax send function is illustrated in Figure .
Phadke assesses the result of tests using the L9 orthogonal array in the following manner:
Detect and isolate all single mode faults. A single mode fault is a consistent problem with any level of =any
single parameter. For example, if all test cases of factor P1 1 cause an error condition, it is a single mode
failure. In this example tests 1, 2 and 3 [ Figure 23.10 ] will show errors. By analyzing the information about
which tests show errors, one can identify which parameter values cause the fault. In this example, by noting
that tests 1, 2, and 3 cause an error, one can isolate [logical processing associated with “send it now” (P1 =
1)] as the source of the error. Such an isolation of fault is important to fix the fault.
Detect all double mode faults. If there exists a consistent problem when specific levels of two parameters
occur together, it is called a double mode fault. Indeed, a double mode fault is an indication of pair wise
incompatibility or harmful interactions between two test parameters.
Multimode faults. Orthogonal arrays [of the type shown] can assure the detection of only single and double
mode faults. However, many multi-mode faults are also detected by these tests.

Differences between Black Box Testing vs White Box Testing:

Black Box Testing White Box Testing
It is a way of software testing in which It is a way of testing the software in
the internal structure or the program or which the tester has knowledge about
the code is hidden and nothing is known the internal structure or the code or the
about it. program of the software.
Implementation of code is not needed Code implementation is necessary for
for black box testing. white box testing.
It is mostly done by software
It is mostly done by software testers. developers.
No knowledge of implementation is Knowledge of implementation is
needed. required.
It can be referred to as outer or external It is the inner or the internal software
software testing. testing.
It is a functional test of the software. It is a structural test of the software.
This testing can be initiated based on This type of testing of software is
the requirement specifications started after a detail design document.
document.
It is the behavior testing of the
software. It is the logic testing of the software.
No knowledge of programming is It is mandatory to have knowledge of
required. programming.
It is applicable to the higher levels of It is generally applicable to the lower
testing of software. levels of software testing.
It is also called closed testing. It is also called as clear box testing.
It is least time consuming.
In Black box testing, time consumption It is most time consuming.
depends upon the availability of the It takes a long time to design test cases
functional specifications. due to lengthy code.
It is not suitable or preferred for
algorithm testing. It is suitable for algorithm testing.
Testing method : Can be done by trial Testing method : Data domains along
and error ways and methods. with inner or internal boundaries can be
better tested.
Example: Search something on google Example: By input to check and verify
by using keywords loops
Black-box test design techniques-
 Decision table testing White-box test design techniques-
 All-pairs testing  Control flow testing
 Equivalence partitioning  Data flow testing
 Error guessing  Branch testing

Types of Black Box Testing: Types of White Box Testing:

 Functional Testing  Path Testing
 Non-functional testing  Loop Testing
 Regression Testing  Condition testing
It is less exhaustive as compared to It is comparatively more exhaustive
white box testing. than black box testing.
Errors : It does not find the errors Errors : In white-box testing, there is
related to the code. the detection of hidden errors. It also
helps to optimize the code.