STQA Lecture Slides 01
STQA Lecture Slides 01
STQA Lecture Slides 01
2
Outline
https://www.triad.co.uk/news/testing-vs-qa
• Introduction
• Why is Testing Necessary?
• Testing Principles
• Test Activities and Tasks
3
Introduction
• ST involves validation
• Validation: Checking whether system meets users’ and other stakeholders’
needs in its operational environment
7
Dynamic Testing and Static Testing
• Work product being tested, test level, risks, software development lifecycle
being followed, and factors related to business context
13
Typical Debugging Process
• Reproduction of a failure
• Diagnosis (finding the root cause)
• Fixing the cause
14
Confirmation and Regression Testing
15
Debugging through Static Testing
16
Why is Testing Necessary?
17
Testing’s Contributions to Success, 1/2
• While people often use terms “testing” and “quality assurance” (QA)
interchangeably, testing and QA are not the same
• Testing is a form of quality control (QC)
20
Testing and Quality Assurance, 2/3
21
Testing and Quality Assurance, 3/3
• Human beings make errors (mistakes), which produce defects (faults, bugs),
which in turn may result in failures
• Humans make errors for various reasons, such as time pressure, complexity
of work products, processes, infrastructure or interactions, or simply
because they are tired or lack adequate training
23
Errors, Defects, Failures, and Root Causes, 2/4
25
Errors, Defects, Failures, and Root Causes, 4/4
26
Testing Principles
• Testing can show that defects are present in the test object, but cannot
prove that there are no defects
• Testing reduces the probability of defects remaining undiscovered in the
test object, but even if no defects are found, testing cannot prove test
object correctness
28
Exhaustive Testing is Impossible
29
Early Testing Saves Time and Money
• Defects that are removed early in the process will not cause subsequent
defects in derived work products
• Cost of quality reduced since fewer failures occur later in SDLC
• To find defects early, both static testing and dynamic testing should be
started as early as possible
30
Defects Cluster Together
31
Tests Wear Out
• If the same tests are repeated many times, they become increasingly
ineffective in detecting new defects
• To overcome this effect, existing tests and test data may need to be
modified, and new tests may need to be written
• However, in some cases, repeating the same tests can have a beneficial
outcome, e.g., in automated regression testing, to assess change impact
32
Testing is Context Dependent
33
Absence-of-Defects Fallacy
• Testing is context dependent, but, at a high level, there are common sets of
test activities without which testing is less likely to achieve test objectives
• These sets of test activities form a test process
• Test process can be tailored to a given situation based on various factors
• Which test activities are included in this test process, how they are
implemented, and when they occur is normally decided as part of the test
planning for the specific situation
35
Test Activities, Testware, and Test Roles, 2/2
• General aspects of test process are described in terms of test activities and tasks,
impact of context, testware, traceability between test basis and testware, and
testing roles
• Standard ISO/IEC/IEEE 29119-2 provides information about test processes
• Test process usually consists of main groups of activities
• Although many of these activities may appear to follow a logical sequence, they are
often implemented iteratively or in parallel
• Testing activities usually need to be tailored to the system and the project
36
Test Activities and Tasks
• Test planning
• Test monitoring and control
• Test analysis
• Test design
• Test implementation
• Test execution
• Test completion
37
Test Planning
• Test planning consists of defining the test objectives and then selecting an
approach that best achieves the objectives within the constraints imposed
by the overall context
38
Test Monitoring and Control
• Test monitoring involves the ongoing checking of all test activities and the
comparison of actual progress against the plan
• Test control involves taking actions necessary to meet objectives of testing
39
Test Analysis
• Test analysis includes analyzing test basis to identify testable features and to
define and prioritize associated test conditions, together with related risks and
risk levels
• Test basis and test objects are also evaluated to identify defects they may contain
and to assess their testability
• Test analysis is often supported by use of test techniques
• Test analysis answers question “what to test?” in terms of measurable coverage
criteria
40
Test Design
• Test design includes elaborating test conditions into test cases and other testware
(e.g., test charters)
• This activity often involves identification of coverage items, which serve as a guide
to specify test case inputs
• Test techniques can be used to support this activity
• Test design also includes defining test data requirements, designing test
environment and identifying any other required infrastructure and tools
• Test design answers question “how to test?”
41
Test Implementation
• Test execution includes running tests in accordance with test execution schedule (test runs)
• Test execution may be manual or automated
• Test execution can take many forms, including continuous testing or pair testing sessions
• Actual test results are compared with expected results
• Test results are logged
• Anomalies are analyzed to identify their likely causes
• Analysis allows us to report anomalies based on failures observed
43
Test Completion
• Test completion activities usually occur at project milestones (e.g., release, end of iteration,
test level completion) for any unresolved defects, change requests or product backlog items
created
• Any testware that may be useful in the future is identified and archived or handed over to
appropriate teams
• Test environment is shut down to an agreed state
• Test activities are analyzed to identify lessons learned and improvements for future iterations,
releases, or projects
• Test completion report is created and communicated to stakeholders
44
Test Process in Context
47
Testware
48
Work Products, 1/3
51
Traceability between Test Basis and Testware,
1/4
52
Traceability between Test Basis and Testware,
2/4
53
Traceability between Test Basis and Testware,
3/4
54
Traceability between Test Basis and Testware,
4/4
• Two principal roles in testing: Test management role and Testing role
• Activities and tasks assigned to these roles depend on factors such as
project and product context, skills of people in roles, and organization
56
Roles in Testing, 2/4
• Test management role takes overall responsibility for test process, test team and leadership
of test activities
• Test management role is mainly focused on activities of test planning, test monitoring and
control and test completion
• The way in which test management role is carried out varies depending on context
• In Agile software development, some of test management tasks may be handled by Agile
team
• Tasks that span multiple teams or entire organization may be performed by test managers
outside of development team
57
Roles in Testing, 3/4
58
Roles in Testing, 4/4
59
Essential Skills and Good Practices in Testing
• Skill is the ability to do something well that comes from one’s knowledge,
practice and aptitude
• Good testers should possess some essential skills to do their job well
• Good testers should be effective team players and should be able to
perform testing on different levels of test independence
60
Generic Skills Required for Testing, 1/2
• Testing knowledge (to increase effectiveness of testing, e.g., by using test techniques)
• Thoroughness, carefulness, curiosity, attention to details, being methodical (to identify defects,
especially the ones that are difficult to find)
• Good communication skills, active listening, being a team player (to interact effectively with all
stakeholders, to convey information to others, to be understood, and to report and discuss defects)
• Analytical thinking, critical thinking, creativity (to increase effectiveness of testing)
• Technical knowledge (to increase efficiency of testing, e.g., by using appropriate test tools)
• Domain knowledge (to be able to understand and to communicate with end users/business
representatives)
61
Generic Skills Required for Testing, 2/2
62
Whole Team Approach, 1/4
• Important skill for a tester is ability to work effectively in a team context and
to contribute positively to team goals
• Whole team approach – a practice coming from Extreme Programming –
builds upon this skill
63
Whole Team Approach, 2/4
• Testers work closely with other team members to ensure that desired
quality levels are achieved
• Collaborating with business representatives to help them create suitable
acceptance tests and working with developers to agree on test strategy and
decide on test automation approaches
• Testers can thus transfer testing knowledge to other team members and
influence product development
65
Whole Team Approach, 4/4
66
Independence of Testing, 1/5
67
Independence of Testing, 2/5
68
Independence of Testing, 3/5
• For most projects, it is usually best to carry out testing with multiple levels
of independence (e.g., developers performing component and component
integration testing, test team performing system and system integration
testing, and business representatives performing acceptance testing)
69
Independence of Testing, 4/5
73