Chapter 1:

Software Quality Assurance Fundamentals

1 Definition of Quality, Quality Assurance, Quality Control, Difference

between QA and QC,
2 Software Quality Assurance, SQA Planning & Standards
3 SQA Activities
4 Building Blocks of SQA
5 Software Quality factors
6 Software Quality Metrics: Process Metrics & Product Metrics
7 Software Reliability & Reliability Measurement Factors: ROCOF, MTTF,
MTTR, MTBF, POFOD, Availability
What is Quality?

Quality is meeting the requirement, expectation, and needs of the customer is free
from the defects, lacks and substantial variants. There are standards needs to
follow to satisfy the customer requirements. 

Product which should meet its specification.

a) Customer Quality requirements


Efficiency
 Reliability

b) Developer Quality requirements

 Maintainability, reusability
 Reliability, portability
What is Assurance?
Assurance is provided by organization management, it means giving a positive
declaration on a product which obtains confidence for the outcome. It gives a
security that the product will work without any glitches as per the expectations or
requests.

What is Quality Assurance?

Quality Assurance is known as QA and focuses on preventing defect. Quality
Assurance ensures that the approaches, techniques, methods and processes are
designed for the projects are implemented correctly.

Quality assurance activities monitor and verify that the processes used to manage
and create the deliverables have been followed and are operative.

Quality Assurance is a proactive process and is Prevention in nature. It recognizes

flaws in the process. Quality Assurance has to complete before Quality Control.

What is Control?
Control is to test or verify actual results by comparing it with the defined
standards.

What is Quality Control?

Quality Control is known as QC and focuses on identifying a defect. QC ensures
that the approaches, techniques, methods and processes are designed in the project
are following correctly. QC activities monitor and verify that the project
deliverables meet the defined quality standards.

Quality Control is a reactive process and is detection in nature. It recognizes the

defects. Quality Control has to complete after Quality Assurance.

What is The Difference in QA/QC?

Many people think QA and QC are the same and interchangeable but this is not
true. Both are tightly linked and sometimes it is very difficult to identify the
differences. Fact is both are related to each other but they are different in origins.
QA and QC both are part of Quality Management however QA is focusing on
preventing defect while QC is focusing on identifying the defect.

Here is the exact difference between Quality Control and Quality Assurance
that one needs to know:
Quality Assurance Quality Control

It is a process which deliberates on providing QC is a process which deliberates on

assurance that quality request will be fulfilling the quality request.
achieved.
Quality Assurance Quality Control

A QA aim is to prevent the defect. A QC aim is to identify and improve

the defects.

QA is the technique of managing quality. QC is a method to verify quality.

QA does not involve executing the program. QC always involves executing the
program.

All team members are responsible for QA. Testing team is responsible for QC.

QA Example: Verification QC Example: Validation.

QA means Planning for doing a process. QC Means Action for executing the
planned process.

Statistical Technique used on QA is known as Statistical Technique used on QC is

Statistical Process Control (SPC.) known as Statistical Quality Control
(SPC.)

QA makes sure you are doing the right things. QC makes sure the results of what
you've done are what you expected.

QA Defines standards and methodologies to QC ensures that the standards are

followed in order to meet the customer followed while working on the product.
requirements.

QA is the process to create the deliverables. QC is the process to verify that

deliverables.

QA is responsible for full software QC is responsible for software testing

development life cycle. life cycle.

Real-life scenario Examples for QA/QC

QA Example:
Suppose our team has to work on completely new technology for an upcoming
project. Our team members are new to technology. So, for that, we need to create a
plan for getting the team members trained in the new technology.

Based on our knowledge, we need to collect pre-requisites like DOU (Document of

Understanding), design document, technical requirement document, functional
requirement document, etc. and share these with the team.

This would be helpful while working on the new technology and even would be
useful for any newcomer in the team. This collection & distribution of
documentation and then kicking off the training program is a part of the QA
process.

QC Example:
Once the training is completed, how can we make sure that the training was
successfully done for all the team members?

For this purpose, we will have to collect statistics e.g. the number of marks the
trainees got in each subject and the minimum number of marks expected after
completing the training. Also, we can make sure that everybody has taken training
in full by verifying the attendance record of the candidates.

If the marks scored by candidates are up to the expectations of the

trainer/evaluators, then we can say that the training is successful otherwise we will
have to improve our process in order to deliver high-quality training.

Another way to improve the training process would be collecting feedback from
the trainees at the end of the training program. Their feedback will tell us what was
good about the training and what are the areas where we can improve the quality of
training. So, such activities are a part of the QA process.

Conclusion
Key Points:
 In QA, processes are planned to evade the defects
 QC agreements with the discovery of the defects and modifying them while
making the product
 QA detects weakness
 QC detects defects
 QA is process oriented
  QC is product oriented
 QA is a failure prevention system
 QC is a failure detection system.
QA & QC both are different from each other and required as part of quality
management. They should not be misunderstood as interchangeable terms. QA is
process focused while QC is end-product focused.

Quality control is inspecting something (a product or a service) to ensure that it is

working fine. If the product or service is not working fine, then the issue needs to
be fixed or eliminated in order to meet conformance standards. So, it aims at
detecting and correcting issues.

Quality assurance, on the other hand, aims at preventing the issues from occurring
in the future by improving the process.

To summarize, we can say that Quality assurance does not eliminate the need for
Quality control as QC lies at the very core of Quality management.

Quality Testing:

It is assessment of the extents which a test object meets given

requirements.
Error: Errors are a part of our daily life. Human makes errors in their
thoughts, actions and in the products that might result from their
actions. Errors occur wherever humans are involved in taking
actions and making decisions.
Error is a state of the system, it could lead to failure.
Bug: Due to which program fail to perform its intended function
correctly.

Defect: It is bug – roughly say Fault.

Fault: It is adjudged cause of error.

Failure: Failure is said to occur whenever the external behavior of a system

does not confirm to that prescribed in the system specification.

 Error & Bug

 Errors are a part of our daily life. Humans make errors in their thoughts,
actions, and in the products that might result from their actions. Errors
occur wherever humans are involved in taking actions and making decisions.

Error – Due to which program failed to execute.

Bug – Due to which program fail to perform its intended function correctly.

Bug Life Cycle

 1. New: When the bug is posted for the first time, its state will be “NEW”.
This means that the bug is not yet approved.

 2. Open: After a tester has posted a bug, the lead of the tester approves that
the bug is genuine and he changes the state as “OPEN”.
  3. Assign: Once the lead changes the state as “OPEN”, he assigns the bug to
corresponding developer or developer team. The state of the bug now is
changed to “ASSIGN”.

 4. Test: Once the developer fixes the bug, he has to assign the bug to the
testing team for next round of testing. Before he releases the software with
bug fixed, he changes the state of bug to “TEST”. It specifies that the bug
has been fixed and is released to testing team.

 5. Deferred: The bug, changed to deferred state means the bug is expected
to be fixed in next releases. The reasons for changing the bug to this state
have many factors. Some of them are priority of the bug may be low, lack of
time for the release or the bug may not have major effect on the software.

 6. Rejected: If the developer feels that the bug is not genuine, he rejects the
bug. Then the state of the bug is changed to “REJECTED”.

 7. Duplicate: If the bug is repeated twice or the two bugs mention the same
concept of the bug, then one bug status is changed to “DUPLICATE”.

 8. Verified: Once the bug is fixed and the status is changed to “TEST”, the
tester tests the bug. If the bug is not present in the software, he approves that
the bug is fixed and changes the status to “VERIFIED”.

 9. Reopened: If the bug still exists even after the bug is fixed by the
developer, the tester changes the status to “REOPENED”. The bug traverses
the life cycle once again.

 10. Closed: Once the bug is fixed, it is tested by the tester. If the tester feels
that the bug no longer exists in the software, he changes the status of the bug
to “CLOSED”. This state means that the bug is fixed, tested and approved.

Why we concern about quality

B’coz of:

 Increased competition

 Customer Orientation
  focus on Customer

Cost of inadequate quality

 Loss of reputation

 Loss of customers

 Business at stake

 Imperative by law

Software quality assurance (SQA) is a process which assures that all software

engineering processes, methods, activities and work items are monitored and
comply against the defined standards. ... SQA incorporates all software
development processes starting from defining requirements to coding until release.

SQA Benefits

SQA has a host of benefits. It ensures that that software built as per SQA
procedures are of specified quality. SOA helps to

1. Eliminate errors when they are still inexpensive to correct

2. Improves the quality of the software
3. Improving the process of creating software
4. Create a mature software process

SQA planning and standards:

How to implement the quality assurance?
Step 1) Develop SQA Plan
Step 2) Define the standards/methodology
Step 3) Review the process

Step 1) Develop SQA Plan

Testing activity needs Test Plan likewise SQA activity also needs a plan which is
called SQA plan.
The goal of SQA plan is to craft planning processes and procedures to ensure
products manufactured, or the service delivered by the organization are of
exceptional quality.

During project planning, Test Manager makes an SQA plan where SQA audit is
scheduled periodically.

In the SQA Plan, the Test Manager should do as following

Step 1.1) Identify the role and responsibilities of SQA team

In a project team, every member must have responsibility for the quality of his or
her work. Each person has to make sure their work meet the QA criteria.

The SQA team is the group of person who plays the major role in the project.
Without QA, no business will run successfully. Therefore, the Test Manager has to
make clear the responsibility of each SQA member in SQA plan as below:

 Review and evaluate the quality of project activities to meet the QA criteria

 Coordinate with management board and project teams to assess
requirements and engage in project review and status meetings.
 Design, track and collect metrics to monitor project quality.
 Measure the quality of product; 
 ensure the product meet the customer expectations.

For example, in the SQA Plan of the project Guru99 Bank, you can create the list
members of SQA team as below

No Member Roles Responsibility

Develop and document quality standard and process for all management pro
1 Peter SQA Leader
Manage software quality assurance activities for the project
2 James SQA auditor Perform SQA tasks, report to SQA leader the result of SQA review.
3 Bean SQA auditor Perform SQA tasks, report to SQA leader the result of SQA review.
Step 1.2) List of the work products that the SQA auditor will review and audit
The Test Manager should

 List out all the work products of each Test Management Process

 Define which facilities or equipment the SQA auditor can access to perform
SQA tasks such as process evaluations and audits.
For example, for the project Guru99 Bank, you can list out the work products of
each Test Management Process and define permission for SQA members to access
these work products as per the following table

No Management Phases Work product Path Permission

1 Risk analysis Risk Management document [Server path] Read
2 Estimation Estimation and Metrics report … Read
3 Planning Test Planning document … Read
4 Organization Human resource plan, training plan … Read
5 Monitoring and Control Collected metrics of project effort … Read
6 Issue Management Issue management report … Read
7 Test report Test Report document … Read
Step 1.3) Create the schedule to perform the SQA tasks
In this step, the Test Manager should describe the tasks to be performed by SQA
auditor with special emphasis on SQA activities as well as the work product for
each task.

Test Manager also creates the scheduling of those SQA tasks. Normally, the SQA
schedule is driven by the project development schedule. Therefore, an SQA task is
performed in relationship to what software development activities are taking place.

In the SQA plan, Test Manager makes the schedule for management review. For
example

Personal in
Date SQA Tasks Description
charge

– Software Specification Rev

30-Oct- Evaluate project planning,
James
2014 tracking and oversight processes
– Estimation, Master Schedule and Project
Review
15-Dec-
Review requirement analysis James
2014 – Review the software requirement development
30-Mar- Review and Evaluate Test
James
2015 Design – Review the Test Design document
30-Mar-
Review release Bean
2015 – Process Audit: Final Release
2-Apr-
Review Project closing Bean
2015 – External review after final delivery to customer
Step 2) Define the standards/methodology
To review the Management activities against the standards process, you should do
the following steps

1. Define the policies and procedures intended to prevent defects from

occurring in the management process
2. Document the policies & procedures
3. Inform and train the staff to use it

Step 3) Review the process

Review project activities to verify compliance with the defined management
process. In the management review, the SQA members have to perform 5 SQA
reviews as following

1. Review project planning

2. Review software requirement analysis
3. Review test Design
4. Review before release
5. Review project closing
In each SQA phase, the SQA members provide consultation and review of the
project plans, work product, and procedures regarding compliance to defined
organizational policy and standard procedures.

Importance of standards:
(i) Encapsulation of best practices – avoid repetitions of past mistakes.
(ii) They are framework for QA process.
(iii) They involved checking compliance to standards
(iv) They provide continuity

SQA Activities:
  Software quality assurance is composed of a variety of tasks associated with
two different constituencies – the software engineers who do technical work
and an SQA group that has responsibility for quality assurance planning,
record keeping, analysis and reporting.
 Following activities are performed by an independent SQA group:
1. Prepares an SQA plan for a project: The plan is developed during
project planning and is reviewed by all stakeholders. Quality assurance
activities performed by the software engineering team and the SQA group
are governed by the plan. The plan identifies evaluations to be performed,
audits and reviews to be performed, standards that are applicable to the
project, procedures for error reporting and tracking, documents to be
produced by the SQA team, and amount of feedback provided to the
software project team.
2. Participates in the development of the project’s software process
description: The software team selects a process for the work to be
performed. The SQA group reviews the process description for compliance
with organizational policy, internal software standards, externally imposed
standards (eg. ISO-9001), and other parts of software project plan.
3. Reviews software engineering activities to verify compliance with the
defined software process: The SQA group identifies, documents, and tracks
deviations from the process and verifies that corrections have been made.
4. Audits designated software work products to verify compliance with
those defined as a part of the software process : The SQA group reviews
selected work products, identifies, documents and tracks deviations, verifies
that corrections have been made, and periodically reports the results of its
work to the project manager.
5. Ensures that deviations in software work and work products are
documented and handled according to a documented
procedure: Deviations may be encountered in the project plan, process
description, applicable standards, or technical work products.
6. Records any noncompliance and reports to senior management: Non-
compliance items are tracked until they are resolved.

Building blocks of SQA

Software Quality factors

McCall’s Factor Model

McCall’s Factor Model „ Classifies all software requirement into 11 software

quality factors, grouped into three categories:

1. Product operation factors: Correctness, Reliability, Efficiency, Integrity,

Usability.

2. Product revision factors: Maintainability, Flexibility, Testability.

3. Product transition factors: Portability, Reusability, Interoperability

Quality Definitions
Factors
1 Correctness The extent to which a program satisfies its
specifications and fulfills the user’s mission /
objectives.
2 Reliability The extent to which a program can be expected to
perform its intended functions with required
precision.
3 Efficiency The amount of computing resources and code
requirement by a program to perform a function.
4 Integrity The extent to which access to software or data by
un-authorized person can control.
5 Usability The effort required to learn, operate, prepare input
pout and interpret output of a program.
6 Maintainability The effort required to locate and fix defects in an
operational program.
7 Testability The effort required to test a program to ensure that
it perform its intended functions.
8 Flexibility The effort required to modify an operational
 program.
9 Portability The effort required to transfer a program from one
hardware or software environment to another.
10 Reusability The extent to which parts of software can be reused
in other applications.
11 Interoperability The effort required to couple one system with
another.
Software Quality Criteria / Metrics:
Software Reliability Measurement Techniques

Reliability metrics are used to quantitatively express the reliability of the software

product. The option of which parameter is to be used depends upon the type of
system to which it applies & the requirements of the application domain.

Measuring software reliability is a severe problem because we don't have a good

understanding of the nature of software. It is difficult to find a suitable method to
measure software reliability and most of the aspects connected to software
reliability. Even the software estimates have no uniform definition. If we cannot
measure the reliability directly, something can be measured that reflects the
features related to reliability.

The current methods of software reliability measurement can be divided into

four categories:
1. Product Metrics

Product metrics are those which are used to build the artifacts, i.e., requirement
specification documents, system design documents, etc. These metrics help in the
assessment if the product is right sufficient through records on attributes like
usability, reliability, maintainability & portability. In these measurements are taken
from the actual body of the source code.

i. Software size is thought to be reflective of complexity, development effort,

and reliability. Lines of Code (LOC), or LOC in thousands (KLOC), is an
initial intuitive approach to measuring software size. The basis of LOC is
that program length can be used as a predictor of program characteristics
such as effort &ease of maintenance. It is a measure of the functional
complexity of the program and is independent of the programming language.
ii. Function point metric is a technique to measure the functionality of
proposed software development based on the count of inputs, outputs,
master files, inquires, and interfaces.
iii. Test coverage metric size fault and reliability by performing tests on
software products, assuming that software reliability is a function of the
portion of software that is successfully verified or tested.
iv. Complexity is directly linked to software reliability, so representing
complexity is essential. Complexity-oriented metrics is a way of determining
the complexity of a program's control structure by simplifying the code into
a graphical representation. The representative metric is McCabe's
Complexity Metric.
v. Quality metrics measure the quality at various steps of software product
development. An vital quality metric is Defect Removal Efficiency (DRE).
DRE provides a measure of quality because of different quality assurance
and control activities applied throughout the development process.

2. Project Management Metrics

Project metrics define project characteristics and execution. If there is proper

management of the project by the programmer, then this helps us to achieve better
products. A relationship exists between the development process and the ability to
complete projects on time and within the desired quality objectives. Cost increase
when developers use inadequate methods. Higher reliability can be achieved by
using a better development process, risk management process, configuration
management process.

These metrics are:

o Number of software developers

o Staffing pattern over the life-cycle of the software
o Cost and schedule
o Productivity

3. Process Metrics

Process metrics quantify useful attributes of the software development process &
its environment. They tell if the process is functioning optimally as they report on
characteristics like cycle time & rework time. The goal of process metric is to do
the right job on the first time through the process. The quality of the product is a
direct function of the process. So process metrics can be used to estimate, monitor,
and improve the reliability and quality of software. Process metrics describe the
effectiveness and quality of the processes that produce the software product.

Examples are:

o The effort required in the process

o Time to produce the product
o Effectiveness of defect removal during development
o Number of defects found during testing
o Maturity of the process

4. Fault and Failure Metrics

A fault is a defect in a program which appears when the programmer makes an

error and causes failure when executed under particular conditions. These metrics
are used to determine the failure-free execution software.

To achieve this objective, a number of faults found during testing and the failures
or other problems which are reported by the user after delivery are collected,
summarized, and analyzed. Failure metrics are based upon customer information
regarding faults found after release of the software. The failure data collected is
therefore used to calculate failure density, Mean Time between Failures
(MTBF), or other parameters to measure or predict software reliability.

Reliability metrics are used to quantitatively expressed the reliability of the

software product. The option of which metric is to be used depends upon the type
of system to which it applies & the requirements of the application domain.

Some reliability metrics which can be used to quantify the reliability of the
software product are as follows:

1. Mean Time to Failure (MTTF)

MTTF is described as the time interval between the two successive failures.
An MTTF of 200 mean that one failure can be expected each 200-time units. The
time units are entirely dependent on the system & it can even be stated in the
number of transactions. MTTF is consistent for systems with large transactions.

For example, It is suitable for computer-aided design systems where a designer

will work on a design for several hours as well as for Word-processor systems.

To measure MTTF, we can evidence the failure data for n failures. Let the failures
appear at the time instants t1,t2.....tn.

MTTF can be calculated as

2. Mean Time to Repair (MTTR)

Once failure occurs, some-time is required to fix the error. MTTR measures the

average time it takes to track the errors causing the failure and to fix them.

3. Mean Time Between Failure (MTBR)

We can merge MTTF & MTTR metrics to get the MTBF metric.

                  MTBF = MTTF + MTTR

Thus, an MTBF of 300 denoted that once the failure appears, the next failure is
expected to appear only after 300 hours. In this method, the time measurements are
real-time & not the execution time as in MTTF.

4. Rate of occurrence of failure (ROCOF)

It is the number of failures appearing in a unit time interval. The number of

unexpected events over a specific time of operation. ROCOF is the frequency of
occurrence with which unexpected role is likely to appear. A ROCOF of 0.02
mean that two failures are likely to occur in each 100 operational time unit steps. It
is also called the failure intensity metric.

5. Probability of Failure on Demand (POFOD)

POFOD is described as the probability that the system will fail when a service is
requested. It is the number of system deficiency given several systems inputs.

POFOD is the possibility that the system will fail when a service request is made.

A POFOD of 0.1 means that one out of ten service requests may fail.POFOD is an
essential measure for safety-critical systems. POFOD is relevant for protection
systems where services are demanded occasionally.

6. Availability (AVAIL)

Availability is the probability that the system is applicable for use at a given time.
It takes into account the repair time & the restart time for the system. An
availability of 0.995 means that in every 1000 time units, the system is feasible to
be available for 995 of these. The percentage of time that a system is applicable for
use, taking into account planned and unplanned downtime. If a system is down an
average of four hours out of 100 hours of operation, its AVAIL is 96%.