Quality Management

Quality ?
 Quality,simplistically, means that a
product should meet its specification
◦ The software product should deliver
the required functionality (functional
requirements) with the required
quality attributes (non–functional
requirements)
Software Quality
Conformance to explicitly stated functional and
performance requirements, explicitly
documented development standards and
implicitly characteristics that are expected of
all professionally developed software
Software Quality
 Software requirements are the foundation from
which quality is measured. Lack of conformance
to requirements is lack of quality.

 Specified standards define a set of development

criteria that guide the manner in which software
is engineered. If the criterion is not followed the
lack of quality will almost surely result.

 If software conforms to its explicit requirements

but fails to meet implicit requirements, software
quality is suspect.
QUALITY IS CENTERED AROUND
SATISFYING THE CUSTOMER
 If a product performs as needed and expected, and
satisfies its customer, then it is said to be a quality
product.

 Product must satisfy the customer’s problems and

needs, and it must meet the customer expectation.
The expectations include function, cost, minimal
defects, reliability, good level of service, and so on.
 QUALITY IS CENTERED
AROUND SATISFYING THE
CUSTOMER
User satisfaction= compliant product +
good quality +
delivery within budget and
schedule
 Quality applies to the life of the product and
covers all aspects.
Product quality attributes
Attributes of good software (beyond delivering the required functionality):
 Efficiency
◦ Software should not make wasteful use of system resources
(disk and memory space, CPU time, etc.) and should present
appropriate response times
 Usability (ease of use)
◦ Software must be usable by the users for which it was
designed
 Dependability (reliability, availability, security, safety,…)
◦ Software must be trustworthy
 Maintainability (ease of maintenance)
◦ Software must evolve to meet changing needs
◦ Software costs more to maintain than it does to develop. For
systems with a long life, maintenance costs may be several
times development costs
 …
Quality-related activities (1)
 Software Verification and Validation (V & V)
◦ Goals:
 Establish the existence of defects in a product
 Assess whether or not the product is usable in an operational
situation
◦ Verification
 Ensure that we are building the product right, i.e., according to
its specification
◦ Validation
 Ensure that we are building the right product, i.e., according to
user needs
◦ V & V are integral part of the development process
◦ Concerned directly with product quality
Verification and Validation (V&V)

 Build the product right —

 a process related statement.

 Build the right product –

 a product related statement

Verification -> Are we building product right?

Validation -> Are we building right product?
Verification and Validation (V&V)
Verification: Means are we doing right thing. i.e we have to check whether we are
implementing right process. verification is concerned with whether the system is
well-engineered, error-free, and so on.  
Validation: Means are we doing things right. i.e we have to check whether we have
developed a software as per the client requirement. Validation is concerned with
checking that the system will meet the customer’s actual needs
 Verification evaluates documents, plans, code, requirements, and specifications.

 Validation, on the other hand, evaluates the product itself.

 The inputs of verification are checklists, issues lists, walkthroughs or inspection

meetings, reviews and meetings.

 The input of validation, on the other hand, is the actual testing of an actual

product.
 The output of verification is a nearly perfect set of documents, plans,

specifications, and requirements document.

 The output of validation, on the other hand is the actual/perfect product.

Verification comes before Validation.

ACTIONS IN BUILDING THE RIGHT
PRODUCT
 It is all about understanding the customer’s
problems to solved and needs to be addressed
 Product requirements: understand the customer’s
problem and needs
 Product objectives: define the high level solution
 Product specifications: define the detailed solution
 Ongoing customer involvements: validate customer
requirements
Building the Product Right
 Follow a software development process that
offers a way to measure the conformance of
the emerging product to the product
specification.
 The process used also must ensure that the

development of the product cannot pass from

one phase or activity to the next until certain
conditions are met.
Quality-related activities (2)
 Software Quality Management (SQM)
◦ Goals: Ensure that the required level of quality is achieved in
software products, namely, that defined standards and
procedures are followed
◦ SQM should aim to develop a ‘quality culture’ where quality is
seen as everyone’s responsibility
 Sub-activities:
 (Organization–wide) Quality assurance
 Establish organisational procedures and standards for quality in a quality
manual
 (Project–wide) Quality planning
 Select applicable procedures and standards for a particular project and modify
these as required. Produce a quality plan.
 (Project–wide) Quality control (QC)
 Ensure that procedures and standards are followed by the software
development team. Produce quality review reports
◦ Concerned directly with process quality and, indirectly,
product quality
Quality Concepts
 Variation control is the heart of quality control (software
engineers strive to control the process applied, resources
expended, and end product quality attributes).
 Quality of design - refers to characteristics designers
specify for the end product to be constructed
 Quality of conformance - degree to which design
specifications are followed in manufacturing the product
Quality Concepts
 Quality control - series of inspections, reviews,
and tests used to ensure conformance of a
work product to its specifications
 Quality assurance - consists of the auditing
and reporting procedures used to provide
management with data needed to make
proactive decisions
VARIATION CONTROL
 Controlling variation is the key to a high quality product. In
the software context, we strive to control the variation in the
process we apply, the resources we expend and the quality
attributes of the end product.

 From one project to another we want minimize the difference

between the predicted resources needed to complete the
project and the actual resources used.

 Minimize the number of defects with each release.

 Minimize the differences in speed and accuracy.

Software quality assurance
 Software quality assurance is an umbrella activity
that is applied throughout the software process.

 Software quality assurance is a planned and

systematic pattern of actions that are required to
ensure high quality in software.
SQA ENCOMPASSES
 Formal Technical reviews
 Testing strategy
 Control of software documentation
 Ensure compliance (conformity) with software

development standards
 Measurement and reporting mechanisms
Cost of Quality
Includes all costs incurred in the pursuit(search, detection,
tracking down) of quality or perform quality related work.
 Prevention costs - quality planning, formal technical
reviews, test equipment, training
 Appraisal costs - in-process and inter-process
inspection, equipment calibration and maintenance,
testing
 Failure costs
◦ Internal failure costs - rework, repair, failure mode analysis
◦ External failure costs - complaint resolution, product return and
replacement, help line support, warranty work
Prevention Appraisal Failure Costs
Cost Costs
Quality In process and Internal External
planning inter process failure failure
inspection

Formal Equipment Rework Complaint

technical calibration and resolution
reviews maintenance

Test Testing Repair Warranty work

equipment

Training Failure mode Help line

analysis support

Product return
and
replacement
SQA ACTIVITIES
 Prepares an SQA plan for a project
◦ Evaluations to be performed
◦ Audits and reviews to be performed
◦ Standards that are applicable to the project
◦ Procedures for error reporting and tracking
◦ Documents to be produced by the SQA group
 Participates in the development of the
software process description
Software Reviews
What is software reviews?
a “filter” for the software engineering process.
Purpose: serves to uncover errors in analysis, design, coding, and testing.
Why software reviews?
 To err is human
 Easy to catch the errors in engineers’ work
A review --> a way to
- identify the needed improvements of the parts in a product
- confirm the improvement parts of a product.
- achieve technical work of more uniform, predicable, and manageable.
Different types of reviews:
- Informal reviews: informal meeting and informal desk checking
- Formal reviews: (design to an audience of customers, management, and staff)
Walkthrough, inspection, and round-robin reviews)
Software Reviews - Objectives
 The primary objective of technical reviews is
to find errors during the process so that they
do not become defects after the release of
the software.
 Point out needed improvements in the

product
 Conform those parts of a product in which

improvement is either not desired or not

needed
Software Reviews
 Software engineers (and others) conduct
formal technical reviews (FTRs)
 Using formal technical reviews (walkthroughs

or inspections) is an effective means for

improving software quality.
FORMAL TECHNICAL REVIEW
Formal Technical review (FTR) is a software quality assurance
activity performed by software engineers with the following
objectives:
- To uncover errors in function, logic or implementation of
the software.
- verify that the software meets its requirements.
- ensure that the software has been developed according to
the standards. 
- achieve uniform (standardized)software.
- make projects manageable.
FTR includes walkthroughs, inspections, round-robin
reviews.
Each FTR is conducted as a meeting and is considered
successful only if it is properly planned, controlled and attended.
Formal Technical Reviews
The purpose of formal technical review serves as a
training ground for junior engineers and to promote
backup and continuity.
Constraints of formal technical review meeting’s
include
 Involves 3 to 5 people (including reviewers)

 Advance preparation (no more than 2 hours per

person) required
 Duration of review meeting should be less than 2 hours

 Focus of review is on a discrete (separate) work product

 focus on a specific part of a software product .

Formal Technical Reviews
 Reviewers ask questions that enable the producer to
discover his or her own error (the product is under review
not the producer)

 Producer of the work product walks the reviewers through

the product

 Recorder writes down any significant issues raised during

the review

 Reviewers decide to accept or reject the work product and

whether to require additional reviews of product or not
REVIEW MEETING GUIDELINES
 Review the product not the producer
 Set an agenda and maintain it
 Minimize the debate and discussions.
 The defect areas should be pointed but no solution should be
provided.
 Take written notes
 Limit the number of participants and insist upon advance
preparation
 Develop a checklist of each product that is likely to be reviewed
 Allocate resources and schedule time for FTRs
 Review your early reviews
Software Defects
 Industry studies suggest that design activities
introduce 50-65% of all defects or errors during the
software process

 Review techniques have been shown to be up to 75%

effective in uncovering design flaws which ultimately
reduces the cost of subsequent activities in the
software process

 Defect amplification models can be used to show that

the benefits of detecting and removing defects from
activities that occur early in the software process
Defect Amplification Model
Software Reliability
 The probability of failure free operation of a
computer program in a specified environment for a
specified time.

 It can measured, directed and estimated.

 The most important dynamic characteristics of most

software systems is their reliability.

 The reason for this is that the costs of system

failure often exceed the cost of developing the
software system.
  
Software Reliability
 Software Failure: non conformance to software
requirements
 Software faults cause software failure.

 Reliability is related to the probability of an error

occurring in operational use.

 Removing software faults from the parts of the

system that are rarely used makes little real
difference to the perceived reliability.
Software Reliability
 Reliability in a software system can be achieved using three
complementary strategies:
 
Fault avoidance Fault tolerance Fault detection

 A simple measure of reliability is mean time between failure (MTBF).

“Mean Time” means, statistically, the average time.
MTBF= MTTF + MTTR

 * MTBF = Mean Time Between Failure(is the time from one failure to
another. )
 *MTTR = Mean Time to Repair(is the average time that it takes to repair
something after a failure)
 *MTTF = Mean Time to Failure(is something that cannot be repaired)
 
Software Availability
 Availability =MTTF/(MTTF + MTTR) * 100%

 Software availability is the probability that a

program is operating according to requirements at
a given point in time

21
November 15, 1997
Reliability should always take
precedence over efficiency.

Why?
Reasons
 Computers are now cheap and fast
 Unreliable software is liable to be discarded

by users
 System failure cost may be enormous
 Unreliable systems are difficult to improve
 Unreliable systems may cause information

loss
 Inefficiency is predictable
Software Safety
 Software safety is software quality assurance activity that
focuses on the identification and assessment of potential
hazards that may affect software negatively and cause entire
system to fail.

 Early identification of software hazards allows developers to

specify design features can eliminate or at least control the
impact of potential hazards.

 Software reliability involves determining the likelihood that a

failure will occur, while software safety examines the ways in
which failures may result in conditions that can lead to a
mishap.
Examples
 An example of software safety in action is in rockets being
sent to space.Obviously, for rockets, safety is critical to
ensure there a minimized risk of an accident.

 Another example of safety critical software is a system that

controls movement of a car, such as Honda’s ADAS
 (Automated Driver Assistance System). This automated
steering system is able to control the car without the driver
turning the wheel at all. Obviously, the need for safety is vital
in this situation.
 Example Fault Tree -- Thermal

Loss of heat

..
Power failure Computer failure
. Incorrect
input
SW failed to
throw switch

Computer failure SW failed to

..
throw switch
Logic reversed

.
November 15, 1997
Statistical Quality Assurance
 Information about software defects is
collected and categorized
 Each defect is traced back to its cause
 Using the Pareto principle (80% of the defects

can be traced to 20% of the causes) isolate

the "vital few" defect causes
 Once the vital few causes have been

identified, move to correct the problems that

have caused the defects.
Categories of Errors
 Incomplete or erroneous specification (IES)
 Misinterpretation of customer communication

(MCC)
 Intentional deviation from specification (IDS)
 Violation of programming standards (VPS)
 Error in data representation (EDR)
 Inconsistent component interface (ICI)
 Error in design logic (EDL)
What is Six Sigma??
Six Sigma is a set of techniques and tools for process
improvement.
 a philosophy
• a performance measurement
• an improvement framework
• a set of improvement tools
• a structured approach for business improvement (a business
strategy)
Six Sigma Software Engineering

 Define customer requirements, deliverables, and

project goals via well-defined methods of customer
communication.

 Measure each existing process and its output to

determine current quality performance (e.g.,
compute defect metrics)

 Analyze defect metrics and determine vital few

causes.
Six Sigma Software Engineering
 For an existing process that needs improvement
◦ Improve process by eliminating the root causes for
defects
◦ Control future work to ensure that future work does
not reintroduce causes of defects
 If new processes are being developed
◦ Design each new process to avoid root causes of
defects and to meet customer requirements
◦ Verify that the process model will avoid defects and
meet customer requirements
Mistake-Proofing Software
 Poka-yoke can be used wherever something can go wrong
or an error can be made. It is a technique, a tool that can
be applied to any type of process be it in manufacturing or
the service industry. 
 What is Poke-yoke? A method that uses sensor or other devices
for catching errors
 Poka-yoke effects two key elements of ZDQ:
1. Identifying the defect immediately ( Point of Origin Inspection)
2. Quick Feedback for Corrective Action
 Poka-yoke devices help us avoid defects, even when
inadvertent errors are made.
 http://www.mistakeproofing.com/software.html
ISO 9000 Quality Standards
 Quality assurance systems are defined as the
organizational structure, responsibilities, procedures,
processes, and resources for implementing quality
management.

 ISO 9000 describes the quality elements that must be

present for a quality assurance system to be compliant
with the standard, but it does not describe how an
organization should implement these elements.

 ISO 9001:2000 is the quality standard that contains 20

requirements that must be present in an effective
software quality assurance system.
SQA Plan
 Management section - describes the place of
SQA in the structure of the organization
 Documentation section - describes each work

product produced as part of the software

process
 Standards, practices, and conventions section

- lists all applicable standards/practices

applied during the software process and any
metrics to be collected as part of the software
engineering work
SQA Plan
 Reviews and audits section - provides an
overview of the approach used in the reviews
and audits to be conducted during the project
 Test section - references the test plan and

procedure document and defines test record

keeping requirements
SQA Plan
 Problem reporting and corrective action
section - defines procedures for reporting,
tracking, and resolving errors or defects,
identifies organizational responsibilities for
these activities
 Other - tools, SQA methods, change control,

record keeping, training, and risk

management