42 12

Quality-Enhanced Roadmap for ISO 9001 Registered

Organizations
Wen-Kui Chang1 Shih-Fang Huang2 Jeng-Feng Yang3
1
Dept. of Computer Science & Information Engineering, Tunghai University, Taichung
2
Motive Power Industry Co., Ltd. Changhua
3
GIS Research Center, Feng Chia University, Taichung
Email: [email protected], [email protected], [email protected]
Abstract
Normally starting a software quality program from scratch is time consuming and a task often
doomed to failure before it is begun. Inadequate preparation and failure to specify the tailored process
sequences in the organization are only a few of the pitfalls waiting for the overanxious practitioner.
Standards are the keystone of a software quality system to provide the basis against which activities can
be measured and evaluated. In nowadays, there exist many international standards and development
guides such as DO-178B, ISO/IEC 12207, ISO/IEC 15504, CMMI, etc., to provide common methods
and practices while developing a complex project. In the CMMI continuous representation, a target
profile establishes a target of capability level profile for process improvement. In particular, an
organization, before deploying an improvement program, has to specify her target staging in terms of a
sequence of target profiles to describe the path of process improvement. In this paper, we investigate the
issue of process improvement in a software-intensive organization and establish a quality system for
emphasizing software verification and validation (V&V) in the aspects of capability levels and its
integrity levels. Considering the practical resources at a middle-scale software-intensive organization in
Taiwan area, we propose a feasible, efficient and economical roadmap for software process
improvement, no matter the company is ISO 9001:2000 registered or not yet, which then provides a
shortcut to enhance V&V tasks for the ISO-9001 registered software organizations to achieve the
second integrity level.

Keywords: Process Improvement, Verification and Validation, Capability Maturity Model

Integration (CMMI), Target Profile, Integrity Level

1. Introduction
Recently more organizations play much emphasis on verification and validation (V&V) tasks in
order to deliver quality product. To reach the objectives of the software validation process, both static
and dynamic techniques of system checking and analysis are usually employed. However, static
techniques can only check the correspondence between software and its specification, i.e., the so-called
verification process. They cannot demonstrate sufficiently that the software is operationally valid.

Standards are the keystone of a software quality system to provide the basis against activities,
which can be measured and evaluated. In nowadays, there exist many international standards and
development guides that may provide common methods and practices so that the same task can be
1

accomplished the same way each time it is done. For instance, DO-178B provides guidance for
determining the software aspects of airborne systems and equipment complied with airworthiness
requirements; and ISO/IEC 15504 establishes a quantitative standard in the area of process assessment,
etc. In this research, we investigate the full set of Capability Maturity Model Integration (CMMI)
models (Chrissies, Konrad & Shrum, 2003) released by the Software Engineering Institute (SEI) in
2002. In the CMMI continuous representation, a target profile establishes a baseline of capability level
profile to represent an objective for process improvement. In particular, an organization before
deploying an improvement program has to specify her target staging to denote a sequence of target
profiles that describes the path of process improvement to be followed.

The CMMI has been proven that it is efficient for achieving product and process improvement and
widely accepted and implemented in the IT area (Carter et. al., 2002). However, it usually requires lots
of project budget and effort for an organization to implement the CMMI framework for reaching a
higher maturity level. It may become a bottleneck on promoting CMMI to middle-scale software
organizations, which popularly exist in Taiwan areas. Accordingly, we propose a software
quality-enhanced framework for the middle-scale, ISO 9001:2000 registered software organization, in
performing the required V&V tasks from the perspective of the Process Areas (PAs) of the CMMI
model, by the fact that most of current information technology companies in Taiwan area have been
registered ISO 9000. In our belief, once an organization that has achieved ISO 9001 registration, but she
hopes to improve processes continuously, CMMI can be a strong candidate because it provides a more
detailed roadmap for process improvement.

Furthermore, in referring with IEEE Standard 1012-2004-Standards for Software Verification and
Validation Plans (IEEE, 2004), we summarize the minimum V&V tasks and use a
software-integrity-level scheme based upon software intended use and quantify application of the
system to criticality. By integrating the software integrity level involves capability level in the
continuous representation CMMI model; these minimum V&V tasks establish a stepwise roadmap for
capability level from the first to the fourth level. The benefit of our proposed roadmap of improving
software process within a middle-scale organization provides an effective, efficiency and economical
approach, no matter the companies is ISO 9001:2000 registered or not yet.
In the following, we first give a bird-view on the related international standards in the subsequent
section. Section 3 investigates relative mappings of CMMI PAs to ISO 9001:2000. In Section 4, we
summarize the minimum V&V tasks and use a software-integrity-level scheme upon which the
proposed roadmap bases. Finally, target profiles for V&V efforts with the second integrity level in
introduced in Section 5.
2. Major Standards
Essentially, the full set of CMMI models released by SEI in January 2002 aims to provide
guidance for improving an or
g
a
ni
z
a
t
i
on
spr
o
c
e
s
s
e
sa
n
da
bi
l
i
t
yt
oma
na
g
et
hede
v
e
l
opme
nt, acquisition,
and maintenance of his software products (Ahern, Clouse & Turner, 2001). Furthermore, CMMI model
may be useful for appraising its organizational maturity or process area capability, establishing
priorities for improvement, and implementing these improvements.
In CMMI models, process areas describe key aspects of such processes as requirements
management, configuration management, verification, validation, and many others. Specifically, a
process area (PA) provides a list of the required practices to achieve its intended goals, but it does not
describe how an effective process is executed, e.g., entrance and exit criteria, roles of participants and
2

resources. Currently there are two types of CMMI model representations: staged and continuous. The
staged representation uses predefined sets of process areas to define an improvement path for an
organization. A maturity level is a well-defined evolutionary plateau toward achieving the improved
organizational processes. Oppositely, the continuous representation allows an organization to select a
specific process area and improve relative to it. This representation uses capability levels to characterize
relative improvement to an individual process area.
On the other hand, the ISO 9000 family of international quality management standards and
guidelines has become an international reference basis for establishing quality management systems. In
particular, ISO 9001:2000 specifies requirements for a quality management system where an
organization needs to demonstrate its ability to consistently produce product that meets customer and
applicable regulatory requirements, while aims to enhance customer satisfaction through the processes
for continual improvement of the system.
Since their popularity of the ISO 9000 family and CMMI models, relationships between the two
models have been studied in this paper. Paulk studies the relationship between ISO 9001 paragraphs
and CMM Key Process Areas (Paulk, 1995). Due to the CMMI published by SEI in 2002, its content of
models is quite different with that of CMM. In this paper, some mapping results of the ISO 9001:2000
clauses to their corresponding CMMI PAs will be discussed in the subsequent section.
3. Coverage of CMMI PAs with ISO 9001
Since late 2000, significant interest has been seen in certification and registration for many
organizations under ISO 9001:2000 and in transition from the CMM to the CMMI. In contrast to ISO
9001:2000 that can be applied to any organization regardless of its field in which it operates. The
CMMI specifically focuses on organizations that develop products and systems containing software.
While the CMMI provides a roadmap for achieving process capability or maturity levels (Mutafelija,
2001), ISO requires all of its requirements to be fulfilled before certification can be issued. Furthermore,
both ISO and the CMMI are based on principles of systems engineering and a process approach. In the
following, we strive to compare ISO requirements to CMMI PAs and specific practices to depict their
corresponding mappings. To be specific in this paper, we limit our discussion to the ML2 and ML3 PAs
of CMMI SE/SW disciplines.
3.1 Mapping CMMI process areas to ISO 9001:2000 clauses
As stated in the CMMI technical report (Carter et. al., 2002), the Requirements Management
(REQM) PA essentially maintains the project requirements. It describes activities for obtaining and
controlling requirement changes to ensure other relevant plans and data currently kept. Furthermore, it
provides traceability of requirements from customer to product, till the product component. After
analyzing the corresponding interpretations, the goals of the REQM PA may be equally performed by
the clauses 4.1 and 4.2 of ISO 9001, as shown in the first row on Table 1. Similarly, according to
(Ahern, Clouse & Turner, 2001; Yoo, 2006), the Project Planning (PP) PA involves the various tasks
such as developing the project plan, interacting with stakeholders appropriately, getting commitment to
the plan and maintaining the plan. By the interpretation rationale of reaching the same purposes, we
conclude that the tasks of the PP PA correspond to the clauses 4.1, 5.1, 5.4 and 7.1 of ISO 9001, as
listed on the second row on the same table.

On the same way, we summarize the result for all PAs of the CMMI model in Table 1 after careful
study, the cross mappings from CMMI process areas at different capability levels to the corresponding
clauses under Sections 58 in ISO 9001:2000. Naturally, the illustrated mapping between ISO
3

9001:2000 and CMMI seems a subjective association. Actually, we have concluded this result based on
our several experimental case studies, which have tried to verify its correctness and evaluate its
contribution. Due to the space limitation, the detailed information will not repeat here.
Generally, the ISO 9001:2000 allows an organization more flexible in the way chose to document
its quality management system. ISO 9001 does not contain any explicit requirements for the software
development process, because it was originally designed for application in a broad number of topics,
including development of products, systems or services. In a sense, this "flexibility" makes ISO 9001
quite difficult to implement. CMMI add value and detail to ISO 9001:2000 clause descriptions
(Mutafelija, 2001).

3.2 Primary & auxiliary process areas

According to the CMMI framework, the verification (VER) PA ensures that selected work
products meet the specified requirements. VER is generally an incremental process, starting with
product-component verification and usually concluding with verification of fully assembled products.
The validation (VAL) PA incrementally validates products against t
hec
u
s
t
ome
r

sne
e
d
sand may be
performed in the operational environment or a simulated operational environment.
With studying the generic goals of the VER and VAL PAs in a software development process,
some related process areas including RD, REQM, and TS as listed in (Chrissis, Konrad & Shrum, 2003)
are required to establish the baseline infrastructure. The RD PA is needed for the generation and
development of customer, product, and product-component requirements in order to validate
requirements, while the REQM PA aids for managing requirements. Moreover, the TS PA may provide
assistance to transform requirements into product specifications for the corrective action when
validation issues are identified to affect the product or product-component design. Thus, in this paper,
these five PAs will be regarded as the primary process areas to perform the required V&V tasks as
briefed in Table 3.
On the other hand, while in software project development, both the PP and MA PAs are usually the
key to successful implementation of a variety of process areas. Furthermore, from the perspective of
IEEE Std 1012-2004, some other PAs such as PPQA, CM, PI, PMC are essentially needed in
implementation the V&V tasks. In the following, PAs such as PP, MA, PPQA, CM, PI and PMC are
named the auxiliary process areas to suggest a requisite framework for an organization who has not yet
obtained ISO 9001 registration but desires to improve her software improvement in the interest of V &
V areas.

Table 1. Comparison of CMMI ML 2 -3 PAs and ISO Sections 58.

CMMI process
areas
Requirement
Management

Project Planning

Acronyms

PP

PMC

Supplier
Agreement
Management

SAM

Measurement and
Analysis

MA

Configuration
Management

PPQA

CM

Requirement
Development

RD

Technical Solution

TS

Product Integration

PI

Verification

VER

Validation

VAL

Organizational
Process Focus

OPF

Organizational
Process Definition

OPD

ISO 9001:2000 clauses

4.1 General requirements
4.2 Documentation requirements
7.3.2 Design and development inputs
7.3.A.1Design and development process management
4.1 General requirements
5.1 Management commitment
5.4 Planning
7.1 Planning of product realization
7.3.1.1Establishing design and development plan
4.1 General requirements
5.1 Management commitment
7.3.4Design and development review
7.6 Control of monitoring and measuring devices
8.2 Measurement and monitoring
4.1 General requirements
7.4 Purchasing
7.4.1Purchasing process
7.4.3Verification of purchased product
7.5 Production and service operations
8.2 Measurement and monitoring
8.2.3.1Monitoring and measurement of processes
8.2.3.2Monitoring and measurement of product
8.4 Analysis of data
8.4.A Measurement management
4.1 General requirements
5.1 Management commitment
5.2 Customer focus
5.3 Quality policy
8.2.2Internal audit
4.2 Documentation requirements
7.3 Design and/or development
7.3.7Control of design and development changes
7.5 Produuction and service operations
7.5.3Identification and traceability

REQM

Project Monitoring
and Control

Process and
Product Quality
Assurance

Capability level mapping

0 1 2 3 4 5

5.2 Customer focus

7.2 Customer-related processes
7.2.1Determination of requirements related to the product
7.2.2Review of requirements to the product
7.3 Design and/or development
7.3 Design and/or development
7.3.A.2Technical solution
7.3 Design and/or development
7.3.A.3Production integration
7.5.5Preservation and delivery of product
7.1 Planning of product realization
7.3 Design and/or development
7.3.5Design and development verification
7.5 Production and service operations
7.1 Planning of product realization
7.3 Design and/or development
7.3.6Design and development validation
5.5 Administration
8.2.2Internal audit
8.4 Analysis of data-?
8.5.2.1Deploying improvement
4.2 Documentation requirements
4.2.1General
4.2.2.1Organization
s set of standard process
4.2.2.2 Organization
s set of standard process tailoring criteria
and guidelines
4.2.5Process assets mangement
4.2.6Measurement management-MA?
5.3 Quality policy
5.4 Planning
5.5 Administration

6.2 Human resources

6.2.2Competence, awareness and training
4.2.3Control of documents
4.2.5Process assets mangement
5.4 Planning
5.5 Administration
7.1 Planning of realization processes
7.3.1.1Establishing design and development plan
7.3.1.2Team composition and operation
7.3.4 Design and development review
6.1 Provision of resources
7.4 Purchasing
7.4.1Purchasing process
7.4.2Purchasing information
7.4.3Verification of purchased product

Organization
Training

OT

Integrated Project
Management

IPM

Integrated Supplier
Management

ISM

Risk Management

RSKM

5.1 Management commitment

7.3.1.3Risk management
7.3.4Design and development review

4. V&V efforts under the second integrity level

In practice, software systems exhibit different levels of criticality based upon their intended
purposes and cost impact due to their system failures. To consider the trade-offs between the criticality
levels and the paid effort, software-development organizations may strategically choose a lower
integrity level to save the development effort (IEEE, 2004), if its cost impact, once it is happened, is
acceptable or negligible.
More detailed, software integrity levels denote a range of software criticality values that are
necessary to maintain risks within an acceptable limit. These software quality metrics include safety,
security, software complexity, performance, reliability, correctness or other characteristics. Generally,
critical and high-integrity software typically requires a larger set and more rigorous application of V&V
tasks. To identify the minimum V&V tasks that apply to the different integrity-level software systems,
the software developers may refer to the IEEE STD 1012-2004 for the complete list. The IEEE STD
1012-2004 is a technical discipline of systems engineering. The purpose of IEEE STD 1012-2004 is to
help the development organization build quality into the software during the software life cycle. In this
paper, we are limited ourselves to the non-critical commercial applications that exist the most popular
in the medium-scale organizations in Taiwan. Thus, for the system in non-critical uses, Table 2
delineates the minimum V&V tasks assigned to integrity level 2, in correspondence with the ISO
9001:2000 clauses and CMMI PAs and capability level (CL) as well.
In that table, we accept the general framework on dividing the whole software life cycle (SLC) into
5 periods: concept phase, requirement phase, design phase, implementation phase and test phase. For
the concept phase, the minimum V&V tasks include two tasks, i.e., the concept-documentation
evaluation and criticality analysis, which derive from the IEEE STD 1012-2004. Each task is further
corresponding to the associate clauses of ISO 9001. The final column of the table shows the
corresponding PA and its capability level that will be attained after the concerned task is performed. For
instance, the concept-documentation evaluation task in (IEEE, 2004) will perform the same effect as
both the REQA PA at the capability-level 3 and the PP PA at the capability-level 2 as well. In terms of
CMMI terminology, a capability-l
e
v
e
l2pr
oc
e
s
si
sc
ha
r
a
c
t
e
r
i
z
e
da
sa
ma
na
g
e
dpr
oc
e
s
s
,while a
capability-level 3 as a
defined process.
A critical distinction between a managed process and a defined process is the scope of application
of the process descriptions, standards, and procedures. For a managed process, the process descriptions,
standards, and procedures are applicable to a particular project, group, or organizational function. As a
result, the managed processes for two projects within the same organization may be very different.
Whereas, at the defined capability level, the organization is interested in deploying standard processes
that govern all related projects.
6

More detailed, the essential process elements for each minimum V&V tasks are investigated and
summarized in Table 2 in order to be ready for implementation.
Table 2. Minimum V&V tasks at the second integrity level.
SLC Phase

Concept

Minimum
V&V Tasks
Concept
Documentation
evaluation
Criticality Analysis

Acceptance V&V test

plan generation and
verification
Requirement

Criticality Analysis

Design

Implementation

ISO 9001:2000 Clauses

4.1 General requirements
4.2 Documentation requirements
7.1-Planning of product realization
7.2.1-Determination of requirements related to the product
7.2.2-Review of requirements related to the product
7.3.A.1Design and development process management
5.2 Customer focus
5.3 Quality policy
5.4.1-Quality objective
5.4.2-Quality management system planning
7.2.1- Determination of requirements related to the product
7.5.3-Identification and Traceability
7.2 Customer-related processes
7.2.1Determination of requirements related to the product
7.3.2-Design and development inputs
7.3.3-Design and development outputs

Component V&V test

7.3.1-Design and development planning
plan generation and
7.3.5-Design and development verification
verification
7.2.2Review of requirements to the product
Criticality Analysis
7.3.4-Design and development review
Component V&V test
7.3.1-Design and development planning
execution and
7.3.6-Design and Development Validation
verification
Criticality Analysis 7.3.7-Control of design and development changes

Test

Acceptance V&V test

7.5.1-Control of production and service provision
execution and
8.2 Measurement and monitoring
verification
5.4.1-Quality objectives
Acceptance V&V test 5.4.2-Quality management system planning
procedure generation 7.3.6-Design and Development Validation
and verification
7.5.2-Validation of process for production and service
provision

associated
CMMI PAs & CL
REQM (CL4)
PP (CL3)
REQM (CL4)

PPQA (CL4)
PP (CL3)

RD (CL4)

VER (CL4)
PP (CL3)
RD (CL4)
VER (CL4)
VAL (CL3)
CM (CL4)
PI (CL3)
CM (CL4)
PMC (CL3)
VER (CL4)

VER (CL4)
VAL (CL3)
PP (CL3)

5. Proposed roadmap to process improvement

Naturally, there are many ways to enhance software process improvement within a software
organization by the approach of implementing CMMI model. In the following, we propose a practical
and systematic sequence for those middle-scale software organizations, from the perspective of
minimum V &V effort at the second integrity level to save the software development cost. At this stage,
we set the goal of software improvement from none to the third capability-level with the purpose of
making the proposed roadmap is easily implemented for an organization that starts to employ the
philosophy of CMMI model.
In general, the amount of V&V effort required for software project depends on its performance
requirement and naturally, it does not related directly to the size of a software organization. It is noted
that software-integrity levels relate to the project criticality instead of software organization. In this
paper, we have observed that most local middle-scale software organizations in Taiwan area are
developing non-critical business projects. That is why we are concerned with the software projects of
the integrity-level 2, although some higher integrity-level software of specific domain are actually kept
in a small size to reduce V&V cost and carried by a relatively small team.
7

Furthermore, in considering with some software-related organizations in Taiwan, which have been
ISO 9001:2000 registered, the suggested roadmap may have two different options basing on their
current situations as illustrated in Fig. 1 and the proposed target profile in Table 3. As shown in Fig. 1, a
software organization that has been ISO-9001 registered may directly perform the primary process
areas (i.e., RD, REQM, TS, VER and VAL) to benefit her achievement from ISO efforts. Taking
REQM as an example, she needs to enhance the concept-documentation evaluation task by improving
her original 7.1 clause (Planning of product realization) to the capability level 4, as illustrated on the
first row of Table 2. Alternatively, if a software organization has not ever practices of ISO-9001 but
desires to implement her process improvement through the continuous CMMI model, she has to start
with those auxiliary process areas as suggested in Fig. 1. Accordingly, the proposed roadmap provides
an obvious shortcut to enhance process improvement for the ISO-9001 registered software
organizations.

CL 0
ISO 9001:2000
Registered?
YES
Equivalent
ML3

Level 5
Level 4

Capability
Level

CL 0

Level 3
Level 2

ISO 9001:2000
Registered?
NO

Level 1
Level 0

VAL
RD

REQM

TS

VER

Primary process areas

Level 5
Level 4
Level 3
Level 2
Level 1
Level 0

MA
PP

PPQA

PI

CM

PMC

Auxiliary process areas

Fig. 1 The proposed roadmap for minimum V&V tasks at the second integrity level

Table 3. Target profiles for V&V efforts with the second integrity level.
Category

PA

CL1

CL2

CL3

CL4

CL5

RD
REQM

Primary

TS
VER
VAL
PP
MA
PPQA

Auxiliary
CM
PI
PMC

Dug into a software life cycle, the proposed framework identifies the most important tasks for
performing the minimum V&V tasks in order to ensure the developed software in accordance with
functional specifications and customer
s expected performance. Emphasis on the V&V tasks lies from
8

the fact that identification and correction of errors early in the development cycle are less costly than
that in later phases, and thus the quality of software are significantly improved. As a result, a middle
scale organization will benefit greatly from its software process improvement by the proposed
budget-acceptable, feasible and effective approach.
6. Conclusion
To learn lessons from the common practices on the previous projects in order to enhance process
implication, the proposed roadmap will be much helpful to the ISO 9001 registered organizations.
Furthermore, the roadmap will assist the implemented organizations in performing gap analysis and
maintaining their quality manual without any difficulty while adopting the CMMI model. With the
desire to making the suggested roadmap easily implemented for an organization starting to employ
philosophy of the CMMI model, we set the target goal of improvement path from none to the third
capability level in the continuous representation CMMI model.
In summary, the proposed framework establishes the bottom line to be performed for software
process improvement in a software organization. Within a software project life cycle, the effort on
verification and validation is highly emphasized to ensure that both quality control and quality
assurance are implemented as scheduled plans. Based on several observations on practical application
demonstration, significant improvements have been found over some interested metrics such as
productivity, defect injection rate and defect removal rate. The benefit of our proposed roadmap
provides an effective, efficiency and economical approach no matter the middle-scale company is ISO
9001:2000 registered or not yet.

References
[1]

Ahern, D.M., Aaron Clouse and Richard Turner (2001). CMMI Distilled. Addison-Wesley.

[2]

Carter, L., C. Graettinger, M. Patrick, G. Wemyss, S. Zasadni (2002). The Road to CMMI: Results
of the First Technology Transition Workshop. CMU/SEI-2002-TR-007, Pittsburgh, PA: Software
Engineering Institute, February.

[3]

Chrissis, M.B., M. Konrad, S. Shrum. (2003). CMMI: Guidelines for Process Integration and
Product Improvement, Addison Wesley.

[4]

IEEE. (2004). STD 1012-2004, IEEE Standards for Software Verification and Validation.

[5]

ISO. (1995). ISO/IEC 12207: 1995, Information TechnologySoftware Life Cycle Processes.

[6]

ISO. (2001). Software and Systems EngineeringGuidelines for the Application of ISO
9001:2000 to Software, TC/SC SC7 WG18N61, ISO/IEC CD 9000-3, and Version for FCD Ballot,
Geneva, Switzerland: ISO.

[7]

ISO. (2003). ISO/IEC 15504-2:2003. Information Technology-Process Assessment-Part 2: A

Reference Model for Process and Process Capability, Geneva, ISO.

[8]

Mutafelija, B. (2001). Software Process Improvement: Synergy between ISO 9001:2000 and
CMMI, in: SEPG Conference, New Orleans, LA.

[9]

Paulk, M.C. (1995). How ISO 9001 Compares with the CMM. IEEE Software, January, 7483.

[10] Yoo, C., J.Yoon, B. Lee, C. Lee, J. Lee, S. Hyun, and C. Wu (2004). An Integrated Model of ISO
9001:2000 and CMMI for ISO Registered Organizations. 11th Asia-Pacific Software Engineering
Conf
e
r
e
nc
e(
APSEC
0
4)1
5
0-157.
[11] Yoo, C., J.Yoon, B. Lee, C. Lee, J. Lee, S. Hyun, and C. Wu (2006). A unified model for the
implementation of both ISO 9001:2000 and CMMI by ISO-certified organizations. Journal of
Systems and Software, Web published, June.
9