BISE – RESEARCH PAPER

Developing Maturity Models for

IT Management – A Procedure
Model and its Application
In IT management, maturity models have proved to be an important instrument because
they allow for a better positioning of the organization and help find better solutions for
change. Over the last few years, over a hundred maturity models have been developed to
support IT management. However, the procedures and methods that led to these models
have only been documented very sketchily. Using a scientific approach, we have developed
criteria for the development of maturity models. These criteria also serve as a basis for
the comparison of sparsely documented maturity approaches. The results thus obtained
have been generalised and consolidated into a generally applicable model. A case study
will illustrate the applicability of our model. The results of this paper are meant to serve as
a manual for methodically well-founded designs and evaluations of maturity models.

DOI 10.1007/s12599-009-0044-5

1 The importance of maturity company, derive and prioritize improve-

The Authors models for IT management ment measures and subsequently control
the progress of their implementation.
Prof. Dr. Jörg Becker IT support of business processes has Maturity models are helpful tools for
Dr. Ralf Knackstedt become indispensible for a lot of compa- addressing these issues (de Bruin et al.
Dipl.-Wirt. Inform. Jens Pöppelbuß nies (Müller et al. 2006, p. 101). Moreover, 2005). A maturity model consists of a
Westfälische Wilhelms- innovative IT systems generally offer great sequence of maturity levels for a class
Universität Münster opportunities to improve a company’s of objects. It represents an anticipated,
European Research Center competitiveness (Henderson and Venka- desired, or typical evolution path of these
for Information Systems traman 1993; McFarlan and Nolan 2003; objects shaped as discrete stages. Typi-
Leonardo-Campus 3 Pößneck 2007). The responsibility for cally, these objects are organizations or
48149 Münster effective and efficient design and use of processes. The bottom stage stands for an
Germany IT lies with the company’s IT manage- initial state that can be, for instance, char-
{joerg.becker | ralf.knackstedt | jens. ment. The main goal here is to continually acterized by an organization having little
poeppelbuss}@ercis.uni-muenster.de improve IT performance with regard to its capabilities in the domain under consid-
economic efficiency. eration. In contrast, the highest stage rep-
Received: 2008-08-04 Continual improvement requires the resents a conception of total maturity.
Accepted: 2009-03-06 company’s positioning with regard to its Advancing on the evolution path between
Accepted after four revisions by IT capabilities and the quality of its goods the two extremes involves a continuous
the editors of the special focus. and services. As a rule, this positioning progression regarding the organization’s
involves a comparison with the compa- capabilities or process performance. The
This article is also available in Ger- ny’s goals, external requirements (e. g. maturity model serves as the scale for the
man in print and via http://www. customer demands, laws or guidelines), appraisal of the position on the evolution
wirtschaftsinformatik.de: Becker J, or benchmarks. To achieve an objective path. It provides criteria and character-
Knackstedt R, Pöppelbuß J (2009) assessment of a company’s position, how- istics that need to be fulfilled to reach a
Entwicklung von Reifegradmodellen ever, often proves to be a difficult task. particular maturity level. During a matu-
für das IT-Management – Vorgehens- For each aspect of the company’s IT under rity appraisal, a snap-shot of the organiza-
modell und praktische Anwendung. investigation, the questions arise what tion regarding the given criteria is made.
WIRTSCHAFTS­INFORMATIK . doi: needs to be measured and how, and what The characteristics found are evaluated
10.1007/11576-009-0167-9. to compare it with, in order to assess the to identify the appropriate organization-
as-is situation of a company and to assign individual maturity level. The application
it a specific quality or degree of maturity. of maturity models can be supported by
IT management therefore needs support- predetermined procedures, e. g. by ques-
ive tools to assess the as-is situation of a tionnaires. Based on the results of the as-is

Business & Information Systems Engineering 3 | 2009 213

BISE – RESEARCH PAPER

analysis, recommendations for improve- in favor of our choice of argumentative problem of delimiting the evaluation
ment measures can be derived and priori- premise. Despite the absence of empirical criteria, see Zelewski 2007, pp. 92–93).
tized in order to reach higher maturity lev- research in this field, it is probably safe to The last requirement is also emphasized
els (IT Governance Institute 2007). speculate that future reviewers of publica- in “Guideline 3: Evaluation.” Hevner et al.
Studies have shown that more than a tions on maturity models will, for want of (2004) point out that the evaluation of all
hundred different maturity models have well-established alternatives, increasingly (intermediary) results must be carried out
been proposed (de Bruin et al. 2005). The rely on these assessment criteria (Zelewski with appropriate scientific methodology.
constant publication of new maturity 2007, pp. 111–114). Accordingly, the criteria Since different methods may be used in
models for often fairly similar applications established by Hevner et al. (2004) suggest the development of the artifact and in
however suggests a certain arbitrariness. themselves as a basis for the development the evaluation of the particular results,
The authors only rarely reveal their moti- of maturity models. design science characteristically adopts a
vation and the development of the model, By adopting these criteria, we do not, multi-methodological procedure (Hevner
or their procedural method and the results however, wish to abet the confrontation of et al. 2004). As accentuated in “Guideline
of their evaluation. behavioral and design science introduced 5: Research Rigor,” the selected methods
The aim of this paper is therefore to pro- by Hevner et al. (2004), nor do we wish, need to be rigorously attuned. For our
pose a procedural model for the design of at the exclusion of other paradigms, to purposes, this yields the following
maturity models hoping to remedy these assign the development of maturity mod- requirement:
widespread shortcomings. In a first step, els exclusively to design science. For our j R4 (Multi-methodological Procedure):
the requirements necessary for the devel- further line of argument, the reasonable The development of maturity models
opment process of maturity models will be applicability of the criteria to the develop- employs a variety of research meth-
identified (section 2). On the basis of these ment of maturity models will be sufficient. ods, the use of which needs to be well-
requirements, the few well-documented Particularly Zelewski’s criticism of some founded and finely attuned. (Zelewski
development processes of maturity mod- guidelines (Zelewski 2007, pp. 91–103) (2007, p. 98) points at the difficul-
els will be compared (section 3). The has in some cases motivated the slightly ties of operationalising research rigor
results from section 3 provide the ground- divergent interpretation of Hevner’s et al. and proposes an ontological approach
work for the construction of a procedure (2004) criteria and their considered adap- (Frank 2004, p. 377) to address those.)
model for the development of maturity tation to the design of maturity models. “Guideline 2: Problem Relevance” states
models (section 4). In section 5, the pro- The aim of design science is to develop that the problem-solving artifact must not
cedure model will be illustrated by the an innovative problem-solving artifact only be innovative, but the problem to be
development of a maturity model for the (“Guideline 1: Design as an Artifact”) solved must also be relevant for research-
implementation of IT performance mea- that will contribute to current research ers and/or practitioners. This relevance
surement supported by BI tools. Finally, (“Guideline 4: Research Contributions”). can again be established through different
an outlook on further research will com- For the development of maturity models scientific methods, e. g. by interviewing
plete this article (section 6). this means: potential users of the maturity model
j R1 (Comparison with existing maturity in question. Establishing relevance
models): The need for the development also requires the exact definition of the
2 Requirements for the of a new maturity model must be sub- problem, which in turn is prerequisite
development of maturity models stantiated by a comparison with exist- for ensuing evaluations. Accordingly, the
ing models. The new model may also following requirements are considered:
In order to establish a reasonable cata- just be an improvement of an already j R 5 (Identification of Problem Rele-
logue of requirements for the design of existing one (Zelewski 2007, pp. 93– vance): The relevance of the problem
maturity models, the seven guidelines for 98). solution proposed by the projected
design science defined by Hevner et al. The process description of design science maturity model for researchers and/or
(2004) have been chosen as the basis for requires an iterative procedure for the practitioners must be demonstrated.
our argument. Design science aims at the development of a problem solution (Pef- j R6 (Problem Definition): The prospec-
improvement of problem-solving capa- fers et al. 2007). “Guideline 6: Design as a tive application domain of the maturity
bilities by creating innovative artifacts, Search Process” also emphasizes that, by model, as well as the conditions for its
such as constructs, models, methods and using the available means, solutions must application and the intended benefits,
instantiations (March and Smith 1995). be iteratively proposed, refined, evaluated, must be determined prior to design.
Thus, maturity models may be under- and, if necessary, enhanced. For the design Documentation of the research process
stood as artifacts which serve to solve the of a maturity model this means: is of vital importance for the scientific
problems of determining a company’s j R 2 (Iterative Procedure): Maturity procedure. “Guideline 7: Communica-
status quo of its capabilities and deriving models must be developed iteratively, tion of Research” emphasizes that the
measures for improvement therefrom. It i. e., step by step. presentation of results must be targeted
can therefore be assumed that the devel- j R 3 (Evaluation): All principles and at the specific user groups. It must, how-
opment of maturity models falls within premises for the development of a ever, be pointed out here that, unlike the
the application area for the guidelines maturity model, as well as usefulness, practitioners (technicians and managers)
developed by Hevner et al. (2004). The quality and effectiveness of the artifact, Hevner et al. (2004) focus on, researchers
considerable attention that the paper by must be evaluated iteratively (for the make specific demands not only on the
Hevner et al. (2004) has received, speaks presentation of results, but also on the

214 Business & Information Systems Engineering 3 | 2009

 Building Global Leaders

Business oder Technologie?

Wieso oder?
Oder kennen Sie ein Business,
das ohne Technologie auskommt?
McKinsey sucht Berater(innen) für das Business Technology Office.

Der effektive Einsatz von Technologie

löst unternehmerische Probleme und
steigert somit den substanziellen Wert
eines Unternehmens nachhaltig. Genau
das ist das Ziel unserer Beratung. Wir
helfen unseren Klienten, Investitionen
in Technologie erfolgreich zu managen
und mit den strategischen Prioritäten
des Geschäfts in Einklang zu bringen.
Wenn Sie diese vielfältige Herausforde-
rung annehmen, werden Sie auf über
500 Kollegen an rund 50 Standorten
in mehr als 25 Ländern treffen.
Und werden damit Teil des globalen
Business Technology Office – eines der
größten und am schnellsten wachsen-
den Büros von McKinsey & Company.
Alle weiteren Informationen finden Sie
auf unserem Karriereportal.

bto.mckinsey.de
BISE – RESEARCH PAPER

Tab. 1 Evaluation of maturity models on the basis of scientific documentation Maturity Model (DPMM), which is also
Criteria for scientific documentation Value of documentation quality included in our study. This model focuses
on documentation as an important sup-
R8-I documentation includes reference to existing models: - + - + +
In describing the design process reference is made to existing porting factor in software development
maturity models (generally Capabiltiy Maturity Model). (Cook and Visconti 2000). Another
R8-II documentation indicates steps of - - + + + model included is the E-Learning Matu-
design and evaluation processes: rity Model (eMM), published by the Victo-
The documentation states that different stages of the design
process were discussed. This may include, e. g., workshops with ria University of Wellington and designed
the designers, experts, and practitioners or single case studies. to help colleges and other institutions to
R8-III detailed documentation of the design process: - - - - + assess their capabilities with regard to a
The individual phases of the model design sustained development, introduction and
are presented clearly and in detail.
use of e-learning and to compare their
Number of Maturity Models (n=51) 15 13 3 11 9 results with other institutions (Marshall
Percentage of Maturity Models (n=51) 29.4 25.5 5.9 21.6 17.6 2007). Finally, the IS/ICT Management
Key: [-] Requirement R8-x not met; [+] Requirement R8-x met Capability Maturity Framework (IC/ICT
CMF) represents a maturity model for IT
management (Renken 2004).
documentation of the research process. only little information about the design In spite of their comparatively good
Correspondingly, the two following process, we asked model designers per e- documentation, the following models
requirements need to be included in our mail to advise us of potential publications have been excluded from the synopsis: the
catalogue: on the subject. This, however, led in only Business Process Maturity Model (Lee et
j R7 (Targeted Presentation of Results): a few cases to an improvement of the data al. 2007), which is less well documented
The presentation of the maturity available. The maturity models have been than the BPMM, a maturity model from
model must be targeted with regard to described according to requirement R8 the same domain; the Capability Maturity
the conditions of its application and the using three criteria which may also work Model (CMM; Paulk et al. 1993), since it is
needs of its users. in combination (see Tab. 1). Our results the precursor of CMMI; and the Knowl-
j R8 (Scientific Documentation): The show that the documentation quality of edge Management Capability Assess-
design process of the maturity model these models is generally fairly patchy (for ment (KMCA; Freeze and Kulkarni 2005;
needs to be documented in detail, con- more detail see Becker et al. 2009). Kulkarni and Freeze 2004), since the main
sidering each step of the process, the For the appraisal of the remaining phases of its development process match
parties involved, the applied methods, requirements only those models can be those of the BPMM (de Bruin et al. 2005).
and the results. considered that comply with requirement It is noteworthy that, for all six mod-
In the following section, a comparison R8-III. On the basis of this appraisal, six els, a screening of existing maturity mod-
will show to what extent existing matu- maturity models were chosen and synopti- els was conducted prior to development.
rity models meet these requirements. On cally reviewed with regard to requirements Similarly, an iterative procedure can be
the basis of these results, a generically R1 to R7 (see Tab. 2). Naturally, require- observed in each case, in which particu-
applicable model for the development ment R8 will not be reappraised here. larly case-study evaluations of intermedi-
of maturity models will be extracted, One of the selected models is the Anal- ate versions (ACMM, eMM) led to subse-
enabling designers of maturity models to ysis Capability Maturity Model (ACMM), quent modifications of the model. Com-
fulfil the requirements established above which has been developed for the US- prehensive literature research invariably
in the course of the design process. American National Reconnaissance formed the basis for the core elements of
Office (NRO). It has been designed to eval- all maturity models, and was frequently
uate processes of organizations that con- complemented by the consultation of
3 Comparison of design processes duct state-commissioned studies (Covey domain experts (DPMM, IS/ICT CMF).
of specific maturity models and Hixon 2005). The second model is the In individual cases, the problem relevance
maturity model of Rosemann et al. (2006), was ascertained by specific assignments,
An essential prerequisite for a comparison which investigates Business Process Man- but in general it was based on more generic
of design processes of specific maturity agement Maturity (BPMM). The design- derivations. The problem definitions tend
models can be found in requirement R8 ers emphasize that their model must com- to foreground the evaluation and the com-
(Scientific Documentation). Only maturity ply with scientific standards (de Bruin and parison of businesses with regard to their
models for which a detailed documentation Rosemann 2007; de Bruin et al. 2005). capabilities in specific domains – nota-
is available can be effectively compared. The third model included in our synop- bly in the domain of IT management. The
In order to identify adequate objects for sis is Capability Maturity Model Inte- manner in which the results are presented
the comparison, 51 maturity models were gration (CMMI), which integrates sev- varies considerably and ranges from a sin-
culled from the internet and pertinent eral models that evolved from the context gle conference paper (IS/ICT CMF) to
literature, and then analyzed. Each of of the initially very popular Capability reports and procedure reports of several
these models was checked for free and Maturity Model (CMM; CMMI Prod- hundred pages (CMMI). Discretionary
publicly available information about the uct Team 2006; Paulk et al. 1993). On the questionnaires for self-assessment stand
design process (as of July 2008). In order basis of CMM, Cook and Visconti started out favorably (DPMM, eMM).
to be able to include models that offered in 1992 to develop the Document Process

216 Business & Information Systems Engineering 3 | 2009

 BISE – RESEARCH PAPER

Tab. 2 Synopsis of Design Processes of Maturity Models

Requirement Analysis Business Process Capability Maturity Documentation E-Learning Maturity IS/ICT Capability
Capability Maturity Management Maturity Model Integration Process Maturity Model (eMM) Maturity Framework
Model (ACMM) (BPMM) (CMMI) Model (DPMM) (IS/ICT CMF)
Comparison Comparison with CMMI Analysis of existing, Transfer of Crosby’s Based on Software Adoption of Analysis of IT
with existing Using CMMI-stages but unsatisfactory Quality Maturity Process Maturity concepts from management matu-
maturity as pattern maturity models in Grid to software Framework and CMM and SPICE rity models (Nolan’s
models (R1) Business Process development CMM (Software Process Stage Theory, CMM,
Management Improvement Strategic Grid)
and Capability
Determination)
Iterative Development of Derivation of four Initially Discussion of Application in Identification of
Procedure (R2) a first model with dimensions development of preliminary versions case studies and initial indicators
literature research and “Factor”-dimension Software Process with software workshops led through literature
expert interviews initially comprises Maturity Framework engineers to modifications research
Integration of process five factors Further develop- Iterative cycle of model Iterative modeling
areas from CMMI Use of Delphi studies ment into CMM of (further) to eliminate and
Model modification on for the determination Integration of other development and combine indicators
the basis of case study of “factors” and subor- models into CMMI application led Validation through
dinate capability areas Review of inter- to four versions interviews
mediary versions (in 1993, 1995, Final adjustments
1996, and 2000)
Evaluation Application by Application in case Preliminary versions Feedback from Validation of first Semi-structured
(R3) independent persons studies over two years of CMM were software industry version at New interviews with
in case study Explorative studies made accessible Practical Zealand university regard to indicators
No evaluation of matu- Use of Delphi studies for review applications Workshops in Perspective: quan-
rity levels four and five for the determination Discussion of CMM Additional Australia and titative empirical
of model components v1.0 in a workshop validation by Cost Great Britain study for validation;
with ca. 200 experts Construction Model Applications implementation of
Widespread applica- (Boehm 1981) in several an assessment tool;
tion by practitioners organisations benchmarking study
Multi-metho- Literature research Literature research Literature research Analysis of other Literature research Literature research
dological on phases of Delphi method on product quality theories and on e-learning Iterative modeling
Procedure (R4) analytical processes Expert interviews models in software processes Semi-structured
Expert interviews development interviews
Discussion of
model with
software industry
Identification Design commissioned According to several Originally Empirical data show Only few empirically Other authors point
of Problem by US American studies, process government absent or deficient grounded findings out the need for a
Relevance (R5) National Reconnais- management is an commission for the documentation to on the success and combination of the
sance Office (NRO) important issue for generation of tools be a major cause failure of e-learning fragmented methods
companies; they for the evaluation of for shortcomings initiatives for the improvement
seek tools to support software businesses of software of IT management
their initiatives engineering
Problem Evaluation of Classification of Development Evaluation and Support of a IT management
Definition (R6) organizations businesses with of a tool for the improvement of the comparison of faces new and
providing analyses regard to their evaluation of documentation of of colleges etc. complex challenges
process management software businesses software develop- Knowledge transfer Methods for support
capabilities ment to improve with regard to of IT management
software quality e-learning are fragmented
Prioritization of
investment in
e-learning systems
Targeted 66 pp report incl. Academic publications 573 pp report Web page including Comprehensive Only conference
publication of description of model No complete Separate 246 pp explanatory notes web page paper
results (R7) and design procedure description of description of and questionnaire Description of
model and design appraisal procedure Ca. 15 publications model and design
procedure so far (SCAMPI) from 1992 to 2002 procedure
Excel questionnaire
Examples

4 Procedure model for the procedure model, which is based on a flow ment processes and illustrates possible
development of maturity models chart (see DIN 1966), notations for refer- procedures for each phase by examples
ences to requirements R1R7 have been from the synopsis.
In the following, we propose a proce- applied to the individual procedure model According to R6, the procedure model
dure model that distinguishes eight phases elements. Requirement R8 has been incor- starts with the problem definition. All
in the development of maturity models porated by identifying the documents gen- reviewed models start by defining the
(see Fig. 1). The elements of this model erated in the course of the maturity model problem. For this purpose both the tar-
are informed by the requirements iden- design, referenced by the document sym- geted domain (e. g. IT management as a
tified above and by correspondent proce- bol which has been assigned to R8 in the whole (see IS/ICT CMF) vs. partial disci-
dures from the well-documented exam- caption. Moreover, the model generalizes pline (see CMM, DPMM)) and the target
ples. In the graphic representation of the the reviewed well-documented develop- group (e. g. intra-corporate vs. external)

Business & Information Systems Engineering 3 | 2009 217

BISE – RESEARCH PAPER

may be organized hierarchically (BPMM).

After this basic structural design, the indi-
vidual dimensions and their attributes
must be devised to flesh out the model
architecture.
According to R4, appropriate methods
have to be chosen for each abstraction
level. Widespread is the use of literature
analysis (eMM), which extracts assess-
ment criteria for the maturity model from
success factors and typical developments.
Explorative research methods, like, for
instance, the Delphi method (BPMM),
and creativity techniques (e. g., the iter-
ative consolidation of indicators in IS/
ICT CMF) are also suitable. In the next
step, the selected model section needs to
be designed in accordance with the cho-
sen procedure. In order to comply with
R3 (evaluation), the result then must be
tested for comprehensiveness, consistency,
and problem adequacy. The result of this
evaluation will decide on the sequel of the
design procedure.
In the phase following the design of
the maturity model, conception of trans-
fer and evaluation, the different forms
of result transfer for the academic and
the user communities need to be deter-
mined. Requirement 4 prescribes a rea-
soned selection of the different forms that
Fig. 1 Procedure model for developing maturity models the targeted communication of the matu-
rity model can take. Besides the wide-
of the maturity model need to be deter- A comprehensive comparison is req- spread publication of document-based
mined. uisite for a reasoned determination of the check lists (DPMM, eMM) and manuals
At the same time, according to R5, the design strategy, which, according to R8, (CMMI), software-tool supported accessi-
problem relevance, i.e. the actual demand needs to be documented as well. The most bilty of the maturity model (e. g. via inter-
for the maturity model, must be clearly important basic strategies that can be dis- net) offers another alternative. Possiblities
demonstrated. The reviewed models, cerned are: the completely new model for the evaluation of the problem solution
however, generally explain this demand design, or the enhancement of an existing proposed by the maturity model should be
only by pointing out the relevance of the model (CMM); the combination of sev- incorporated into the transfer design. This
targeted domain, except for ACMM and eral models into a new one (CMMI); as requirement guarantees users the possi-
CMMI, which were initiated by state com- well as the transfer of structures (DPMM, bilty for feedback as early as the design
missions. eMM) or contents (ACMM, IS/ICT CMF) stage of the media (e. g. questionnaires or
Although varying in scope, a compari- from existing models to new application forms for change requests in the manuals,
son of existing maturity models, as stipu- domains. comprehensive data collection via soft-
lated in R1, can be found in all reviewed The central phase of the procedure ware tools). If the evaluation includes the
examples. Shortcomings or lack of trans- model is the iterative maturity model differentiation between groups, the trans-
ferability often motivate improvements of development, which reflects requirement fer concept must allow for the possibilty
older models (ACMM, BPMM). Occasion- R2. The sub-steps of this phase, select- to discriminate, e. g., between an exper-
ally, a renewed search for models already ing the design level, selecting the approach, iment group and a control group. The
addressing these problems is omitted. designing the model section, and testing reviewed design projects have made com-
Apart from active research for existing the results will be iterated. The design paratively little use of these integration
models that is motivated by the problem level with the highest degree of abstrac- potentials. For the IS/ICT CMF, a quanti-
definition, the publication of a new matu- tion provides the architecture, i. e. the fun- tative analysis via a software-based assess-
rity model by a third party may motivate damental structure of the maturity model. ment tool was announced, but could not
a comparison (reactive initiation). In this Besides the one-dimensional sequence of be retrieved.
case, examination of the new model may discrete steps (DPMM), a multi-dimen- The purpose of the phase implementa-
provide an incentive for modifications of sional maturity assessment is common tion of the transfer media is to make the
one’s own maturity model. (IS/ICT CMF). The different dimensions maturity model accessible in the planned

218 Business & Information Systems Engineering 3 | 2009

 BISE – RESEARCH PAPER

fashion for all previously defined user Deloitte Consulting GmbH for the devel- maturity models could be detected that
groups. At this stage, the most impor- opment of a maturity model designed to view IT management and BI as separate
tant point is to target the transfer media, assess the application of IT performance fields.
as specified in requirement R7. In the measurement. The development of the Among the maturity models that refer to
reviewed projects, voluminous reports IT Performance Measurement Maturity IT management in a broader sense are, e. g.,
prevail (ACMM, CMMI). Self-assessment Model (ITPM3) is aimed at providing a the Capability Maturity Model (CMM) and
questionnaires are sometimes available tool for the structured enhancement of its further development, Capability Matu-
(DPMM, eMM), but are, for commercial Business Intelligence (BI) applications for rity Model Integration (CMMI Product
reasons, often not made generally acces- the control of IT-formative and IT-sup- Team 2006; Paulk et al. 1993); the matu-
sible (e. g. when management consultants ported processes in a company. rity models designed for the COBIT pro-
develop maturity models for their own cesses (IT Governance Institute 2007); the
business). 5.1 Problem definition IT Balanced Scorecard Maturity Model
According to requirement R3, the eval- (Van Grembergen and Saull 2001); and
uation should establish whether the matu- Using ratios and ratio systems, IT Perfor- the IS/ICT Management Capability Matu-
rity model provides the projected benefits mance Measurement aims at providing rity Framework (Renken 2004).
and an improved solution for the defined a comprehensive representation of a These maturity models generally disre-
problem. The defined goals are to be com- company’s IT. In alignment with the gard computer-based support of IT man-
pared with real-life observations. For this company’s overall strategy, it tracks and agement that, for instance, BI systems
purpose, the reviewed projects have car- monitors the implementation of the IT would afford. There are, however, sepa-
ried out case studies, in which a small strategy, the realization of projects, the use rate maturity models for the latter, e. g. the
exclusive group applies the new maturity of resources, the process performance and two Business Intelligence Maturity Models
model (BPMM). Alternatively, the model IT performance (IT Governance Institute by Chamoni and Gluchowski (2004) and
may be made accessible on the internet for 2007). Besides the traditonally strong by Eckerson (2006, pp. 89–95), as well as
free access. This has the advantage that a focus on costs, particular attention is here the Maturity Model for Performance Mea-
great number of users, e. g. via web-based also given to the performance and value of surement Systems by Wettstein and Küng
self-assessment, will generate a great num- IT (Bendl et al. 2004; Eul et al. 2006). (2002).
ber of datasets which can be compared Business Intelligence Systems (BI sys-
with the expected distribution of matu- tems) can collect internal cost and perfor- 5.3 Determination of development
rity levels in business reality. mance data as well as external market data strategy
The outcome of the evaluation may from a variety of sources, process these
cause a reiteration of the design process data into significant information about The comparison of existing maturity
(R2). It is also possible that the maturity IT performance and support the IT man- models with the problem definition
model may be retained unchanged, while agement by supplying this information for suggests a design strategy that combines
the conception of transfer and evaluation executive tasks (Chamoni and Gluchowski several models into a new one. We decided
may need to be modified. Lastly, nega- 2004). So far, in comparison with other to use the identified maturity models as a
tive results may lead to a rejection of the business areas, IT management has been starting point for the design process, since
model, in which case the model should be supported very little by BI systems (Cham- they already covered aspects of BI sup-
purposefully, explicitly, and if possible, oni and Gluchowski 2004, p. 125). Thus, ported IT performance measurement. On
actively taken off the market. the ITPM3 is meant to serve as an instru- the one hand, their graded structure and
Maturity models inherently become ment that calls attention to this possible their differentiation of individual maturity
obsolete because of changing conditions, neglect of BI for the purposes of IT man- levels by key areas provided basic solutions
technological progress or new scientific agement and, in the process of a position- for the structuring of the maturity model.
insights. If an unchanged maturity model ing, shows ways to improve the status quo. On the other hand, it was expected that
is supposed to be permanently valid for its The relevance of the proposed model has parts of the contents (e. g., descriptions
problem area, it needs to be validated reg- been confirmed in interviews conducted of individual maturity degrees) could be
ularly by appropriate evaluations. Mod- in the first half of 2007 with ten IT man- applied, or at least expediently transferred,
ifications that may become necessary agement representatives of German com- to our problem area.
in time, can be accommodated in a new panies. All reviewed companies strive for
model version. Several existing models the improvement of their IT performance 5.4 Iterative maturity model
may become invalid if replaced by an inte- measurment. The identification of present development
grated model as was the case with CMMI. shortcomings and future fields for action
Pertinent decisions need to be communi- by means of a maturity model was broadly The maturity model design underwent five
cated to potential users. welcomed. iterations. In the first iteration, a primary
architecture was drafted, which allowed
5.2 Comparison of existing maturity for four development stages: fragmented
5 Application of procedure model models IT reporting, consolidated spreadsheet-
assisted IT reporting, IT performance
The procedure model is currently used Maturity models that explicitly address BI dashboard, and information portal for
by the European Research Center for supported IT performance management IT management. Following the biMM by
Information Systems (ERCIS) and the could not be identified. Instead, several Chamoni and Gluchowski (2004), these

Business & Information Systems Engineering 3 | 2009 219

BISE – RESEARCH PAPER

Fig. 2 The ITPM3 and influences of existing maturity models

stages were described by the dimensions architecture was therefore not necessary and architectures that are employed. The
of contents, organization, and technology. in the fifth iteration. Only final modifica- criteria factor in more specific questions
Based on an extensive literature research, tions of the criteria for the dimension con- of each dimension. Apart from the grada-
the dimensions and their attributes were tents were carried out for the – at present – tion into maturity levels and the dimen-
defined during the second iteration. At last version of ITPM3. sions, single criteria like cost-benefit anal-
this stage, nine semi-structured inter- The model, which has been released ysis have been adapted form other matu-
views with IT managers from German for externally targeted transfer and sub- rity models. Further criteria are based
companies (duration: ca. 45 to 60 min.) sequent evaluation, describes the analyti- on the maturity attributes of the generic
confirmed the plausibilty of our model, cal BI process that transforms fragmented COBIT maturity model (such as policies,
whereas the strong orientation towards internal and external data into action-ori- standards, and procedures). The maturity
the technical aspects of IT performance ented information about the efficiency model for the COBIT process, ME1 (IT
management was criticized. To enable a and effectiveness of the company’s infor- Governance Institute 2007), and the IT-
more differentiated mapping of weakly mation infrastructure, ranging from (0) BSC Maturity Model by van Grembergen
developed IT performance measurement non-existent to (5) optimized (see, also for and Saull (2001) also had a major influ-
solutions, a refinement of the gradation the following, Fig. 2). The evolution path ence on the principal characterization of
was also deemed necessary. Consequently, outlined here starts from a complete lack the individual stages.
the architecture of the model was adjusted of IT performance transparency and ends
accordingly in the third iteration and – fol- at its highest level in a networked IT per- 5.5 Conception of transfer and evaluation
lowing the maturity models in COBIT (IT formance measurement process that is
Governance Institute 2007) – six degrees supported by companywide integrated BI Besides academic publications, the cur-
of maturity, ranging from non-existent (0) tools. To facilitate a differentiated analy- rent conception of transfer and evaluation
to optimized (5), were included. In order to sis of the IT performance measurement, includes the development of a web page
diminish the technical orientation, a pro- the maturity model provides three dimen- enabling companies to calculate their
cess-oriented approach based on Grothe sions that are characterized by five criteria. degree of maturity. This is meant to expand
and Gentsch (2000) was adopted. The dimension contents measures the rel- the empirical basis, so far consisting of
In the fourth iteration, the modified evance in substance of the applied IT per- data from the expert interviews, for the
architecture led to a second and completely formance measurement solution for the IT evaluation of the model. When calculating
revised version of the maturity model. In a management. The dimension organization their level of maturity, companies specify
group discussion with IT consultants the looks at both the integration of the solu- their values of the maturity model criteria.
model received a largely positive feedback. tion into the organization and operations These specifications of the BI support
Greater precision with regard to the con- of the IT department and at its integration for their IT performance measurement
tents of the dimension contents was how- into companywide concepts. The dimen- should allow a statistical survey of the
ever desired. Further adjustment of the sion technology examines the components distribution of the degrees of maturity in

220 Business & Information Systems Engineering 3 | 2009

 BISE – RESEARCH PAPER

individual companies (see, e. g., the studies dards, developmental period, economic
in Chamoni and Gluchowski (2004), and efficiency, or utility of the product. This Abstract
Philippi et al. (2006)). Additionally, the opens up potentials for improvement to
web page should provide for a census of the procedure model as presented here, Jörg Becker, Ralf Knackstedt, Jens Pöppel-
the model acceptance, which may indicate e. g. in form of a detailed specification and buß
need for further development. operationalization of particular require-
ments and their further epistemological Developing Maturity Models for
substantiation, which could not have been IT Management – A Procedure
6 Outlook achieved within this article (for more detail Model and its Application
see Zelewski 2007).
Maturity models are of major importance Maturity models are valuable instruments
for IT management. With the great number for IT managers because they allow the
of maturity models that have been designed References assessment of the current situation of a
in the last few years, there is however a company as well as the identification of
Becker J, Knackstedt R, Pöppelbuß J (2009) Doku-
danger of increasing arbitrariness in the reasonable improvement measures. Over
mentationsqualität von Reifegradmodellent-
development of these models. The increase wicklungen. Arbeitsberichte des Instituts für
the last few years, more than a hundred
in deficient documentations may serve as Wirtschaftsinformatik, No. 123, WWU Münster maturity models have been developed
an indicator for this trend. By applying the Bendl H, Gleich R, Kraus P (2004) Wettbewerbs- to support IT management. They address
guidelines that Hevner et al. (2004) have vorteile durch strategieorientierte Steuerung a broad range of different application
proposed for design science to the design of der IT. HMD: Praxis der Wirtschaftsinformatik areas, comprising holistic assessments of
maturity models, eight requirements were 41(239):39–47 IT management as well as appraisals of
postulated for the design process and an Boehm BW (1981) Software engineering econo- specific subareas (e. g. Business Process
mics. Prentice-Hall
appropriate procedure model was devel- Management, Business Intelligence).
Chamoni P, Gluchowski P (2004) Integrations­
oped. This provides a sound framework trends bei Business-Intelligence-Systemen. The evergrowing number of maturity
for the methodologically well-founded WIRTSCHAFTSINFORMATIK 46(2):119–128 models indicates a certain degree of
development and evaluation of maturity CMMI Product Team (2006) CMMI for develop- arbitrariness concerning their develop-
models. Such a reasoned approach is ment. http://www.sei.cmu.edu/pub/docu- ment processes. Especially, this is high-
particularly necessary if maturity models ments/06.reports/pdf/06tr008.pdf. Accessed lighted by incomplete documentation
are not to be reduced to the status of a mere 2008-07-30 of methodologies applied for maturity
Cook CR, Visconti M (2000) Documentation pro-
marketing tool for business consultants. model development.
cess maturity. http://web.engr.oregonstate.
The main purpose of the procedure model edu/~cook/doc/documentation.htm. Accessed
In this paper, we will try to work against
proposed here is to raise awareness for a 2008-12-12 this trend by proposing requirements
methodologically well-founded maturity Covey RW, Hixon DJ (2005) The creation and use concerning the development of maturity
model design. of an Analysis Capability Maturity Model models. A selection of the few well-docu-
The procedure model can be used as (ACMM). http://stinet.dtic.mil/cgi-bin/GetTRDo mented maturity models is compared to
a guideline by projects designing matu- c?AD=ADA436426&Location=U2&doc=GetTR these requirements. The results lead us
rity models. An important task for future Doc.pdf. Accessed 2008-07-30 to a generic and consolidated procedure
de Bruin T, Rosemann M (2007) Using the Delphi
research would be to conduct empirical model for the design of maturity models.
technique to identify BPM capability areas. In:
studies on hypotheses about the effects 18th Australiasian conference on information It provides a manual for the theoretically
that this application may have. One of systems (ACIS). Toowoomba founded development and evaluation of
these hypotheses would be that using the de Bruin T, Rosemann M, Freeze R, Kulkarni U maturity models. Finally, we will apply
procedure model for the design of matu- (2005) Understanding the main phases of de- this procedure model to the develop-
rity models leads to improved documen- veloping a maturity assessment model. In: 16th ment of the IT Performance Measure-
tation and to more profitable results than Australasian conference on information sys- ment Maturity Model (ITPM3).
tems (ACIS). Sydney
an intuitive procedure without recourse to Keywords: Maturity model, IT manage-
DIN (1966) DIN 66001: Sinnbilder für Datenfluß-
a reference manual. und Programmablaufpläne. http://www.fh-je-
ment, IT performance measurement,
Our research focuses on the develop- na.de/~kleine/history/software/DIN66001– Design science, Epistemology
ment of the IT Performance Measurement 1966.pdf. Accessed 2008-12-12
Maturity Model (ITPM3). At the time of Eckerson W (2006) Performance dashboards:
writing this is in the conceptual phase of measuring, monitoring, and managing your
transfer and evaluation of maturity mod- business. Wiley, Hoboken
els. Eul M, Hanssen S, Herzwurm G (2006) Systemati-
sche Leistungsbestimmung in der IT. Control-
The procedure model was developed
ling (1):25–30
on the basis of a specific criteria catalog. Frank U (2004) E-MEMO: Referenzmodelle zur
It is of course possible that, by expanding ökonomischen Realisierung leistungsfähiger
the criteria catalog, or through the choice Infrastrukturen für Electronic Commerce. WIRT-
of a different argumentative premise, the SCHAFTSINFORMATIK 46(5):373–381
procedure model may be improved with Freeze R, Kulkarni U (2005) Knowledge manage-
regard to certain success criteria, such as ment capability assessment: validating a
knowledge assets measurement instrument.
documentation quality, scientific stan-

Business & Information Systems Engineering 3 | 2009 221

BISE – RESEARCH PAPER

In: 38th Hawaii international conference on sys- conference on system sciences (HICSS). Hawaii
tem sciences. Wettstein T, Küng P (2002) A maturity model for
Grothe M, Gentsch P (2000) Business Intelligence: performance measurement systems. Depart-
aus Informationen Wettbewerbsvorteile ge- ment of Informatics, Fribourg University,
winnen. Addison-Wesley, München Schweiz
Henderson JC, Venkatraman N (1993) Strategic Zelewski S (2007) Kann Wissenschaftstheorie be-
alignment: leveraging information technology hilflich für die Publikationspraxis sein? In: Leh-
for transforming organizations. IBM Systems ner F, Zelewski S (eds) Wissenschaftstheore-
Journal 38(2&3):472–484 tische Fundierung und wissenschaftliche Ori-
Hevner AR, March ST, Park J, Ram S (2004) Design entierung der Wirtschaftsinformatik. Berlin,
science in information systems research. MIS 71–120
Quarterly 28(1):75–105
IT Governance Institute (2007) COBIT 4.1. The IT
Governance Institute
Kulkarni U, Freeze R (2004) Development and va-
lidation of a knowledge management capabili-
ty assessment model. In: 25th international
conference on information systems (ICIS)
Lee J, Lee D, Sungwon K (2007) An overview of
the Business Process Maturity Model (BPMM).
In: International workshop on process aware in-
formation systems (PAIS 2007). Huang Shan
(Yellow Mountain), China, 384–395
March ST, Smith G (1995) Design and natural sci-
ence research on information technology. Deci-
sion Support Systems 15(4):251–266
Marshall S (2007) E-Learning maturity model.
http://www.utdc.vuw.ac.nz/research/emm/
index.shtml. Accessed 2008-07-22
McFarlan FW, Nolan RL (2003) Why IT does mat-
ter. http://hbswk.hbs.edu/item/3637.html. Ac-
cessed 2008-12-09
Müller A, von Thiemen L, Schröder H (2006) IT-
Controlling. So messen Sie den Beitrag der In-
formationstechnologie zum Unternehmenser-
folg. Der Controlling-Berater(1):99–122
Paulk M, Curtis B, Chrissis M, Weber C (1993) Ca-
pability maturity model for software, Version
1.1. http://www.sei.cmu.edu/pub/docu-
ments/93.reports/pdf/tR24.93.pdf. Accessed
2008-07-23
Peffers K, Tuunanen T, Rothenburger MA, Chat-
terjee S (2007) A design science research
methodology for information systems
research. Journal of Management Information
Systems 24(3):45–77
Philippi J, Gronwald H, Schulze K-D, Dittmar C,
Müller T (2006) Business Intelligence-Studie
2006. Steria Mummert Consulting AG, Düssel-
dorf
Pößneck L (2007) IT does matter – sagen Analys-
ten. http://www.silicon.de/cio/strate-
gie/0,39038989,39183665,00/it+does+matter+
sagen+analysten.htm. Accessed 2008-12-09
Renken J (2004) Developing an IS/ICT manage-
ment capability maturity framework. In: Re-
search conference of the South African Insti-
tute for Computer Scientists and Information
Technologists (SAICSIT). Stellenbosch, 53–62
Rosemann M, de Bruin T, Power B (2006) A model
to measure business process management ma-
turity and improve performance. In: Jeston J,
Nelis J (eds) Business process management
van Grembergen W, Saull R (2001) Aligning busi-
ness and information technology through the
balanced scorecard at a major Canadian finan-
cial group. Its status measured with an IT BSC
maturity model. In: 34th Hawaii international

222 Business & Information Systems Engineering 3 | 2009

Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.