See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/24077601

Multi-criteria evaluation model for the selection of architectural consultants

Article in Construction Management and Economics · February 2002

DOI: 10.1080/01446190210159818 · Source: RePEc

CITATIONS READS

128 2,468

3 authors, including:

Franco K.T. Cheung Martin Skitmore

Birmingham City University Bond University
33 PUBLICATIONS 964 CITATIONS 606 PUBLICATIONS 23,223 CITATIONS

SEE PROFILE SEE PROFILE

All content following this page was uploaded by Franco K.T. Cheung on 16 October 2014.

The user has requested enhancement of the downloaded file.

 COVER SHEET

Cheung, Franco K.T. and Kuen, Judy L.F. and Skitmore, R.M. (2002) MULTI-
CRITERIA EVALUATION MODEL FOR SELECTION OF ARCHITECTURAL
CONSULTANTS. Construction Management and Economics 20(7):pp. 569-580.

Copyright 2002 Taylor & Francis

Accessed from:
https://eprints.qut.edu.au/secure/00004130/01/JUDY_CME7.doc
 MULTI-CRITERIA EVALUATION MODEL FOR SELECTION OF
ARCHITECTURAL CONSULTANTS

Franco K T Cheung1, Judy Leung Fung Kuen1 and Martin Skitmore2

1
Department of Building and Construction
City University of Hong Kong

2
School of Construction Management and Property
Queensland University of Technology

Corresponding Author:

Professor Martin Skitmore

School of Construction Management and Property
Queensland University of Technology
Gardens Point
Brisbane Q4001
Australia

23 APRIL 2002 (VERSION 3)

 1

WORD COUNT (EXCLUDING FIGURES AND TABLES)

Title, Abstract and Keywords 168

Main text 3336
References 434
Appendix 1257
------
Total 5195
====
 2

MULTI-CRITERIA EVALUATION MODEL FOR SELECTION OF

ARCHITECTURAL CONSULTANTS

ABSTRACT

The selection of a competent architect is vital to the success of a development. Like

many developed countries in the world, developers in Hong Kong select architect based

on a set of criteria. Although price is not the only consideration in the process,

decisions rely heavily on subjective judgement. By conducting a questionnaire survey,

this research identified the common criteria for selection and their relative importance

to an objective selection. This involved the use of an evaluation method called the

analytical hierarchy process (AHP). Survey data from projects with similar

characteristics was used to compute the criteria weights. Multi-criteria models for 7

out of 27 categories of project were built with reference to the computed weights

derived from survey. In addition, a computer programme named as “Architectural

Consultant Selection System” (ACSS) was designed to illustrate a logical approach for

the evaluation of prospective consultant architects based on the constructed models.

Key words: selection criteria, analytical hierarchy process, consultant architect

 3

INTRODUCTION

A good design provided by the architect not only leads to pleasing structures and

facilities; it determines the social, cultural and economic quality of the built

environment now and in the future. Furthermore, the secret to a successful project lies

in the professional, business, and personal relationships between the owner and the

architect. Therefore, selecting an architect is one of the most important decisions that

the client makes when undertaking a building project. The better job the client does in

the selection of an architect, the better the client will like the results of his/her

construction project.

Different architect selection processes have been developed for different kinds of clients.

For example, public sector clients typically require more competitive procedures than

the private sector (Molenaar et al, 1998). The Architectural Institute of British

Columbia (AIBC) (1998) divides these selection processes into two groups - direct

selection (where a single architect or architectural practice is considered and approached

on the basis of reputation, personal acquaintance or some personal recommendation) or

comparative selection (where several candidates are considered and ranked in some

way) – a favoured approach as it can generate a broad search for the best solution to a
 4

particular building opportunity (Haviland, 2000). One form of comparative selection

is the design competition, as it benefits from available state of the art knowledge as well

as generating prototypical ideas (AIBC, 1998). Most commonly, comparative

selection involves the consideration of the price to be charged for the service – an aspect

that has been considered in detail by the Construction Industry Council (CIC), who list

out five basic methods of selection in which price plays a part (CIC, 1998). In fact,

price is a central issue today, with selection procedures being seen as comprising a

continuum ranging between price and quality (Molenaar et al., 1998).

The advantage of price as a selection criterion is, of course, in its objectivity and

transparency, and architects are increasingly being selected for projects solely on basis

of their price to be charged for their services (Dwight, 2000) - in exactly the way the

majority of construction contractors are appointed. However, many commentators

have argued that this practice is not appropriate for the provision of services (Gronroos,

1984; Latham, 1994; AIBC, 1998) and that any form of price competition drives fee

levels down, thus reducing the quality of services involved (eg., Day, 1998; American

Council of Engineering Companies (ACEC), 2000). What is needed, it seems, is for

the selection to be based predominantly on qualifications and competence (CIC 1998).

The two-envelope system attempts to do this but is said to fail because it requires a fee
 5

bid to be prepared when the scope and nature of the services required are not well

defined (Peck, 1998). Perhaps the most promising method to date is Quality Based

Selection (QBS), which is a process that enables the project owner to obtain the services

of a highly qualified design professional at a fair and reasonable cost (Consulting

Engineers Council of Pennsylvania (CEC/PA), 2000). This is said to be the most

widely endorsed legal method for selecting a design professional by overseas public

owners (Consulting Engineers Council of Delaware (CEC/DE), 2000) and is

recommended by the Australian Council of Building Design, the Architects Council of

Europe (ACE), Association of Japanese Consulting Engineers and the American Public

Works Association and various other organizations around the world (CIC, 1998).

Although the recently developed selection processes have attempted to consider more

criteria, the basis for making decision remains judgemental.

The research described in this paper was aimed at (1) describing the nature of architect

selection systems in Hong Kong, (2) identifying the criteria needed for architect

selection in Hong Kong and (3) the development of a new selection system through the

application of a multi-criteria evaluation model based on the Analytic Hierarchy Process

(AHP) (Saaty, 1980). To help accomplishing these aims, it was necessary to carry out

a postal questionnaire survey from which it was shown that, in Hong Kong, public
 6

sector departments usually have their own selection systems but there are no

standardized procedures developed in private market. In both public and private

sectors, the systems used are rather subjective and lack a systematic way to evaluate the

architectural consultants. Thus, the proposed system seeks to improve the objectivity

in the selection process. The survey is used to show that the selection criteria in the

literature are commonly used in Hong Kong. Based on a selection of these under a

classification containing four critical factors, a theoretical model was developed. The

logic of the model was further used to produce a computer evaluation programme.

This programme is used in the paper to demonstrate how an objective decision can be

made.

MODELS DESIGN

Selection Criteria

Table 1 summarises the selection criteria used/proposed by ten different

organizations/parties and grouped under similar characteristics. Of these, twelve were

selected as being applicable in Hong Kong, i.e.,

 7

Firms’ background :-

• Reputation;

• Technical competence/qualification;

• Experience with similar project.

Past performance :-

• Cost control;

• Quality of work;

• Time control.

Capacity to accomplish the work :-

• Present workload;

• Availability of qualified personnel;

• Professional qualification/experience.

Project approach :-

• Approaches to time schedule;

• Approaches to quality;

• Design approach/methodology.

In addition, the consultant fee, being one of the factors thought to be most likely to be

considered by clients in Hong Kong, was added as a further criterion.

 8

Factors affecting criteria weights

Different clients have different objectives. For example, public organizations are likely

to concentrate more on serving the public interest than making profit when initiating a

development, while private developers are more likely to be concerned with the

financial benefits to be gained. Hence, the nature of the client is likely to affect the

weights of the evaluation criteria. For example, the public sector may place a greater

emphasis of the design approach, while the private developer may be more concerned

with the consultants’ fees. For similar reasons, the size of organization is also likely to

affect the criteria weights, as the organizations’ objectives are also affected by the size

of organizations. Moreover, the size of organization is often related to the size of

project.

In Hong Kong, the most common types of development comprise residential buildings,

commercial buildings, mixed residential and commercial buildings, and other public

facilities such as schools. The type of project is likely to be a critical factor affecting the

weights of evaluation criteria, as different project categories place different

requirements on the design team. For example, residential building designs are quite

similar in Hong Kong and most design firms are able to do this kind of work.
 9

However, some commercial developments, such as large-scale shopping malls or hotels,

require much more specialised treatment from designers.

The size of project is also likely to affect the weights of the evaluation criteria. Clients

in Hong Kong do not allow small size companies to undertake large-scale projects. In

such a situation, the current workload and availability of qualified staff are going to be

very important aspects affecting the selection process.

The Use of Analytical Hierarchy Process (AHP)

The selection of architectural consultants is a multi-criteria problem, i.e., a situation in

which one has a set of criteria to consider on a set of alternatives, in order to determine

the best alternative. There are many different types of techniques and theories for this,

including Decision Analysis models (based on multi-attribute utility theory) and

Multi-criteria Decision Analysis Techniques. However, most of the criteria here are

qualitative in nature and often can only be represented by subjective judgement in

linguistic format. Therefore, a multi-attribute evaluation model based on the concept

of the Analytic Hierarchy Process (AHP) (Saaty, 1980) was introduced. In the process,

perceptions, feelings, judgments and memories are organized in a hierarchy and are
 10

compared in pairwise to determine their relative magnitudes that influence decision

results. AHP arranges these dimensions in a network structure that provided a

framework for breaking down a problem (e.g. selection of architect suggested in this

paper) into its smaller constituent parts (Saaty, 2000). It is a “decision-aiding tool for

dealing with complex and multi-criteria decision” (Partovi, 1992) and has been used in

similar construction industry contexts: for the selection of projects (Korpela and

Tuominen, 1997), selection of project proposals (Mohanty, 1992; Mathur and

Deshpande, 1995), selection of procurement method (Cheung et al. 2001) selection of

contractors (Mustafa and Ryan, 1990; Assaf and Jannadi, 1994; Kong and Cheung, 1996;

Fong and Choi, 2000; Al-Harbi, 2001).

DEVELOPMENT OF MODELS

AHP goes through three stages, they are:

1. Constructing the hierarchy for criteria and sub-criteria

2. Comparing the weight of criteria and sub-criteria

3. Evaluating the alternatives by computation of the relative weight of criteria and

sub-criteria

A five level hierarchy was constructed in Figure 1, with the top level being the selection
 11

goal; the five main criteria of “firm’s background”, “past performance”, “capacity to

accomplish the work”, “project approach” and “consultant fee forming the second level;

with the third level comprising the sub-criteria expanding these five issues. After

Liberatore et al (1992), the fourth level assigns a rating to each sub-criterion for every

alternative, instead of the normal pair-wise comparisons, by means of a five-point rating

scale that codes as outstanding (O), good (G), average (A), fair (F) and poor (P), with

the priority weights of these five scales (i.e. O, G, A, F, P) being established using

pair-wise comparisons.

Since it was assumed that the relative importance of the evaluation criteria might be

affected by the “nature of client”, “size of organisation”, “type of project” and “sum of

project”, the structure of the theoretical evaluation model was divided into 27 categories

as shown in Figure 2. To test this assumption, the results were categorized accordingly.

In the figure, it shows 27 categories (from Group 1 to Group 27) classified under

different levels. In order to resolve the issue, the data were classified into groups with

similar characteristics. The first level is the “nature of client” which includes the

public, semi-public and private sectors. This is classified into three types of firm size -

“under 75”, ”75 - under 150” and “150 or above”. These are then subdivided into the

four types of project usually found in the Hong Kong construction market - residential,
 12

commercial, residential and commercial mix and other public facilities development.

Following this is the project size, categorized into “below HK$ 100million”, “between

HK$ 100million – HK$ 500million” and “above HK$ 500million”. Since public

facilities comprise only a small portion of all the developments, and in order to make

the structure more clear, the level under public facility is indicated by just a dotted line.

Evaluation of Criteria

Based on the developed hierarchy and the classification of project, a questionnaire was

set out to collect data for evaluating the weight of the identified criteria. A total of 259

questionnaires were mailed to a random sample of public, semi-public and private

clients obtained from the list of the real estate developers association of Hong Kong.

57 questionnaires were returned. Of these, 53 were fully completed, representing a

19.5% usable response rate. In determining the weights of the selection criteria, AHP

provides a consistency checking where a sample with consistency ratio less than 0.1

would be rejected. 36 sets sample data were found to be consistent. Among them, 35

sets came from the private sector and only 1 from the public sector, therefore only those

results from the private sector were categorized. Only seven groups of data were

available from the survey: Group 5, Group 11, Group 15, Group 21, Group 22, Group

23 and Group 27. Two-tailed t-distribution test is used to exclude samples that contain
 13

inconsistent weights. An appendix is attached in this paper to describe the

questionnaire survey and the allocation of weight in detail.

General Models

Group 5

Group 5 comprised 7 responses. One of these (Sample 3) was significantly different

from the sample mean so this was considered to be a special case that should be

excluded from the model. After its removal, a consistent weight was found for the 13

evaluation criteria within this category.

Group 21

This group also comprised 7 responses. Two of these (Sample 12 and Sample 13)

were significantly different from the sample mean, so these samples were removed.

Since Sample 12 had 7 criteria outside the confidence interval, it was removed first.

After excluding Sample 12, there were only two samples with 4 weights out of 78

outside the interval, and this was considered to be acceptable.

 14

Group 23

Group 23 comprised 8 responses. Sample 34 had the most (5) significantly different

criteria and was removed first. The next to be removed was Sample 31. After removing

these 2 samples, there was still one sample with one criterion outside the interval and

this was also excluded in this group.

Group 27

Group 27 comprised 9 sets of responses. All of these had at least one significantly

different weight. After removing Samples 18 and 19, only one sample had all criteria

within the interval, so removal of sample was still required. Samples 23, 24 and 26 were

also removed by trial and error.

Groups 11, 15 and 22

There were 2 responses in Group 11 and 1 in each of Group 15 and 22. Since the

sample size of Group 11, 15 and 22 were too small, the same method could not be used

to test whether the weights of the evaluation criteria with these groups are distributed

with the same mean.

 15

Remaining groups

Table 2 summarises the sample means of the 7 groups after this procedure. Due to

small sample size in Group 11, 15 and 22, the sample mean of these three groups is less

reliable. Therefore, these three groups were then excluded. Figure 3 summarises the

results for the remaining groups.

In Figure 3, among Groups 5, 21, 23 and 27, the criteria “professional

qualification/experience” had a relatively high weight. This may be because the

quality of the design is usually dependent on the ability of a single person in the

organisation, and therefore the qualification or experience of that person is a key issue.

Moreover, property development is a business investment, focusing on making profit

and reducing development costs is one way to increase project returns. Consequently

developers have a particular interest in the fee proposed by the consultants.

“Reputation”, on the other hand, had relatively low weights among the 13 evaluation

criteria, suggesting that potential design firms selected for the final decision have

already been screened by some pre-qualification process, making it of relatively low

importance at this stage.

It was noticed that Groups 21 and 27 were very similar and this was confirmed by a
 16

t-test. On the other hand, the weights of the 13 evaluation criteria for Groups 5 and 23

are quite similar. Both have the same project type and value classification but differ in

firm size (Group 5 is a small developing firm while Group 23 is a large firm).

However, the weights of the 13 evaluation criteria for these two groups are quite similar,

suggesting that the size of the developer has little influence on the criteria weights in

this case.

Application of the Multi-Criteria Evaluation Model

To show how multi-criteria evaluation could be applied in a simple and efficient way, op

a computer programme called “Architectural Consultant Selection System” (ACSS) was

developed based on the criteria weights of the seven groups. It was written by Delphi

4.0. ACSS aims at providing an efficient, consistent and objective method for

selection of architects. The evaluation process using ACSS contains four steps: (1)

Determination of project particulars; (2) Comparison of fee; (3) Assessment of

performance; and (4) Computation of score. An example of the use of ACSS is given

below.

Determination of project particulars

The first step a client has to consider in the selection of an architect for a particular
 17

project is the characteristic of the project. The programme allows clients to choose the

particulars of a project and their choice would determine which group the project

belongs to. Figure 4 shows an example of the particulars of a Group 11 project in

ACSS.

Comparison of consultant fees

Different from the other criteria identified in the next step, the consultant fee proposed

by an architect is a measurable criterion. In the programme, the rating for “consultant

fee” is measured by the ratio of the actual amount of the lowest fee to the fee proposed

by an architect under evaluation. Figure 5 shows the consultant fees of three architects

under evaluation in ACSS.

Assessment of Performance

The most important task in the selection process is to evaluate the performance of the

architects. Figure 6 shows the assessment of the three architects on their performance

in each sub-criterion. A five-point rating scale named as outstanding (O), good (G),

average (A), fair (F) and poor (P) with corresponding rating of 1.000, 0.500, 0.250,

0.125 and 0.063 respectively is adopted.

 18

Computation of score

After rating the performance, the programme would automatically calculate the score.

Figure 7 shows the overall weightings of the architects. They were computed by adding

the normalized weights for the selected ratings of each sub-criterion multiplied by the

global priority weight of the sub-criterion itself.

Under the new system of ACSS, the decision makers or the evaluators only need to refer

to the hierarchy in Figure 1 and assess the performance of architects according to the

steps suggested to make objective selection.

CONCLUSIONS

Traditionally, building clients appoint various design consultants such as architects,

structural engineers, building service engineers, etc. to provide professional services for

the management of the whole project. During the design and construction of the projects,

the architect plays a major role and his/her selection is therefore one of the most

important decisions the client makes. As the survey has shown, these evaluations are

currently conducted in a rather subjective manner and it is likely that the process will
 19

benefit from a more objective method of evaluation. Therefore, a multi-criteria

evaluation model is developed to tackle the task of interpreting subjective judgement in

a logical and systematic manner.

Based on the selected criteria and four critical factors, a theoretical multi-criteria model

based on the concept of Analytic Hierarchy Process (AHP) was developed. In order to

modify this theoretical model to a general model that can be applied to the actual

situation in Hong Kong, the weight of each criterion was collected by means of a

questionnaire survey. By applying the methodology of AHP to the collected

questionnaires, 36 sets with C.R. value less than 0.1 were used, with 35 of these coming

from the private sector. These were classified into 27 categories for the seven groups

of data.

With the exception of Groups 11, 15 and 22, which were too small, particular cases in

each group were excluded by using a two-tailed t-distribution test. The final criteria

weights for these seven groups are shown in Table 3. It was also shown that the

criteria set identified in the literature review are commonly used in the selection of

architectural consultants in Hong Kong, which provides some validation of the model.

Furthermore, the final criteria weight of the seven groups were used to construct a
 20

computer programme “ACSS”. The programme is sufficient to show the simplicity

and objectivity of the use of the multi-criteria approach to select architects.

The study also identified four factors that influence decision makers in assigning the

weights to the evaluation criteria. In the actual market situation, it is expected that the

developing organization would consider other aspects in addition to these four factors.

Hence, further study is necessary to derive a comprehensive set of influencing factors

for the selection of architectural consultants. In addition, the evaluation model was

developed for the private sector with only three of the seven groups being investigated,

suggesting ample scope for further investigation.

ACKNOWLEDGEMENTS

The guidance and valuable suggestions of Mr. Paul Ho of the Division of Building

Science and Technology, City University of Hong Kong is gratefully acknowledged.

 21

REFERENCES

ACEC (2000) Bidding is not the solution: case studies in bidding. American Consulting

Engineers Council, http://www.acec.org/programs/bidding.htm.

Al-Harbi, K. M. (2001) Application of the AHP in project management. International

Journal of Project Management 19(1) 19-27.

AIBC (1998) How to find, select and engage an architect. Architectural Institute of

British Columbia, http://www.aibc.bc.ca/public/seeking_an_arch/selecting.html.

Assaf, S. and Jannadi, O.M. (1994) A multi-criterion decision-making model for

contractor prequalification selection. Building Research and Information 22(6)332–5.

CEC/DE (2000) How to Select an Engineer. Consulting Engineers Council of Delaware,

http://www.cecde.org/qbs.html.

CEC/PA (2000) Selecting the right consulting engineer. Consulting Engineers Council

of Pennsylvania, http://www.cecpa.org/selecting_a_ce.htm.
 22

Cheung, S.O., Lam, T.I., Leung, M.Y. and Wan Y.W. (2001) An analytical hierarchy

process based procurement selection method. Construction Management and

Economics 19(1) 427-437.

CIC (1998) A guide to quality based selection of consultants (a key to design quality.

London: Construction Industry Council.

Day, W. (1998) Performance over price. American School & University,

http://www.asumag.com/magazine/Archives/0898profsvcs.html.

Dwight, B. (2000) The low bid heart surgeon. Consulting Engineers Council of

Minnesota, http://www.cecm.org/low_bid_heart_surgeon.htm#T.

Gronroos, C. (1984) Strategic management and marketing in the service sector. Helsinki:

Swedish School of Economics and Business Administration.

Haviland, D. (2000) You and your architect. American Institute of Architects,

http://www.aiapvc.org/yourarch.htm.
 23

Kong, W.K. and Cheung, S.M. (1996) A multi-attribute tender evaluation model.

Proceedings CIB W89 Beijing International Conference, 21-4 Oct.

Korpela, J. and Tuominen, M. (1997) Group decision support for analysing logistics

development projects. Proceedings of the Hawaii International Conference on System

Sciences 30(2) 493-504

Latham, M. (1994) Constructing the Team, HMSO, London

Liberatore, M.J., Nydick, R.L. and Sanchez, P.M. (1992) The evaluation of research

papers. Interfaces, 22(2) 92–100.

Mathur, U. P. and Deshpande, V. B. (1995) Decision making in value engineering using

AHP. Proceedings, Value engineering in project management, Krishnan, P. Saxena, K.

R. (eds), Association of Consulting Civil Engineers, New Delhi, Dec, ACCE; Oxford

and IBH Publishing Co, 125-36.

Mohanty, R.P. (1992) Project selection by a multiple-criteria decision-making method:

 24

an example from a developing country. Project Management, 10(1)31–8.

Molenaar, K., Zimring, C. and Augenbroe, G. (1998) A guide to project delivery for

federal buildings. Georgia Institute of Technology, http://cem.ce.gatech.edu/GSA/

Mustafa, M.A and Ryan, T.C. (1990) Decision support for bid evaluation. Project

Management, 8(4)230–35.

Partovi, F.Y. (1992) Determining what to benchmark: an Analytic Hierarchy Process

approach. International Journal of Operations Production, 14(6) 25–39.

Peck, M. (1998) NSW Two Envelope System not the solution. Royal Australian Institute

of Architects, http://www.raia.com.au/media/html/pr23jan98.htm.

Saaty, T.L., 1980, The analytic process : planning, priority setting, resources allocation,

London : McGraw-Hill.

Saaty, T.L. (2000) Fundamentals of Decision Making and Priority Theory with the

Analytic Hierarchy Process – Vol. VI, University of Pittsburgh.

 25

APPENDIX: QUESTIONNAIRE SURVEY AND ALLOCATION OF WEIGHTS

Questionnaire Survey

The questionnaire was structured into two parts to obtain (I) background information of

the respondents, and (II) weights of the evaluation criteria. Part I comprised five

questions. The first two questions were designed to obtain general information about the

nature and size of the respondent firms. Question (3) was designed to identify the

selection criteria considered by the respondents and Questions (4) and (5) were used to

collect comments on the selection systems used by the respondents. Part II comprised

eight questions, relating to the respondents’ real, past or present, projects. Questions

(1) and (2) in Part II sought brief information on the selected projects, in terms of type

and value. Questions (3) to (7) concerns the relative importance of five criteria - “firm’s

background”, “past performance”, “capacity to accomplish the work”, “project

approach” and “consultant fee” – and their sub-criteria. Finally, the last question,

Question (8), was used to seek opportunities for further data collection.

Background information concerning the respondents comprised five areas - the nature

and size of their organizations, the criteria used for the selection of architectural

consultants, the degree of satisfaction with the selected architectural firms and opinions
 26

on the selection systems used.

94% of the respondents were from the private sector, with only 4% from the public

sector and 2% from the semi-public sector. The reason for the low response rate from

these two latter sectors is that most of the development carried out by the public sector

is handled by just two departments - the Hong Kong Housing Department and

Architectural Services Department (ASD).

Most (71.7%) of the respondents were from large firms with total number of employees

of 150 or above. Of these, 35 were from the private sector, 2 were from the public sector

and 1 from the semi-public sector. Only 10 responses (18.9%) came from small firms,

and approximately one-tenth of the responses were received from medium firms. This

is to be expected, due to the Hong Kong property market being in recession since the

end of 1997, resulting in only the larger developers being able to maintain their numbers

of projects.

Of the five criteria, the criterion “firm’s background” was used by all respondents.

“consultant fee” was the least important criterion, being used by only 79.2% of

respondents. Four additional criteria were suggested by the respondents. These are
 27

“previous working relationship with the client”, “innovative design”, “take ownership

of the project” and “which key staff to be assigned”.

Only two thirds of the respondents were satisfied with the performance of selected

architects, with 15.1% sometimes satisfied and 13.2% never satisfied. 5.7% respondents

made no comment.

Most (83%) respondents thought their own selection systems to be systematic.

Among the 53 sets of data, there were 22 sets (41.5%) relating to commercial buildings,

with 15 (28.3%) and 14 (26.4%) sets related to residential buildings, and mixed

residential and commercial buildings respectively and only 2 sets (3.8%) related to

public facilities. Nearly half of the projects had a contract sums between HK$100

million to HK$500 million, which is quite large for Hong Kong. Two projects were

below HK$100 million, with the remaining projects above HK$500 million. As

mentioned above, most of the respondents were from large firms, which explains the

preponderance of larger projects.

 28

Allocation of weights

The questionnaire survey responses were used to generate priority weights for the

criteria and sub-criteria. However, the priority weights of the rating scales in level 4

could not covered by the questionnaire as the priority weights for the rating scale of

each criterion can be different. For example, the relative value of an “outstanding”

versus a “good” rating can be different for different criteria. Hence, a potential

complication arises if the respondents are to be asked to compare the rating scales for all

of the 12 sub-criteria as this is very tedious and time consuming. Since making such

fine judgements would be very difficult and impractical, one set of “local” weights for

the five-point rating scale was used. The priority weights of the rating scale were

established by assuming the difference in relative importance between two adjacent

scales with respect to a particular scale is constant at 2 times.

The pair-wise comparison judgements were made with respect to attributes of one level

of hierarchy given the attribute of the next higher level of hierarchy, from the main

criteria to the sub-criteria. The relative importance of criteria was determined by

comparing it over the others with a nine-points scale as shown in Table 3, which

indicate the level of relative importance form equal, moderate, strong, very strong to

extreme by 1, 3, 5, 7 and 9, respectively. The intermediate values between two adjacent

 29

arguments are represented by 2, 4, 6 and 8.

The pair-wise comparison judgement matrix (PCJM) for the rating scales obtained this

way is shown in Table 4. This matrix was translated into the largest eigenvalue problem

and solved to obtain the normalized priority weights1.

The normalized priority weights (local priority) of the elements in the matrix were then

obtained by solving the pair-wise comparison judgement matrices (PCJM) obtained

from the questionnaire responses – producing a total of 5 sets of PCJMs. This

involved three steps :

• Adding the values in each column;

• Dividing each value by its column sum to obtain the normalized matrix;

• Averaging over the rows by adding the values in each row of the normalized matrix

and dividing the rows by the number of entries to obtain the normalized priority

weight.

The normalized priority weights (local priority weight, LPW) of all the main criteria

and sub-criteria were next combined together with all successive hierarchical levels to

1
using EXPERT CHOICETM
 30

obtain a global composite priority vector. The evaluation model was then built up with

each of the critical selection criteria assigned with a global priority weight (GPW)

obtained by multiplying the LPW of a particular sub-criterion with LPW of its parent

criteria. For example, the LPW of the sub-criterion “reputation” is 0.093, the GPW of its

main criterion “firm’s background” is 0.054, so the GPW of “reputation” therefore is

0.054 x 0.093=0.005.

To eliminate the possible inconsistency revealed in the criteria weights, the values of

consistency ratio are determined to justify the judgement made by the respondents.

Following Saaty (1980), those with a C.R. value greater than 0.1 were rejected from the

analysis. 36 sets questionnaires had a C.R. value of all PCJM below 0.1, and 17 sets

had more than one C.R. value of the PCJM greater than 0.1. Following Saaty (1980),

those with a CR value greater than 0.1 were rejected from the analysis. T-tests were

used to test whether the weights of the evaluation criteria among the remaining 36 set

data are distributed with the same mean and it was found that most of the sample

weights for these 13 evaluation criteria in the 36 set data were not distributed with the

same mean. There are two possible explanations for this result. Firstly, each set of

data may have its own characteristics and be independent of others so it cannot be

viewed as a whole. The second possibility is that the data which have similar
 31

weightings may analyse with other groups of data that have large differences between

them. However, the first explanation is less likely because some of the data, such as

project type and contract sum, have the same characteristics as others so there should

exist some common pattern for assigning the priority weights.

The results of the survey are described in the GENERAL MODELS section.
 32

CAPTIONS

Figures

1 The proposed classification of projects

2 The hierarchy for selection of architects
3 Summary of the sample mean for Group 5, 21, 23 and 27
4 Screen for determination of project particulars in ACSS
5 Screen for comparison of fee in ACSS
6 Screen for assessment of performance in ACSS
7 Screen for computation of scores in ACSS

Tables

1 Summary of evaluation criteria suggested by professional organisations

2 Summary of the sample mean of global priority weight of the seven groups
3 The fundamental scale as used by Satty (2000)
4 Pair-wise comparison judgement matrix (PCJM) for five-point rating scale
 Level 1 Level 2 Level 3 Level 4
Nature of Client Firm Size Project Type Contract Sum

< HK$100 Million Group 1

Residential (R) HK$ 100 – 500 Million Group 2
> HK$ 500 Million Group 3
< HK$100 Million Group 4
< 75 No. of
Commercial (C) HK$ 100 – 500 Million Group 5
Employees
> HK$ 500 Million Group 6
< HK$100 Million Group 7
Mix (M) HK$ 100 – 500 Million Group 8
> HK$ 500 Million Group 9
Public Facilities (P.F.) …
< HK$100 Million Group 10
Residential (R) HK$ 100 – 500 Million Group 11
> HK$ 500 Million Group 12
< HK$100 Million Group 13
Public / Semi- 75 – 150 No. of
Commercial (C) HK$ 100 – 500 Million Group 14
Public / Private Employees
> HK$ 500 Million Group 15
< HK$100 Million Group 16
Mix (M) HK$ 100 – 500 Million Group 17
> HK$ 500 Million Group 18
Public Facilities (P.F.) …
< HK$100 Million Group 19
Residential (R) HK$ 100 – 500 Million Group 20
> HK$ 500 Million Group 21
< HK$100 Million Group 22
> 150 No. of
Commercial (C) HK$ 100 – 500 Million Group 23
Employees
> HK$ 500 Million Group 24
< HK$100 Million Group 25
Mix (M) HK$ 100 – 500 Million Group 26
> HK$ 500 Million Group 27
Public Facilities (P.F.) …

Fig. 1 : The proposed classification of projects

 vii

Selection of
Level 1: Goal Architectural
Consultants

Capacity to
Level 2: Main Firm's Past Project Consultancy
Accomplish
Criteria Background Performance Approach Fee
the Work

Technical Experience Availability to Professional Design

Level 3: Sub- Cost Quality of Present Approaches to Approaches
Reputation Competence / with Similar Time Control Qualified Qualification / Approach /
Criteria Control Work Workload Time Schedule to Quality
Qualification Project Personnel Experience Methodology

Level 4: Rating Outstanding

Good (G) Average (A) Fair (F) Poor (P)
Scale (O)

Level 5: Architectural Architectural Architectural

Alternatives Firm A Firm B Firm C

Fig. 2: The hierarchy for selection of architects

 vii

0.45
0.40
0.35
0.30 5
Weight

0.25 21
0.20 23
0.15 27
0.10
0.05
0.00
1 2 3 4 5 6 7 8 9 10 11 12 13
Criteria

Fig. 3: Summary of the sample mean for Group 5, 21, 23 and 27

User’s Firm Nature

User’s Firm Size

Project Type

Approximate Contract Sum

Fig. 4: Screen for determination of project particulars in ACSS

 viii

Proposed Consultant Fee

Name of Architectural Consultants

Reference Code

Fig. 5: Screen for comparison of fee in ACSS

A Five-point Rating

The Button “Calculate”

Fig. 6: Screen for assessment of performance in ACSS

 ix

Detailed Breakdown of the Weighted Score

Graphical Presentation

Total Weighted Score

Fig. 7: Screen for computation of scores in ACSS

 x

Table 1: Summary of evaluation criteria suggested by professional organisations

New Hampshire Qualifications Based Selection Coalition

Facilities Management Polices & Procedures Manual

Consulting Engineers Council of Pennsylvania

Architect Engineer Contract Coordination

Consulting Engineers Council of Illinois

American Institute of Architects (AIA)
Professional organizations
Federal Acquisition Regulation
(1997)

(1997)

(1997)

(2000)

(2000)
(1998)

(1998)

(1999)

(1999)

(2000)
Zorn
Day

CIC
Selection Criteria
Firms' background
Firms' qualifications ●
Reputation ● ● ●
Financial stability ● ●
Technical competence / qualification ● ● ● ● ●
Award ●
Experience with similar project ● ● ● ● ● ● ● ● ●
No. of similar projects completed ●
Past performance
Cost control ● ● ● ● ●
Quality of work ● ● ● ●
Time control ● ● ● ●
Capacity to accomplish the work
Phyical resources ● ●
Current no. of contracts ●
Present workload ● ● ● ● ●
Availability of qualified personnel ● ● ● ●
Firms' size ● ●
Professional qualifications/experience ● ● ● ● ● ●
Project approach
Approaches to time schedule ● ● ● ●
Cost control (on schedule) ●
Approaches to quality ● ● ●
Grasp of project requirement ● ●
 xi

Table 2: Summary of the sample mean of global priority weight of the seven groups

Group
5 21 23 27 11 15 22
Criteria
1. Reputation 0.005 0.005 0.022 0.005 0.073 0.028 0.024
2. Technical competence/qualification 0.032 0.026 0.012 0.027 0.072 0.028 0.194
3. Experience with similar project 0.013 0.023 0.009 0.025 0.016 0.083 0.088
4. Cost control 0.030 0.025 0.021 0.023 0.016 0.075 0.083
5. Quality of work 0.035 0.119 0.068 0.113 0.057 0.075 0.083
6. Time control 0.046 0.062 0.019 0.055 0.056 0.075 0.083
7. Present workload 0.038 0.133 0.034 0.123 0.105 0.096 0.037
8. Availability of qualified personnel 0.121 0.060 0.121 0.062 0.057 0.096 0.037
9. Professional qualification/experience 0.400 0.174 0.299 0.166 0.180 0.096 0.037
10. Approaches to time schedule 0.067 0.012 0.023 0.015 0.071 0.060 0.050
11. Approaches to quality 0.045 0.037 0.184 0.042 0.038 0.060 0.050
12. Design approach/methodology 0.016 0.074 0.073 0.086 0.191 0.179 0.050
13. Consultant fee 0.152 0.250 0.114 0.259 0.069 0.051 0.187
 xii

Table 3: The fundamental scale as used by Satty (2000)

Intensity of Definition Explanation

importance

1 Equal importance Two activities contribute equally to the

objective

3 Moderate importance Experience and judgement slightly

favour one activity over another

5 Strong importance Experience and judgement strongly

favour one activity over another

7 Very strong or demonstrated importance An activity is favoured very strongly

over another; its dominance
demonstrated in practice

9 Extreme importance The evidence favouring one activity

over another is of the highest possible
order of affirmation

2, 4, 6, 8 Intermediate values When a compromise is needed

Reciprocals of If activity i has one of the above A reasonable assumption

above nonzero numbers assigned to it when
compared with activity j, then j has the
reciprocal value when compared with I

Rationals Rations arising from the scale If consistency were to be forced by

obtaining n numerical values to span the
matrix

Table 4: Pair-wise comparison judgement matrix (PCJM) for five-point rating scale

O G A F P
O 1 3 5 7 9
G 1/3 1 3 5 7
A 1/5 1/3 1 3 5
F 1/7 1/5 1/3 1 3

P 1/9 1/7 1/5 1/3 1

View publication stats