Sciencedirect: The Relationship Between Project Success and Project Efficiency
Sciencedirect: The Relationship Between Project Success and Project Efficiency
Sciencedirect: The Relationship Between Project Success and Project Efficiency
com
ScienceDirect
Procedia - Social and Behavioral Sciences 119 (2014) 75 – 84
Abstract
Many papers have rightly stated that meeting time, scope and budget goals, often called project efficiency, is not the ideal measure of project
success. Broader measures of success have often been recommended. However, no paper has described the empirical value of the relationship
between efficiency and overall success or shown whether efficiency is important at all to overall project success.
Our aim in this paper is to correct that omission. Through a survey of 1,386 projects we have shown that project efficiency correlates to only
0.6 of overall project success and that that overall success in impacted by efficiency with a R2 of .36.
©
© 2014
2014The
TheAuthors.
Authors.Published
Publishedbyby
Elsevier Ltd.
Elsevier Ltd.
Selection and peer-review under responsibility
Selection and peer-review under responsibility of the IPMA.
of the IPMA.
Keywords: Project; Success; Efficiency
1. Introduction
Project success has been measured in a variety of ways. While the measurement of project success has focused on tangibles,
current thinking is that ultimately, project success is best judged by the stakeholders, especially the primary sponsor, (Turner and
Zolin, 2012). As Shenhar, Levy, & Dvir (1997) note, assessing success is time-dependent: “As time goes by, it matters less
whether the project has met its resource constraints; in most cases, after about one year it is completely irrelevant. In contrast,
after project completion the second dimension, impact on the customer and customer satisfaction, becomes more relevant.” (p12).
Building on that work, Shenhar and Dvir(2007) suggested a model of success based on five dimensions, judged over different
timescales, Table 1. Turner and Zolin (2012) suggest that at the end of the project you judge success by whether the scope is
completed within the constraints of time and cost, and the project’s output is delivered to specification, in the months following
the project success is judged by whether the output performs as required and gives the desired benefit; and in the years following
the project success is judged by whether the organization achieves higher order strategic objectives that improve organizational
performance.
Traditional measures of project success focused on the so called iron triangle; completing the defined scope of work to
specification, and meeting the time and budget goals, (Atkinson, 1999). Munns and Bjeirmi (1996) noted that much of the
project management literature considers “projects end when they are delivered to the customer,” (p83). They continued, “That is
the point at which project management ends. They do not consider the wider criteria which will affect the project once in use.”
(p83, our italics). This focus on the end date of the project is understandable from a project and project manager’s standpoint.
The definitions of a project imply an end date; at that time the project manager is likely to be released or move on to another
project. Also the reward structure in many organizations encourages the project manager to finish the project on cost and time
and nothing else, (Turner, 2009).
The literature has also examined the wider impact of projects on the business. Shenhar et al (1997) note of the three
traditional dimensions of project efficiency: time, budget and scope, scope has the largest role, as it also has an impact on the
customer and their satisfaction. They note, “Similarly, project managers must be mindful to the business aspects of their
company. They can no longer avoid looking at the big picture and just concentrate on getting the job done. They must
understand the business environment and view their project as part of the company's struggle for competitive advantage,
revenues, and profit.” (p10). This view was reiterated by Jugdev and Müller (2005), who reviewed the project success literature
over the past 40 years and found that a more holistic approach to measuring success was becoming more in evidence.
Researchers increasingly measure success by impact on the organization rather than success at only meeting the triple constraint.
Thomas et al (2008) state that measuring project success in not straightforward. “Examples abound where the original objectives
of the project are not met, but the client was highly satisfied. There are other examples where the initial project objectives were
met, but the client was quite unhappy with the results.” (p106). Collyer and Warren (2009) cite the movie Titanic, which was
touted as a late, over budget flop but went on to be the first film to generate more than $1 billion . Munns and Bjeirmi (1996)
also note that a project can be a success despite a poor project management performance.
Similarly, Cooke-Davies (2002) differentiates between project management success, where the project is well managed to
finish the desired scope within time and cost, and project success, where the project achieves its business objectives. The
importance of broader success measures for projects is now the norm. The most recent version of PMI’s ® PMBOK® as an
example, no longer mentions the triple constraint, (Project Management Institute, 2008). It now includes customer satisfaction in
addition to time, budget, and scope.
Table 1 raises the question as to what extent the different measures of success are correlated. Turner and Zolin, (2012)
suggest project efficiency is important to success because if the project is completed late and over budget it will be more difficult
for it to be a business success. One can argue time, budget and scope are an important part of project success; they are necessary
conditions, not sufficient conditions, (Turner and Zolin, 2012; Xue et al, 2013).
Table 1: The five dimensions of project success after Shenhar and Dvir (2007)
Success dimension Measures Time
Prabhakar (2008) notes “There is also a general agreement that although schedule and budget performance alone are
considered inadequate as measures of project success, they are still important components of the overall construct. Quality is
intertwined with issues of technical performance, specifications, and achievement of functional objectives and it is achievement
against these criteria that will be most subject to variation in perception by multiple project stakeholders.” (p7). Kloppenborg et
al (2009) stated that all measures of success contain the traditional factors of time, cost and performance. Dvir et al (2003) state
Pedro Serrador and J Rodney Turner / Procedia - Social and Behavioral Sciences 119 (2014) 75 – 84 77
that “there are many cases where projects are executed as planned, on time, on budget and achieve the planned performance
goals, but turn out to be complete failures because they failed to produce actual benefits to the customer or adequate revenue and
profit for the performing organization.” (p89). They also found, however, “all four success-measures (meeting planning goals;
end-user benefits; contractor benefits; and overall project success) are highly inter-correlated, implying that projects perceived to
be successful are successful for all their stakeholders.” (p94). Zwikael and Globerson (2006) using data collected from 280
project managers showed aspects of success show a similar frequency distribution. Figure 1 shows a highly similar distribution
between technical performance (project efficiency) and stakeholder satisfaction. Here technical performance was analogous to
meeting scope requirements. In addition, they reported a linear correlation between technical performance and customer
satisfaction with an R2of .37 (p < .001). This result showed a strong relationship between the two.
Figure 1: Frequency distribution of technical performance and customer satisfaction, after Zwikael and Globerson (2006)
To date there is no empirical work to investigate to what extent project efficiency contributes to overall project success. This
leads to our research question:
Cooke-Davies (2002) defined project success terminology, and differentiated between project success (measured against the
overall objectives) and project management success (measured against the widespread and traditional measures of performance
against cost, time and quality), (p.185). We adopt more current terminology, which uses project efficiency instead of project
management success (Shenhar et al, 1997; Shenhar and Dvir, 2007):
2. Research Methodology
We adopted a post-positivist approach. Post-positivism falls between positivism, where a completely objective solution can
be found, and phenomenology where all experience is subjective, (Trochim, 2006). Because perception and observation are
based on subjective opinion, our results cannot be fully objective. Some concepts such as project success are not fully
quantifiable and are impacted by subjective judgment. Post-positivism understands that though positivism cannot tell the whole
truth in business research, its insights are still useful.
2.1. Survey
To gather the data we conducted a survey. The questions are shown in Table 2. We asked the respondents to judge success in
three categories:
Project success rating: project team 5 point scale – Müller and Turner
assessment 1. failure (2007)
2. not fully successful
How do you rate the project team’s 3. mixed
satisfaction with the project? 4. successful
5. very successful
Project success rating: client assessment 5 point scale: Müller and Turner
1. failure (2007)
How do you rate the client’s 2. not fully successful
satisfaction with the project’s results? 3. mixed
4. successful
5. very successful
Overall project success rating: 5 point scale: Shenhar and Dvir (2007)
1. failure,
How do you rate the overall success of 2. not fully successful,
the project? 3. mixed,
4. successful,
5. very successful
Data collection ran for approximately 12 weeks from October 29, 2011 to January 31, 2012. Each participant was asked to
provide data on two projects, one more successful and another less successful. A total of 865 people started the survey with 859
completing at least the first portion of it which requested information on one more successful project. However, not all
participants entered data for two projects; therefore the total number of projects available for study was 1,539. After removal of
outliers and bad data, the usable total was 1,386 projects.
It is the case that with most studies of project success that use questionnaires or interviews, the results rely on participants
stating how successful a project was, which is subjective by nature. There may be ways to measure success in objective ways,
but this may only apply to project efficiency. Therefore this paper is mainly concerned with perceived project success as
reported by participants. To measure this factor, questions in the survey were based on a combination of the success dimensions
defined by Müller and Turner (2007) and Shenhar and Dvir (2007).
Table 3 gives a breakdown of respondents by country. Responses were received from 60 countries. The largest percentage of
respondents were from the USA (36%), followed by Canada and India. More than 10 responses were received from Australia,
Spain, Brazil, Singapore and Germany. 183 respondents chose not to answer the question.
3.1. Reliability
A Cronbach alpha analysis was performed on the survey questions. The Cronbach alpha coefficient is a number that ranges
from 0 to 1; a value of 1 indicates that the measure has perfect reliability, whereas a value of 0 indicates that the measure is not
reliable and variations are due to random error. Ideally the alpha value should approach 1. In general, an alpha value of 0.9 is
required for practical decision making situations while a value of 0.7 is considered to be sufficient for research purposes,
Nunnally (1978).
Three items measured project efficiency, and four overall project success. All measures showed a high Cronbach alpha score
which shows they are correlated, Table 4. This fits with results from our literature review that all success measures are to some
extent correlated (Dvir et al, 2003; Zwikael and Globerson, 2006; Prabhakar, 2008; Kloppenborg et al, 2009). Based on this
there is no strong reason to exclude any item. The average is .905 and the alpha will only improve marginally by deleting project
budget goals. The results of the Cronbach’s alpha analysis supported the initial assumptions that the elements identified for
measuring success (Dvir et al, 2003; Zwikael and Globerson, 2006; Müller and Turner, 2007) were valid measures for this survey
and accurately measured the judgments of respondents. Each variable achieved an alpha score greater than Į = .85. In practical
terms, this meant there was a high degree of confidence in the reliability of the data collected, and it is accurate and meaningful
for the purposes of this research. This indicates that all of the factors are interrelated to some extent. However we can note that
the only factors which are close to the threshold for removal are budget goals, scope and time goals. These are key components
of efficiency. Scope is the lowest of this category which is in keeping with Shenhar et al (1997) who stated that scope was the
most important of the triple constraint for overall success.
80 Pedro Serrador and J Rodney Turner / Procedia - Social and Behavioral Sciences 119 (2014) 75 – 84
3.2. Analysis
To facilitate further analysis, the success measures were grouped into three measures of success. These were the measures of
project success used throughout the analysis. They are:
Stakeholder satisfaction measure = mean of the following four responses as a summated scale:
1. Project success rating: sponsor assessment
2. Project success rating: project team assessment
3. Project success rating: client assessment
4. Project success rating: end user assessment
Overall success measure = mean of the following eight responses as a summated scale:
1. Project success: meeting timeline goals
2. Project success: meeting budget goals
3. Project success: meeting scope and requirements goals
4. Project success rating: sponsor assessment
5. Project success rating: project team assessment
6. Project success rating: client assessment
7. Project success rating: end user assessment
8. Overall project success rating?
The means of these factors were reviewed using subgroup analysis. First the measures were compared by industry, Table 5.
In addition, to the three calculated measures of success, the respondents’ response to the single question “Overall project success
rating” was examined. We can see, that construction has the highest project success factor. This is in agreement with the
literature in general that construction has better perceived rates of success than other industries (Zwikael and Globerson, 2006).
However, other trends are more difficult to see and ANOVA analysis does not indicate any of the factors are significantly related
to industry.
Pedro Serrador and J Rodney Turner / Procedia - Social and Behavioral Sciences 119 (2014) 75 – 84 81
Table 6 shows the results analyzed by region. Success ratings do appear to vary by region. The ANOVA results show a
significant p value for all of the factors. The Pacific reports the highest average efficiency followed by the North America and
then the Far East. Russia reports the lowest efficiency followed by Latin America and Africa. One can speculate that Africa is a
more challenging environment and therefore has less projects coming in on time and on budget. On the other hand, North
America reports the highest general success followed by Russia and the Indian subcontinent. The lowest general success was
reported by projects in the Far East, Latin America and Australasia. Arctic and Antarctica was not considered since the sample
was only one project. We do not see an apparent relationship between general success and efficiency at least in the rankings of
success by regions.
Table 7 shows the results analyzed by whether the project was local versus international. There does not appear to be strong
differences between the means for this factor though national projects appear to have the highest reported success rates. The
ANOVA results confirm this with no significant p values found.
Next, we examined the correlation between the respondents’ project success rating and the success factors, Table 8. The
analysis shows close to 90% correlation between this one question and the main success factors other than the efficiency factor.
This indicates a very close correlation between the manager’s overall rating of project success and factors combining the wider
success measures. However, the efficiency factor only shows a .58 correlation with the manager’s assessment.
Table 8: Correlation between project success rating and the success factors
Correlations
Marked correlations are significant at p < .050 N=1386
Efficiency Stakeholder Overall
measure satisfaction success factor
measure
Overall project success rating 0.58* 0.87* 0.88*
* - p<.05
Table 9 compares correlations among all the factors and the project success rating. The respondents’ own overall success
measure, project success rating, had a correlation of .87 with the success factor measuring overall success and the overall success
factor of .88. The success measure that had the lowest correlation with all the other success measures was the efficiency factor,
which had a correlation of .60 with the success factor and .58 with the respondents’ self-reported overall success rating. The
correlation with the overall success factor was higher because efficiency was also one of the components of this factor.
Table 10 shows the correlation of the individual measures of project efficiency, time, cost and scope with the measures of
project success. The correlation with the overall project success rating and the stakeholder satisfaction measure is between 0.4 to
0.6. The highest correlation is with meeting scope goals, as we would expect (Shenhar et al, 1997).
Pedro Serrador and J Rodney Turner / Procedia - Social and Behavioral Sciences 119 (2014) 75 – 84 83
Finally, we completed a regression analysis of the efficiency measure vs. the stakeholder satisfaction measure, Table 11. This
analysis shows a relationship between efficiency and stakeholder satisfaction. It indicates with a quite low p value that they are
related with an R2of .362. The coefficient of determination R2 provides a measure of how well future outcomes are likely to be
predicted by a model. This could indicate that meeting a project’s time, budget and scope goals is 36% responsible for
stakeholder satisfaction. This is concordance with and further generalizes the result of R2 =.37 reported by Zwikael and
Globerson (2006).
Table 11: Regression analysis for efficiency measure versus stakeholder satisfaction
Regression Summary for Dependent Variable: Stakeholder satisfaction
measure
4. Conclusions
As suggested by many authors, (Shenhar et al, 1997; Cooke-Davies, 2002; Jugdev and Müller, 2005; Thomas et al, 2008;
Collyer and Warren, 2009), overall project success is a much wider concept than the traditional so called iron triangle of project
efficiency, (Atkinson, 1999). In this paper we have investigated to what extent project efficiency is correlated with stakeholder
satisfaction and overall project success. Through a survey of 1,386 projects we found that project efficiency is 60% correlated
with stakeholder satisfaction and 56% with overall project success. This supports the assertion by Turner and Zolin (2012) that
project efficiency is an important contributor to stakeholder satisfaction and overall project success, but shows quite clearly that
other factors contribute significantly to both. We can postulate that these other factors might include:
• Performance of the project’s output post implementation, and achievement of the project’s output and impact (Turner et
al, 2010, Turner and Zolin, 2012, Xue et al, 2013)
• Whether the project’s output was what the stakeholders were actually expecting, or whether there was an omission in or
misinterpretation of the specification
• Risks that were not considered or changes to environment that were not anticipated (Munns and Bjeirmi, 1996, Thomas
et al, 2008, Collyer and Warren, 2009)
It has long been postulated that project success is more than the achievement of the project efficiency measures, but we believe
this is the first time the relationship has actually been measured
84 Pedro Serrador and J Rodney Turner / Procedia - Social and Behavioral Sciences 119 (2014) 75 – 84
Whether we will ever be able to wean project practitioners off their beloved iron triangle we cannot know. But this supports the
work of Turner and Zolin (2012) that project managers need project control parameters that look beyond completing the scope of
the project on time and within budget. Practitioners should be aware that when they plan and control the project that the broader
success measures need to be taken into account and made part of the planning and control process. This will improve project and
project manager perceived success especially over the long term. These results also demonstrate that practitioners cannot ignore
project efficiency goals if they want to maximize overall success.
References
Atkinson, R. (1999). Project management: cost, time and quality, two best guesses and a phenomenon, its time to accept other success criteria, International
Journal of Project Management, 17, 337--342.
Collyer, S. and Warren, C.M. (2009). Project management approaches for dynamic environments. International Journal of Project Management, 27, 355 -- 364.
Cooke-Davies, T.J. (2002). The real success factors in projects. International Journal of Project Management, 20, 185--190.
Dvir, D., Raz T. and Shenhar, A. (2003). An empirical analysis of the relationship between project planning and project success. International Journal of
Project Management, 21, 89--95.
Jugdev, K. and Müller, R. (2005). A retrospective look at our evolving understanding of project success. Project Management Journal, 36, 19-31.
Kloppenborg, T.J., Manolis, C. and Tesch, D. (2009). Successful project sponsor behaviors during project initiation: an empirical investigation. Journal of
Managerial Issues, 21, 140 - 159.
Müller, R. and Turner, J R. (2007). Matching the project manager's leadership style to project type. International Journal of Project Management, 25, 21-32.
Munns, A. and Bjeirmi, B. (1996). The role of project management in achieving project success. International Journal of Project Management, 14, 81--87.
Nunnally, J. (1978). Psychometric theory. New York: McGraw-Hill.
Prabhakar, G. (2008). What is project success: a literature review. International Journal of Business and Management, 26, 3-10.
Project Management Institute. (2008). A guide to the project management body of knowledge, (4th edition). Newtown Square, PA: Project Management
Institute.
Shenhar, A.J., Levy, O. and Dvir, D. (1997). Mapping the dimensions of project success. Project Management Journal, 28, 5-9.
Shenhar, A. and Dvir, D. (2007), Reinventing project management: the diamond approach to successful growth and innovation, Harvard Business Press.
Thomas, M., Jacques, P.H., Adams, J.R. and Kihneman-Woote, J. (2008). Developing an effective project: planning and team building combined. Project
Management Journal, 39, 105--113.
Turner, J.R. (2009). The handbook of project-based management, 3rd edition, New York: McGraw-Hill.
Turner, J.R., Huemann, M., Anbari, F.T. and Bredillet, C.N. (2010). Perspectives on projects. London and New York: Routledge.
Turner, R. and Zolin, R. (2012), Forecasting Success on Large Projects: Developing Reliable Scales to Predict Multiple Perspectives by Multiple Stakeholders
Over Multiple Time Frames, Project Management Journal, 43, 87--99.
Trochim, W. (2006). Positivism and post-positivism. Research methods knowledge base.
Xue, Y., Turner, J.R and Anbari, F.T., 2013, “Using results-based monitoring and evaluation to deliver results on key infrastructure projects in China”, Global
Business Perspectives, to appear.
Zwikael, O. and Globerson, S. (2006). Benchmarking of project planning and success in selected industries. Benchmarking: An International Journal, 13, 688--
700.