TM

MARKET SNAPSHOT :
Software Production
Management
By Theresa Lanowitz, Lisa Dronzek | November 19, 2007
 ~ vokeStream
TM
MARKET SNAPSHOT :
Software Production Management
By Theresa Lanowitz, Lisa Dronzek | November 19, 2007

~ SUMMARY ~ TABLE OF CONTENTS

• Executive Summary 2
Software Production Management is
a critical component in the creation • Market Trends 3
and delivery of quality software and is • The Application Lifecycle 2.0 3
a key ingredient in the ever evolving
application lifecycle. • Methodology and Objectives 5
• State Of The Market: Organizations 7
Highly optimized organizations are
experiencing tremendous return on • State Of The Market: Use Models 11
investment (ROI) by recognizing and • State Of The Market: Technology 13
treating Software Production
Management as a critical component • State Of The Market: ROI 14
of the application lifecycle. • Market Forces 16

In this Market Snapshot, we will • SWOTs 17

examine the organization and how it • Net/Net 18
benefits from Software Production
Management, use models and the
state of technology in the Software
Production Management market.

©2007 voke media, llc. All rights reserved. voke, vokeStream, and Market Snapshot are trademarks of
voke media, llc. All other trademarks are the property of their respective companies. Reproduction or
distribution of this document except as expressly provided in writing by voke is strictly prohibited.
Opinions reflect judgment at the time and are subject to change without notice. voke disclaims all
warranties as to the accuracy, completeness or adequacy of information and shall have no liability for
errors, omissions or inadequacies in the information contained or for interpretations thereof. Contact
www.vokeinc.com for additional information.
2 Market Snapshot: Software Production Management

EXECUTIVE SUMMARY
In August and September 2007, voke conducted an independent survey of
organizations regarding Software Production Management and its reach in the market.
The belief is that more mature organizations have better processes and therefore
understand the benefits of build and release management that Software Production
Management delivers.

By Software Production Management, we mean the automation, acceleration and

analysis of the software build, test, and deploy process. The survey encompasses this
emerging technology in the Application Lifecycle 2.0, its use, the state of the
technology, and how it benefits organizations.
TM
This Market Snapshot provides an in depth analysis and key findings of the Software
Production Management market with respect to:
• Organizations using the technology
• Continuous integration
• Application lifecycle development automation

Respondents reported return on investment (ROI) metrics in three categories. These

three justifiers can help others determine where they may be able to enhance
productivity, quality, and a faster time to market through Software Production
Management. The categories identified as ROI justifiers include:
• Monetary
• Headcount redirection to more strategic activities
• Productivity increases

Monetary savings of up to $1 to $2 million per year were identified by 40% of the

survey participants. Respondents identified monetary savings through decreased wait
time for builds, a faster time-to-market delivering competitive advantages and
eliminating the creation and maintenance of internally developed tools to deliver
Software Production Management solutions.

Headcount savings were identified as a major contributor to the purchase of

Software Production Management tools. Management redirected headcount from
tactical to strategic activities. All participants want to deliver more strategic value to
their customers. Incorporating Software Production Management solutions gave
organizations a way to move engineers from manual and error prone tasks to delivering
more valuable business insight and more productivity with the same or fewer resources.

Productivity increases of as much as 10 – 15X were identified by 33% of the

respondents. Increased demand is driven by increased complexity in the software
under development, faster time-to-market based on competition and the demand for
higher quality from customers. Organizations reported that staffing remained largely

November 19, 2007

©2007 voke media, llc. All rights reserved. Reproduction prohibited.

 Market Snapshot: Software Production Management 3

static, however, the demands and expectations of their workloads increased

significantly.

Managers, directors and C-level executives need to understand the crucial role
Software Production Management plays in the application lifecycle and embrace the
business value it delivers.

MARKET TRENDS
Over the past several years we have seen an increased emphasis on the importance of
enterprise IT organizations aligning with the line of business. We forecast that service
levels will continue to be challenged, and the trend dictates that the IT organization
must go beyond alignment to become a strategic partner with the line of business.
(See Future Watch: Enterprise IT Management and Organization October 23, 2006 at
http://www.vokestream.com). A dynamic and strategic IT organization will be required
to utilize application lifecycle tools and Software Production Management solutions to
handle service levels that are expanding to encompass global partners, customers,
vendors, and suppliers.

This required movement by IT organizations translates into modifying behavior to be

more in line with the way independent software vendors (ISVs) and embedded systems
software teams already function. ISVs and embedded systems groups focus on the
complete application lifecycle. They are highly optimized organizations who view
Software Production Management as an ongoing activity within the lifecycle. Most IT
organizations view Software Production Management as a discrete part of the lifecycle.
The goal for any organization creating and delivering software is to take a lifecycle
approach with Software Production Management woven into the many lifecycle phases.

The survey results presented here tell us that the application lifecycle is evolving from
where it was even a year ago. There is more emphasis being placed on customer
satisfaction, a greater need to deliver more value with less headcount, and increasing
demand on the commercial tools market from organizations that see diminished value
and greater cost in building their own. As this evolution continues, components that
drive efficiency and quality are becoming more crucial. Results show that within the
past year, organizations have broadened their use of commercial tools, and that these
tools, especially in the build and release process, gave organizations an advantage.

Software Production Management is a practice that must be embraced universally: in

the ISV community, the embedded market, and the enterprise IT organization.

November 19, 2007

©2007 voke media, llc. All rights reserved. Reproduction prohibited.

4 Market Snapshot: Software Production Management

THE APPLICATION LIFECYCLE 2.0

In the past several years the application lifecycle produced well-defined roles, increased
emphasis on process, consolidated with larger tools vendors delivering more integrated
solutions, and provided opportunities for service organizations to deliver best practices
and guidance for how to work in an application lifecycle environment. While increased
emphasis on process has been of benefit, the application lifecycle has also produced
linear organizations with silos of expertise and activity that lack the collaboration and
communication that a strategic organization requires in an increasingly complex and
global environment.

While the traditional application lifecycle was linear, The Application Lifecycle 2.0 meets
the demand for more strategic activities and more predictable results with a process
that is dynamic and fluid, and that focuses on:
• Communication, collaboration, and connectivity
• The concept of the extended organization – the internal organizations (line of
business, application development, quality assurance, operations), as well as
the external (customers, suppliers, service providers, partners)
• Context-sensitive information that can be disseminated to provide visibility
throughout the application lifecycle
• Best practices that can be refined, repeated and shared with other
organizations
• Bridging software systems, embedded systems, device software
• Bridging the traditional IT application with those embedded systems
• Emerging technologies such as Software Production Management and
virtualization

November 19, 2007

©2007 voke media, llc. All rights reserved. Reproduction prohibited.

 Market Snapshot: Software Production Management 5

METHODOLOGY AND OBJECTIVES

~ SURVEY METHODOLOGY
voke conducted an independent survey of the Software Production
Management market. The details of the survey include:
• Conducted from August 17 – September 20, 2007
• Surveyed a representative sample of unique individuals in the Software
Production Management market:
o Fortune 500 companies
o Enterprise IT organizations
o Independent Software Vendors (ISV’s)
o Embedded Systems companies
• Interviewed a variety of titles – all above practitioner level and responsible for
Software Production Management
• Asked a combination of multiple choice and open-ended queries
• All participants and employers are confidential

~ SURVEY OBJECTIVES
The primary objectives of the survey were:
• Identify the state of the Software Production Management market based on
three primary areas:
o Organization – the demographics: the roles, management of software
production, skill sets
o Use model – how the technology is being used, in what organizations the
use of Software Production Management predominates
o Technology – how mature is it, how ready is the market to accept the
technology
• Validate the market readiness of organizations to accept automated Software
Production Management tools
• Identify common themes and use case scenarios for Software Production
Management solutions
• Identify and quantify resource savings achieved through the use of Software
Production Management solutions

~ HOW TO USE THIS REPORT

This report is targeted to:
• Development and Quality Assurance (QA) managers and
directors investigating or evaluating Software Production Management tools
and automation. These development and QA managers and directors may be
part of:
o Enterprise IT
o Independent Software Vendors (ISV)

November 19, 2007

©2007 voke media, llc. All rights reserved. Reproduction prohibited.

6 Market Snapshot: Software Production Management

o Embedded systems software organizations

• Build managers as part of the development or QA organization focused on
increasing productivity and quality while reducing build time
• CIOs seeking an independent evaluation on the ROI of Software Production
Management technology
• Vendors in the application lifecycle market to understand the need for
emerging technologies to work with foundational application lifecycle tools and
services
• Financial analysts with a need to gain insight into the current state of the
market and how it will impact future opportunities
• Venture capitalists exploring market opportunities

In the emerging market of Software Production Management, it is imperative that

constituents communicate and collaborate to exceed customer expectations through
the software build, test, and deploy process.

November 19, 2007

©2007 voke media, llc. All rights reserved. Reproduction prohibited.

 Market Snapshot: Software Production Management 7

STATE OF THE MARKET: ORGANIZATIONS

We looked at the structure of organizations, their processes, motivations for
addressing Software Production Management, and challenges.

~ STRUCTURE
We asked organizations how many full-time equivalent employees (FTEs) they had
dedicated to the build and release process, whether the staff was in a single location or
distributed, and whether builds were managed onshore or offshore. The dedicated build
and release staffing numbers appear below.

Full time equivalent employees (FTEs)

dedicated to the build and release process
20 - 25 FTEs
16 - 20 FTEs 10%
10%

11 - 15 FTEs
1 - 5 FTEs
10%
60%

6 - 10 FTEs
10%

As for the distribution of the work, 55% reported that build and release personnel were
in a single location, and 45% said that staffing was distributed. 73% reported that
builds were managed onshore, and 27% reported offshore management of the process.

~ PROCESS
We asked organizations to rate the overall maturity of their build and release process,
and the majority reported that the process had structure with respect to planning, but
that their processes were not fully optimized.

November 19, 2007

©2007 voke media, llc. All rights reserved. Reproduction prohibited.

8 Market Snapshot: Software Production Management

Rating of the overall maturity of the

organization's build and release process
Six Sigma CMM Level 2
Optimized 7% 14%
7% Reactive
7%
Silo of Efficiency
14%

Planned
51%

In terms of the build process, results reflect today’s distributed teams with the build
process managed, in the majority of cases, by a central team supporting distributed
teams. Only 25% of builds are run locally.

Centralized versus decentralized builds

All local builds

25%

Central team
supports
Distributed
distributed teams
teams support
58%
themselves
17%

While the size of the source code base remains relatively large across all organizations
surveyed, the length of time of the average build has decreased significantly. Because
of Software Production Management capabilities, organizations are able to increase the
number of platforms to build for, increase their test matrices and systematically break
apart portions of an otherwise large code base to build what is specifically needed.
This ability to manipulate the code base to deliver more builds in less time and with
more purpose is a productivity booster for organizations.

November 19, 2007

©2007 voke media, llc. All rights reserved. Reproduction prohibited.

 Market Snapshot: Software Production Management 9

Average size of source code base

Less than 15,000

20 - 25 million lines of code
lines of code 22%
22%
1 million lines of
code
11%
4 - 5 million lines 3 million lines of
of code code
34% 11%

Survey respondents also cited a more efficient use of time as one of the by-products of
having faster builds. Engineers are able to have the build they need more quickly and
more frequently. This translates to the work being done in a more methodical fashion
rather than building and returning to work at a later time.

Average build time

10 - 20 hours
5 - 10 hours 8%
8% Less than 1 hour
38%

3 - 4 hours
23%

1 - 2 hours
23%

~ DRIVERS FOR IMPLEMENTING SOFTWARE

PRODUCTION MANAGEMENT
Respondents were asked to identify their primary motivations for addressing build
management. Most felt the need to manage releases efficiently. Also identified were
the need for higher quality and the need for more frequent releases. While 64% of
respondents reported that they are using a combination of in-house and
commercial solutions, 36% reported purchasing commercial solutions. Those
using commercial tools reported that internally developed tools were too costly to build

November 19, 2007

©2007 voke media, llc. All rights reserved. Reproduction prohibited.

10 Market Snapshot: Software Production Management

and maintain, and commercially available tools gave them the flexibility, higher quality,
and efficiency they needed in release management.

Primary motivation to address Software Production

Management (build and release)

Need for higher

Need to more
quality
efficiently
36%
manage the
release
46%

Need for more

frequent
releases
18%

The key motivators for purchasing all or part of a Software Production Management
solution were:

• The build and test cycle was too long (32%)

• The commercial solution was finally able to meet their needs (20%)
• Demands from the line of business (16%)
• The in-house tools were too expensive to build and maintain (12%)
• Difficulty supporting distributed teams (8%)
• Initiatives from management (8%)
• Adoption of more agile development techniques (4%)

~ BIGGEST ISSUES
We asked respondents: what were the biggest issues they faced going forward with
Software Production Management. Respondents report that Software Production
Management tools are making broken builds a thing of the past. Looking back to a year
to 18 months ago, 18% of respondents were dealing with the issue of broken builds.
Anecdotally, one respondent reported a decrease from 60% to 0% in broken builds.
Cumulatively, as well as individually, broken builds are no longer an issue among those
using tools from commercial providers. The demands for timely and high quality
releases, and the management of parallel releases are, along with supporting
distributed teams and integrating QA and build cycles emerging as new challenges.

November 19, 2007

©2007 voke media, llc. All rights reserved. Reproduction prohibited.

 Market Snapshot: Software Production Management 11

STATE OF THE MARKET: USE MODELS

How exactly are organizations managing their Software Production Management tools
resources? We found that for the most part resources were either dynamic and shared
on an as needed basis (38%), or in a central data center (39%), with 15% in isolated
groups or departmental clusters, and 8% isolated to a dedicated box in an office.

The build and release process was integrated into the other application lifecycle tools
used by the organizations reasonably well, with respondents reporting 73% integration,
versus 9% separation of tools, and 18% partially integrated.

We wanted to know which particular processes were integrated into other application
lifecycle tools, and the results indicate that source code management and defect
tracking, followed by testing were the processes most integrated with other application
lifecycle tools.

Processes integrated into application lifecycle tools

Internal tools
7%
Source code
management
40% Defect Tracking
40%

Testing
13%

November 19, 2007

©2007 voke media, llc. All rights reserved. Reproduction prohibited.

12 Market Snapshot: Software Production Management

We also asked respondents about changes in the levels of integration in the past year.
We found that there is an enormous increase in the amount of structure around the
build and release process (82%).

According to our respondents, the drivers of this change are:

• Greater and more frequent demands for builds and releases (33%)
• Process improvement: (33%)
• Time to market demands (17%)
• Continuous builds and integration (11%)
• Reduction in force (6%)

~ USE JUSTIFICATION
We asked how organizations were justifying the use of the automated software
production model. What were the benefits and what was the case made with
management for investing in this technology. We found that organizations really wanted
to direct their headcount to more strategic activities, and that emerged as the largest
factor in justifying the use of this technology. Reducing build time, improving time-to-
market, and increasing quality through reduction of errors were also key factors. Fewer
defects and more frequent builds translate to faster time-to-market with higher quality.

Internal justification for the use of an automated software

production management model
Ability to redirect Reduce cycle
or reduce time
dedicated 27%
headcount
30%
Time-to-market
Increased improvement
control and/or 17%
auditability Promise of fewer
13% errors
13%

Software Production Management delivers more flexibility in how and when builds
occur as well as delivering flexibility in managing large source code bases.

November 19, 2007

©2007 voke media, llc. All rights reserved. Reproduction prohibited.

 Market Snapshot: Software Production Management 13

STATE OF THE MARKET: TECHNOLOGY

Here we looked at whether organizations were using a suite of tools. We also queried
respondents about automated testing, and asked them what best practices were most
important to them.

~ APPLICATION DEVELOPMENT LIFECYCLE AUTOMATION

Fewer than half of respondents were using a suite of tools for their application
development lifecycle and of those that were, 36% were still using internally developed
tools, and the remaining 64% were using commercial tools.

We asked about the inclusion of automated testing in the build process and found that
64% of respondents do include automated tests in conjunction with their automated
builds.

Test automation run in conjunction with builds

Acceptance
System testing testing
6% 6% Build verification
22%

Regression
testing Unit testing
33% 33%

Highly optimized organizations automate both their build and test environments. This
high degree of automation is a best practice that enables organizations to deliver more
thorough testing in a variety of areas. Ultimately, automated builds and tests provide a
way for development and QA organizations to meet customer expectations and
demands on schedule, budget and quality. Communication and collaboration with
stakeholders is a primary benefit of complete automation.

~ BEST PRACTICES: CONTINUOUS INTEGRATION

Survey participants were asked how frequently they were running builds, as well as
whether their releases were continuously integrated. Highly optimized IT, ISV, and
embedded software organizations are investing in Software Production Management
solutions. They are working in an environment where frequent builds and continuous

November 19, 2007

©2007 voke media, llc. All rights reserved. Reproduction prohibited.

14 Market Snapshot: Software Production Management

integration are delivering benefits in defect identification and quality. 67% of the
respondents are continuously integrating releases.

Frequency of builds
Hourly
Upon request 6%
17%

Daily
Continuously 43%
28%

Weekly
6%

Software integration is a complex and crucial aspect of delivering complete solutions.

Continuous integration identifies design flaws and architectural and API defects as they
occur. Without continuous integration these often egregious defects are not
discovered or addressed until just prior to release. At that point, these serious defects
are often not addressable until the following release.

Survey respondents leveraging continuous integration as a best practice benefit from

an overall better understanding and more mature adoption of the application lifecycle.
Moving to a best practice of continuous integration delivers higher quality, more
reliability and a faster time to market.

STATE OF THE MARKET: ROI

Return on investment is a major factor in a purchasing decision for Software Production
Management tools. In this part of the Software Production Management survey, we
discussed and asked survey questions about the impact the use of tools was having on
the organization’s headcount and productivity, and how those impacts were translated
into monetary savings.

Respondents cited varied but impressive impacts from using Software Production
Management tools, with annual savings of from $100K to over $2M per year.

November 19, 2007

©2007 voke media, llc. All rights reserved. Reproduction prohibited.

 Market Snapshot: Software Production Management 15

Monetary savings by using software production 
management tools Saving $100,000 -
$150,000 per
Saving $1 - $2 year
million per year 30%
40%

Saving $300,000 -
$500,000 per
year
30%

In both monetary and headcount terms, the survey found that using Software
Production Management tools produced significant benefits. The survey asked about
the impact on headcount.

Organizations found that there was a significant positive impact on headcount by

reduction through natural attrition or by not having to request additional headcount to
deliver more build capacity,

Software Production Management Resource Savings

Saved > 20
headcounts Saved 1 - 5
10% headcounts
40%

Saved 11 - 20
headcounts
20%
Saved 6 - 10
headcounts
30%

Productivity increased significantly with increased build output by factors of two to

fifteen times, all the while increasing quality.

November 19, 2007

©2007 voke media, llc. All rights reserved. Reproduction prohibited.

16 Market Snapshot: Software Production Management

Build productivity increases identified by using software

production management tools
Increased build
output by 2x
22%

Increased build
output by 10x - 15x
33%

Increased build
output by 3x - 5x
45%

The ROI in Software Production Management proved to deliver substantial productivity

increases while providing significant monetary and headcount savings.

Software Production Management is often overlooked or is an area of minimal

investment. These survey findings show that Software Production Management
solutions deliver significant savings and impact to the overall project budget.

Ongoing market forces such as increasing application complexity, demand for faster
time-to-market, and increased quality expectations will drive lifecycle efficiencies to be
made. Software Production Management is a significant component of the evolving
application lifecycle 2.0 and will continue to deliver savings.

Implementing best practices around Software Production Management is an activity for

every organization to undertake.

MARKET FORCES
While Software Production Management is a necessary part of the overall application
lifecycle, its importance is often overlooked as well as misunderstood. Both vendors
and customers need to understand the strengths, weaknesses, opportunities and
threats (SWOT) in the marketplace. The following SWOT chart is representative of the
needs of the evolving and maturing market.

November 19, 2007

©2007 voke media, llc. All rights reserved. Reproduction prohibited.

 Market Snapshot: Software Production Management 17

~ SWOTs
Market SWOT
Strengths Weaknesses

ƒ Application lifecycle broadly accepted ƒ Benefits of Software Production

ƒ Emerging technology underscores the Management not fully articulated to senior
necessity to deliver more advanced solutions management
and embrace the application lifecycle 2.0 ƒ Internally developed tools
ƒ Strong and compelling ROI ƒ Productivity of distributed teams
ƒ Increased productivity ƒ Broken builds
ƒ Monetary savings
ƒ Resource reallocation to more strategic efforts

Opportunities Threats

ƒ Demonstrate cost savings of commercial tools ƒ Not a well-known phase of the application
vs. home-grown lifecycle, consequently little attention is given
ƒ Provide best practices around Software to it
Production Management ƒ False assumptions about Software
ƒ Partner with other emerging technology Production Management in the market
vendors or core application lifecycle providers o Home-grown tools will suffice
to deliver an integrated solution o Problem is already solved
ƒ Deliver open solutions to integrate with home- o Productivity is not impacted –
grown tools engineers can multi-task while
waiting for builds

For customers of Software Production Management solutions, the benefits are about
increasing productivity. The effective use of Software Production Management
solutions yields the following benefits:
• Enhanced productivity and efficiency of distributed teams
• Increased number of target platforms for builds
• More structure around build and release processes
• Reduced or static staff size while increasing output
• Repurpose saved resources to more strategic activities
• Faster time-to-market
• Increased size of the test matrix
• Increased frequency of builds
• Decreased duration of builds
• Effective management of parallel releases

Customers of Software Production Management solutions are still lacking industry-

wide best practices to make their processes more concise. There must also be a
concerted effort on the part of Software Production Management vendors to integrate
with other emerging technologies and core application lifecycle tools. A more

November 19, 2007

©2007 voke media, llc. All rights reserved. Reproduction prohibited.

18 Market Snapshot: Software Production Management

integrated suite of tools will demand more process and visibility around the entire
application lifecycle.

NET/NET
Every organization, regardless of size, can realize benefits from the use of Software
Production Management solutions. Effective build and release management has a
positive impact on quality and resource savings for enterprise IT organizations, ISVs,
and embedded software companies.

The Application Lifecycle 2.0 is evolving to include more sophisticated offerings and
looking at emerging technologies like Software Production Management to help
facilitate the communication, collaboration, and connectivity that organizations must
have with the line of business, their customers, their vendors, their suppliers, and all of
their partners.

~ RECOMMENDATIONS
Software production management is an often overlooked component to the application
lifecycle. When managed effectively, best practices in the software production
management phase of the application lifecycle can yield significant resource savings.

• To make your own organization more efficient, thoroughly understand how the
application lifecycle is being utilized. Once the maturity level of the organization
and its use of the application lifecycle have been determined, assess the roles,
responsibilities and process used in the application lifecycle.

• Determine how your organization is handling the build, test and deploy
functionality. Investigate tool usage and implement a commercial solution if one
is not in use. Commercial tools are evolving and are far more robust than an
internally created solution and will deliver more consistency and functionality.

• Ultimately, the goal for any organization is to deliver a highly satisfactory

product to their customer and respond more quickly in an increasingly complex
environment. To achieve these goals, organizations must be focused on the
strategic business outcomes not tactical execution. Use Software Production
Management solutions to deliver greater value to customers and enable your
team to be more strategic in their activities throughout the application lifecycle.

Adopt commercial solutions for Software Production Management. If you are not
looking at solutions from a commercial provider now, start your evaluation. The ROI
from using a commercial solution is significant in monetary savings, resources savings,
and increased productivity.

November 19, 2007

©2007 voke media, llc. All rights reserved. Reproduction prohibited.

Contact Information
Corporate Office
voke, inc.
2248 Meridian Boulevard
Suite H, Minden, NV 89423
USA

Asia Pacific Office

voke, inc.
75-1027 Henry Street
Suite 111A #424
Kailua-Kona, HI 96740-3154
USA

Phone: +1.808.885.5353
Web: www.vokeinc.com
Blog: www.voke.blogspot.com

~ ABOUT VOKESTREAM
vokeStream is the intellectual property that differentiates voke in the analyst
community. Through streaming insight, voke strives to provoke thought and encourage
discussion while delivering industry opinion in fluid, dynamic, and collaborative ways.
Because of voke’s dynamic model, the content delivered in the vokeStream is updated
on a regular and timely basis.

Subscriptions available at www.vokestream.com