BeyondTechnology CreatingBusinessValuewithDataMesh

Download as pdf or txt
Download as pdf or txt
You are on page 1of 11

W H I T E PA P E R

Beyond Technology:
Creating Business Value
with Data Mesh

Sponsored by
SPONSOR PERSPECTIVE

There has been tremendous advancement in the data and analytics


industry over the past two decades. Growth in available data, both in terms
of volume and in diversity of sources, has been exponential and those
businesses that are able to effectively utilize this data, using advanced
analytics and artificial intelligence (AI), have unlocked a significant
advantage over their competitors.

However, the organizations that actually experience these benefits remain


in the minority. For the rest, the promise of AI and data science remains
elusive, and even effective operational reporting can be a challenge. John Spens
Despite continued investments in data management initiatives and the Director, Data & AI
latest technologies, these organizations continue to struggle with the first Thoughtworks
step: enabling their employees to find and consume the data they need to
make better decisions and successfully deliver value.

This struggle with the first step leads us to an important question: Why
does providing secure and reliable access to high-quality data remain so
difficult?

The surprising answer is that, for the most part, technology is not the
core issue. Instead, most companies find that the operational and
organizational issues they face are their greatest impediments. Overcoming
these challenges requires the business to change how it perceives the
value of data, how business stakeholders view their responsibilities for
effective data stewardship, and how data management teams operate to
support their business.

Data mesh is a new approach to creating data solutions that seeks


to address these challenges by applying modern organizational and
engineering practices. By adopting the principles and practices of data
mesh, IT organizations are able to respond more quickly to the evolving
data needs of their organization and collaborate more effectively with
business partners to create insights.

We have sponsored this research by Harvard Business Review Analytic


Services to share the experiences of business and technology leaders as
they seek to deliver strategic insight to optimize their business and provide
better experiences for their customers. We explore the challenges they
face in creating value from their data and the impact of adopting the data
mesh paradigm to address these challenges.

In our digital economy, the ability to produce, share, and act on business
intelligence has never been more critical to an organization’s success.
Read on to discover the key strategies for winning with data mesh.
Beyond Technology:
Creating Business Value
with Data Mesh

Turning data into actionable insights for better business HIGHLIGHTS


outcomes has been the goal of organizations for well over
a decade now. The pandemic has further underscored
For an organization to become data-
the need for data-driven transformation, just as it has driven and empower business teams
revealed the importance of having visibility into supply with actionable insights, data needs
to become a product, essentially
chains. Companies can benefit exponentially from using a data set that is consumable by
data to more precisely understand their customers’ and anyone in the organization.
employees’ needs and expectations.
Businesses are making the requisite investments in data analytics technologies Data mesh is a federated approach
and skill sets to remain competitive and boost their growth. Fifty-five percent to data management, unlike more
of IT decision makers plan to increase investment in data-focused budgets traditional centralized approaches
where data sits in warehouses or
over the next 12 to 18 months, according to a 2022 study by Foundry.1 The same
lakes and is controlled by a central IT
study found that these companies will spend an average of $12.3 million in the
department.
coming year on data-driven initiatives, including software, services, training,
consulting, and other related costs.
Companies’ success with data analytics will depend on how they approach
The success of data mesh depends
their transformations. For many, IT is the warden of data, but that isn’t always
on the organization’s ability to engage
the most effective path to becoming a wholly data-driven enterprise. “Many technology and business executives
data analytics initiatives do not yield satisfactory business outcomes, because and change the organization’s social
with traditional data architecture, centralized IT teams act as gatekeepers of infrastructure.
data,” says Daniel J. Abadi, professor of computer science at the College Park,
Md.-based University of Maryland. “They are experts at data management
tools but not the data itself.” In this centralized data system, technology is
considered a goal and not an enabler to achieve business results—it focuses
on the output of data-driven initiatives and not their outcomes.
In recent years, organizations have been turning to a new approach to data
management called “data mesh.” Data mesh is a federated approach to data
management, unlike more traditional centralized approaches where data

White Paper | Beyond Technology: Creating Business Value with Data Mesh

1
Harvard Business Review Analytic Services

“Many data analytics initiatives do not yield satisfactory business


outcomes, because with traditional data architecture, centralized
IT teams act as gatekeepers of data,” says Daniel J. Abadi,
professor of computer science at the University of Maryland.

sits in warehouses or lakes and is controlled by a central IT can answer questions about the reliability of supply chains
department. With data mesh, the control and ownership and what demand will look like in the future.
of data are accessible to different domains throughout the Insights from data have the potential to boost companies’
organization, and these domains are responsible for handling competitive advantage and spur growth by improving internal
data and following organization-wide governance rules. business processes (50%), customer insights and engagement
With this approach, data is handled by teams in functions (46%), and customer service (43%), according to the Foundry
besides IT, closer to the business, so the data is better suited study. FIGURE 1
to business needs. But for many organizations, data is more an embarrassment
The essence of data mesh is that it puts the customer first of riches than a strategic asset because they are not able to
and focuses on data yielding outcomes instead of outputs. Just turn data into insights and derive optimal business value
like with any other product in the marketplace, the value of the from it. Data volume is increasing by 63% per month at
data product, or data set, depends on whether data consumers large organizations (those with at least 1,000 employees),
need and use it. This federated approach aims for different and the mean number of data sources per organization is
outcomes and follows various processes and workflows 400, with one-fifth of organizations drawing from 1,000 or
as opposed to traditional centralized approaches; thus, it more data sources.2 These data sources can be structured
requires the creation of new roles, teams, responsibilities, or unstructured; internal or external; raw or formatted; and
and accountability. The biggest impact of data mesh is on the
responsibility for data, which resides with business domains
rather than with IT.
FIGURE 1
The success of data mesh depends on the organization’s
ability to engage technology and business executives and
Looking for Business Insights
change its own social infrastructure. “My recommendation
Top business goals driving investments for
is to approach data mesh with the 70-20-10 mindset. Seventy
data-driven initiatives
percent of the focus should be on change management, 20%
on data and technology, and 10% on models and AI [artificial
50%
intelligence],” says Sanjeevan Bala, group chief data and AI
Improve/automate internal business processes
officer at ITV, a London-based free-to-air television network.
This report discusses the benefits of taking a decentralized
46
approach to data and the disadvantages of a centralized
Improve customer insights and engagement
data management system. It explores how companies are
implementing data mesh, what challenges they are facing, 43
and how they are overcoming them. The report also focuses Improve customer service/support
on how to form data mesh teams and how best to create and
track metrics. 43
Improve/automate IT operations

An Embarrassment of Riches 36
Data is undoubtedly a strategic asset. It can help answer critical Improve existing products
questions that decision makers have about how best to grow
36
their organizations, including which products customers are
Improve information security/cybersecurity
buying and how satisfied they are with those products, and
whether employees are productive and engaged. Data also Source: Foundry, 2022

White Paper | Beyond Technology: Creating Business Value with Data Mesh

2
Harvard Business Review Analytic Services

visual, transactional, or collected by sensors embedded in


machines, among other types.
The amount of data does not help if the data itself is not
trustworthy. “Without focus on data governance, there’s
an overall mistrust in data because even if it is of good
quality, it can be used in the wrong context and under wrong “Data doesn’t deliver value up to its full
assumptions,” says George Firican, director of data governance potential due to traditional practices
and business intelligence at the University of British Columbia of centralization of enterprise data in
in Vancouver. “A lot of organizations are aiming to become
data-driven and are investing more and more in data analytics,
warehouses or data lakes,” says Omar
but they often miss on investing in a strong foundation created Khawaja, global head of business
by data governance.” intelligence and analytics solution and
Even those organizations that focus on data governance—or
a set of internal data standards meant to ensure that data is
services teams for Roche Diagnostics.
consistent and trustworthy—still may not be able to use data
for business decision making due to traditional, centralized
approaches to data management. Typically, data is sitting
in warehouses (repositories for structured, filtered data teams tend to be more disengaged from the business side, and
that already has been processed for a specific purpose) or thus they have less understanding of what the data means.
data lakes (storage repositories that hold a vast amount of Once confronted with data that is untrustworthy, business
raw data in its native format until it is needed for analytics executives may find it hard to regain confidence in using
applications), both of which are typically controlled by IT. data. That point is true even if untrustworthy data appears
Potential consumers, be they business leaders or data analysts, in models or demonstrations during the testing stage.
may not be fully aware of what all the data is, how to access Functions across the business require access to self-service
and understand it, and how it can be useful for them to grow data tools. Bank traders, for instance, need them to generate
the business. useful data-driven insights in real time. Such insights can only
For bank traders, for example, going through the central be produced by people who understand the bankers’ trading
data group to get insights from price fluctuations defeats needs. Marketers, too, need self-service data tools to easily look
the purpose because it takes too long, says Sheetal Pratik, up insights from customer data to determine what advertising
director of engineering and data integration at Adidas and content to distribute. And planners in a manufacturing plant
formerly head of data integration at Saxo Bank, a Copenhagen, who decide on inventory levels or material purchases need
Denmark-based investment bank specializing in online trading such tools to access timely production and sales data.
and investment, where she rolled out a data governance self- It’s no wonder, then, that 83% of IT decision makers agree
service platform. that providing self-service tools to make data more accessible
In a centralized system, those in IT have the responsibility of to business is a top priority, according to the Foundry study.3
transforming data to produce relevant insights without being But that’s not how the generation of data-driven insights
domain experts. They need to rely on subject matter experts to works in most of today’s organizations. Nearly a third of
work together on the rules about how to transform and govern organizations say that data analytics solutions are available
the data. And things don’t always work the first time when for all or most users, but to access and use the data, they
processing and analyzing data or perfecting the algorithm; the need to have specialized data analytics or computer science
programmers need to go back to the subject matter experts to skills. Another 41% or so of organizations say that data and
make adjustments, says Pratik. This requirement creates a lot analytics solutions can only be used by a few skilled teams,
of back-and-forth among different groups working separately such as analytics teams and IT departments. Just one in five
because each group has different skill sets. Additional delays (21%) have data and analytics solutions that are available and
and bottlenecks occur because the central IT group is handling easy for all users to leverage.4
requests from the whole organization, and requests for the “Data doesn’t deliver value up to its full potential due to
analysis of different data sets from multiple domains sit in a traditional practices of centralization of enterprise data in
queue and wait their turn. warehouses or data lakes,” says Omar Khawaja, global head
Another drawback of a centralized system is that it can of business intelligence and analytics solution and services
endanger the reputation of data. Production of data that teams for Roche Diagnostics, a Basel, Switzerland-based
is not trustworthy or relevant is more likely to occur in a multinational health care company. “How can a centralized
centralized system because centralized data management team of 50 or 100 people in an enterprise which has 100,000

White Paper | Beyond Technology: Creating Business Value with Data Mesh

3
Harvard Business Review Analytic Services

“My recommendation is to approach data mesh with the 70-20-10


mindset. Seventy percent of the focus should be on change
management, 20% on data and technology, and 10% on models and
AI,” says Sanjeevan Bala, group chief data and AI officer at ITV.

people make the whole company data-driven? That’s just a understand what shows are about. When a certain product is
fool’s paradise.” spoken about in positive terms on a show, that helps the sales
team sell ads for that product. So, if there is talk about coffee
on an episode of Love Island, a reality dating show, the coffee
Empowering People with Data Mesh advertising does really well. “It’s a natural fit because there
For an organization to become data-driven and empower is no clash with advertising,” says Bala. “Data mesh opened
business teams with actionable insights, data needs to become up collaborations that didn’t exist before.”
a product, essentially a data set that is consumable by anyone Speed is among the top benefits of data mesh. With the
in the organization. Just like any other successful product, data products team in control of producing and using data,
a data product must create and fulfill a need or meet data there is no dependency on any cross-organizational team,
consumers’ expectations, be easily accessible, and be timely. which eliminates backlogs. Users are able to consume data
“Data should be accessible, interoperable, and reusable,” on a real-time basis. For ITV, having relevant data products
says Khawaja. “This means that the data product is published sped up the ability to activate segments promoting shows to
on an internal data marketplace so that people like data targeted viewers from three months to three minutes. “We
scientists, business analysts, or teams sitting in other domains see data mesh as an opportunity to leapfrog and accelerate
can access the data. They can then either generate insights our transformation,” says Bala.
on their own or use those data points to create other data
products to generate further insights by connecting the dots
between different domains.” The New Value-Driven Social Architecture
Data mesh offers a methodology for creating data products. Data mesh disrupts the traditional ideas of a centralized
Adidas’ Pratik, who implemented data mesh when she was system and the foundations of data management, which
at Saxo Bank, sees its value as giving control of the data means the approach must be embraced culturally. “It’s a
back to the business domains while setting up organization- big paradigm shift in terms of the way of working, the
wide protocols. mindset, team structure, technology, and architecture,” says
With this approach, business units become accountable and Roche’s Khawaja.
responsible both for how they produce and use the data for He explains all the major elements that were involved
their own business and for how useful their data products are in the shift to data mesh at Roche Diagnostics. “Instead of
to the whole organization. The way to implement data mesh, funding projects, we funded teams. Instead of having only
says Pratik, is to enable the business teams with technology IT people involved in data analytics, we had cross-functional
by combining business and technical teams. This way, the teams,” Khawaja asserts. “Instead of doing projects, we are
integrated team possesses all the required knowledge and talking about product thinking. Instead of working on a classic
skill sets to create a data product from end to end without on-premises, non-scalable infrastructure, we are using our
having to rely on handoffs to other teams. state-of-the-art, self-service data infrastructure. Instead of
ITV’s Bala, who started pursuing a data mesh approach two just creating dashboard after dashboard, we are talking about
years ago, explains, “Rather than creating a centralized team, a creating reusable data products.”
sort of an ivory tower, we fully decentralized both technology The product teams are responsible for the life cycle of
and data teams in such a way that they could sit with the that data product from end to end, adds Khawaja. With the
business units, in effect going native into the business.” implementation of data mesh, the silos between IT and
ITV has already seen multiple benefits from adopting data other business functions disappeared, and people joined
mesh. For example, data sharing and collaboration between product teams from different domains. “It no longer matters
the sales and content creation teams are bearing fruit in where people come from as long as they follow certain data
contextual advertising. The teams that create TV shows made governance boundaries,” he notes.
a lot of metadata available, which helped the sales team better

White Paper | Beyond Technology: Creating Business Value with Data Mesh

4
Harvard Business Review Analytic Services

In effect, data mesh helps solve the perennial challenge of


trying to make businesspeople understand technology teams,
and vice versa. Once people with different skills collaborate in
a data product team, they start complementing each other. For
example, a data product manager can talk to a colleague in IT
about the availability of certain types of data, and then they “[Within a centralized approach], data
can discuss the potential algorithmic uses of this data with analysts often know how to analyze
data analysts and teammates who are running the business. data, but they may not know why they
This approach makes the team aware of what data they have
and how to use it to create business value even before they
are analyzing it,” says Kevin Hong,
start building the data product, explains Kevin Hong, professor professor of business technology
of business technology at the University of Miami’s Miami at the University of Miami’s Miami
Herbert Business School.
“An important advantage of data mesh is that data scientists
Herbert Business School.
embedded in the domain teams are more likely to ask the
right questions of what can be accomplished with data
because they work closely with business leaders,” says Hong.
“[Within a centralized approach,] data analysts often know forcing a change in the way they have been operating, often
how to analyze data, but they may not know why they are very successfully, for years.
analyzing it.” The first step that Bala took when implementing data
But while data mesh solves many problems, this approach mesh at ITV was to look at the value chain. He engaged
also alters people’s roles and responsibilities, redefines business stakeholders by asking how using data and artificial
individual goals and accountability, and goes across traditional intelligence might change what they do when they produce,
reporting structures. Hong, who has taught multiple executive promote, distribute, or monetize content. “We were helping
courses on data analytics, says that executives often point stakeholders reimagine how their part of the business could
to power plays in organizations as the reason why data operate differently using data and AI,” says Bala.
initiatives do not translate to business outcomes. Simply The paradox of data mesh implementation is that while
put, data gatekeepers who under a centralized system have it ultimately drives business value, it is most often led by
power over the data—with everybody coming to them with technology executives who do not control marketing, sales, or
requests—lose that power when control of data is moved to production. It can be overwhelming for technology executives
the domains. to convince business leaders to embrace data mesh, says
“Organizations are doing a lot of things in terms of adopting Pratik, and it’s common for a tech-led initiative to reach a dead
the latest-generation technologies to enable data mesh. end at the business level. “If [the data mesh transformation]
However, where I’ve typically seen major problems is [not] becomes business-led, it will become magical,” she says.
educating people about the new roles and the accountability For data mesh to succeed, the C-suite should be involved.
that comes with it,” says Pratik. She suggests holding training The data mesh transformation at ITV was part of the company’s
workshops and educating both the business and the technology innovation agenda to drive growth, and it was supported by
sides about new ways of collaborating. the CFO, the management board, and the board of directors.
“Communication [about new roles and responsibilities with Since it was viewed as an organizational and cultural change,
data mesh] is key and often overlooked,” adds the University of versus just a technology implementation, the human resources
British Columbia’s Firican. “The necessary policies, processes, department also played a big role.
standards, roles, and responsibilities need to be established To get the business owners to embrace data mesh in their
in order to clarify who does what, when, and how, but that’s domains, Bala got the right stakeholders involved in mapping
not enough. Communication and overall change management the business benefits that could be achieved with data mesh.
need to be tightly coupled with data governance.” He used a “value tree,” which showed which data products lead
The goal of data mesh is to drive not technical outcomes but to the organization’s desired business outcomes. For example,
business outcomes. That’s why it makes sense for organizations a data product that could help predict how many and which
to start any data mesh transformation by establishing business viewers are likely to churn and what the churn rate depends
value that the data mesh will generate and engaging business on would help inform the business side about what changes
executives in the process from the very beginning. While to make to retain viewers and ultimately protect revenue.
many business executives may be technology savvy, the idea ITV runs control groups against the organization’s data
of a shift to data-driven management with data mesh is still product experimentation to prove value generated by

White Paper | Beyond Technology: Creating Business Value with Data Mesh

5
Harvard Business Review Analytic Services

“The biggest challenge to succeeding with data mesh is creating an


internal data marketplace with data producers and data consumers,”
says Sandra Cannon, the first chief data officer at the University of
Rochester and a data management consultant.

using data products. Control groups allow the company to With data mesh, data management standards are
understand which data products are working and which distributed, which means that every domain working on
data-driven KPIs lead to desired business outcomes. “The data products needs to follow them. The first step is to create
business knows what lever to pull to affect internal KPIs— standards that are distributable and applicable across the
such as monthly active user target or viewing hours targets organization. This step requires creating a master data
or churn—and how these internal data-driven KPIs impact management standard that describes standard definitions
revenues,” says Bala. “That’s how we got the business on our and rules—a common data language, so to speak—to make
side.” Bala’s team now finds itself in the enviable position of sure things don’t get lost in translation between the domains,
being asked by other business executives for help with a data explains Abadi.
mesh approach to growing their business lines. The next step is making sure that data teams are maintaining
Other companies also have had significant success these standards in their products. “Creating consistent
with data mesh. At Roche, data used by manufacturing data requires a lot of work from upstream data products’
planning teams is now available every day, instead of producers,” says Abadi. When, for instance, an American
every month or every quarter. Having implemented data and a European company merge, data products involving
mesh and thus eliminated the bottlenecks of a centralized the newly combined company’s prices need to be compatible.
system, the organization is now able to produce actionable However, the different currencies might mean that data sets
insights from data much faster. Data products highlight cannot be automatically combined. The pricing then needs
up-to-the-moment details of the different materials required to be accurately converted and cleaned up so that when data
in the inventory. And access to timely data insights leads to consumers query different data products, including pricing,
savings in maintaining material inventory and improved they get correct information. And different functions within
efficiency in the manufacturing process. Just last year, a both companies may require different data points. A finance
reduction in the stock safety requirements, achieved thanks department of the overall organization may need converted
to having up-to-date stock data, delivered savings in terms pricing to project overall sales, but marketing departments
of capital investment. in different countries may need pricing in local currencies,
with or without discounts, to establish how to tailor loyalty
programs to their customers.
Creating an Effective Data In order to incentivize and reward data producers, it’s
Products Marketplace necessary to track all the producers’ contributions to the data
“The biggest challenge to succeeding with data mesh is products, notes Abadi. Since many of the products incorporate
creating an internal data marketplace with data producers other data elements and are repackaged as new products, the
and data consumers,” says Sandra Cannon, the first chief data majority of the data teams are both data consumers and data
officer at the Rochester, N.Y.-based University of Rochester and producers. They consume the upstream data products, and
now a data management consultant to government and not- they produce new ones, which are consumable downstream.
for-profit organizations. “You need to make sure that you’ve As an example, the upstream data team may track transactions,
got the right incentives built into the roles and responsibilities while a downstream team may take this data product and add
for the people who are doing all the work. You need to address other data—such as customer locations or identities—and
the issue of what’s in it for them.” make it more valuable.
Much of the work that goes into creating data products is “If you create new data products with references to the
related to data governance. “Data governance is the biggest original data sets, you know where the data is coming from,
stumbling block of data mesh. If you don’t do it right, data and if the product is valuable, you know whom to reward,”
mesh is not going to work,” says the University of Maryland’s says Abadi. “Then the producers are going to be incentivized
Abadi, who performs research on database system architecture and create more products, which will make other people’s
and implementation. lives easier.”

White Paper | Beyond Technology: Creating Business Value with Data Mesh

6
“The business knows what
lever to pull to affect
internal KPIs—such as
monthly active user target
or viewing hours targets
or churn—and how these
internal data-driven KPIs
impact revenues. That’s
how we got the business
on our side.”
Sanjeevan Bala, group chief data
and AI officer at ITV.
Harvard Business Review Analytic Services

“Technology is a key enabler, but without the focus on people,


technology-driven initiatives will become derailed,” says Roche
Diagnostics’ Khawaja.

Spearheading Innovation with Data Mesh It is important that organizations design a functioning
Data mesh has the potential to help companies succeed at data products marketplace. An effective marketplace can be
digital transformations aimed at making them data-driven achieved through the promotion of a healthy data producers
because it addresses the need to transform social architecture and consumers relationship based on a system of incentives
while implementing the technology architecture. But data in which everyone who produces value gets rewarded.
mesh can bring challenges that organizations need to be Organizations should set up the incentives, provide self-service
ready to address. technology platforms, and promote teams’ collaboration in
Traditionally, companies have been focusing on technology a way that enables more and more people to become value-
while not paying enough attention to change management, add data producers.
which is why so many digital transformations have failed. Adopt a “thin-slice, iterative approach,” says Bala, which
“While the right technology choices will enable teams to work means working with small, manageable pieces of infrastructure
in data mesh, do not start with technology,” says Khawaja. and small amounts of data and developing these thin-slice
“Technology is a key enabler, but without the focus on people, use cases end to end. After showing value from these first use
technology-driven initiatives will become derailed.” cases, organizations can build from there. While the start of
Bala advises starting the implementation of data mesh with data mesh may be based on a thin-slice strategy, Bala sees the
the business value and focusing on the “last mile,” which is potential of data mesh not just going enterprise-wide but also
when data consumers get the data product “in their hands.” crossing the boundaries between companies and industries.
This moment of truth reveals whether the created data “As different organizations enable data mesh, it will allow us
products are useful. Before starting to build any models, to use data across industries. With the hyperconnectivity going
organizations need to make sure that marketers, operators, up a level and crossing sectors, we will be able to spearhead
planners, and other business lines will have easy, self-service huge amounts of innovation,” he says.
tools to access the models. They also need to be involved in the
process to make sure the models will yield business outcomes.

Endnotes

1 Foundry, “Data & Analytics Study,” 2022.


2 IDG Market Pulse Survey, “Optimizing Business Analytics by Transforming Data in the Cloud,” 2022.
3 Foundry, “Data & Analytics Study,” 2022.
4 Ibid.

White Paper | Beyond Technology: Creating Business Value with Data Mesh

8
ABOUT US

Harvard Business Review Analytic Services is an independent commercial research


unit within Harvard Business Review Group, conducting research and comparative
analysis on important management challenges and emerging business opportunities.
Seeking to provide business intelligence and peer-group insight, each report is
published based on the findings of original quantitative and/or qualitative research
and analysis. Quantitative surveys are conducted with the HBR Advisory Council,
HBR’s global research panel, and qualitative research is conducted with senior
business executives and subject matter experts from within and beyond the Harvard
Business Review author community. Email us at [email protected].

hbr.org/hbr-analytic-services

Copyright © 2022 Harvard Business School Publishing. MCCRE32170922

You might also like