Aj21 en
Aj21 en
Aj21 en
Learn the discipline, pursue the art, and contribute ideas at www.architecturejournal.net
input for better outcomes
Design Considerations for Software plus Services and Cloud Computing Model Driven SOA withOSLO An Enterprise Architecture Strategy for SOA
Enabling Business Capabilities with SOA Service Registry: A Key Piece for Enhancing Reuse in SOA How the Cloud Stretches the SOA Scope Event-Driven Architecture: SOAThrough the LookingGlass Is SOA Being Pushed Beyond Its Limits?
Contents
Foreword
by Diego Dagum 1
21
10
16
24
29
36
42
48
Founder Arvindra Sehmi Director Lucinda Rowley Editor-in-Chief Diego Dagum, Eliaz Tobias (Guest editor) Editorial Board Jesus Hernandez Sanchez, Martin Salias, LindaChong, Mike Cramer, Kazuyuki Nomura Editorial and Production Services WASSER Studios Dionne Malatesta, Program Manager Ismael Marrero, Editor Dennis Thompson, Design Director
Foreword
Dear Architect,
hen the first Architecture Journal was published in January 2004 (back then, called Microsoft Architects Journal ), it contained an article, Understanding ServiceOriented Architecture, in which two CBDI analysts explored a novel concept of application integration. At that time, most organizations ignored SOA and its implied concepts. Yet, in those companies that had some degree of SOA awareness, there were few or no ongoing projects that went in that direction. A few years later, it was rather common to hear of a given organization thinking to realign its whole IT strategy with an SOA model. While they were clear on the goal, the business justification, and the benefits to be had, the initial attempts were still not clear on the right path to take or (even more CIO-enervating) the accuracy of initial cost estimations. More than five years after that introductory article, thousands of SOA projects have been kicked off, and several of them are still being completed. These days, we have a better understanding about its ROI, low-hanging fruits, hidden costs, and other realities; and the basic, incipient SOA discussion is today specialized in a variety of threadsfrom service-related ones, such as versioning and discoverability, to broader, portfolio-level threads, such as business-process management. As if everything about SOA hasnt already been said, other IT trends insinuate a sudden irruption that could twist the original course of SOA by using alternative techniques in noncritical scenarios. These include the modest, lightweight approach that REST offers, as opposed to the rich but heavier OASIS WS-* standardsthe latter having matured and evolved with SOA as a purpose. This issue of The Architecture Journal covers several of these debates. The main difference with the article that we published in the early days is that, this time, thoughts have emerged as a consequence of a practice; in 2004, thoughts had emerged as a consequence of a vision. Both perspectives are necessary: the vision, to understand the goal and what we intend to achieve, and the practice, to help us understand the real dimension of constraints and how to mitigate (if not avoid) them successfully. These eight articles discuss aspects such as business alignment, service modeling, governance, federation, infrastructure, reusability, convergence, and coexistence with cloud computing and event-driven complementary approaches (among others), and theyre accompanied by guest columns that analyze other tangential aspects. Id like to stop here to thank Eliaz Tobias, IT architect and SOA expert from Microsoft Israel, for his collaboration as guest editor-in-chief. As we promised two issues ago, we were in the process of leveraging more digital formats. In that sense, this set of articles and guest columns is complemented by a series of short videos on yet other aspects of this SOA topic. Since our first issue, SOA has walked a long trail, and we still expect a long journey ahead. However, this checkpoint along our way is a great opportunity to share and reaffirm certain concepts that will be useful for the remainder of our journey. I hope that you enjoy these articles, dear reader. As usual, you may send us your comments at [email protected].
The information contained in The Architecture Journal (Journal ) is for information purposes only. The material in the Journal does not constitute the opinion of Microsoft Corporation (Microsoft) or Microsofts advice and you should not rely on any material in this Journal without seeking independent advice. Microsoft does not make any warranty or representation as to the accuracy or fitness for purpose of any material in this Journal and in no event does Microsoft accept liability of any description, including liability for negligence (except for personal injury or death), for any damages or losses (including, without limitation, loss of business, revenue, profits, or consequential loss) whatsoever resulting from use of this Journal. The Journal may contain technical inaccuracies and typographical errors. The Journal may be updated from time to time and may at times be out of date. Microsoft accepts no responsibility for keeping the information in this Journal up to date or liability for any failure to do so. This Journal contains material submitted and created by third parties. To the maximum extent permitted by applicable law, Microsoft excludes all liability for any illegality arising from or error, omission or inaccuracy in this Journal and Microsoft takes no responsibility for such third party material. ANY CARNEGIE MELLON UNIVERSITY AND SOFTWARE ENGINEERING INSTITUTE MATERIAL CONTAINED HEREIN IS FURNISHED ON AN AS-IS BASIS. CARNEGIE MELLON UNIVERSITY MAKES NO WARRANTIES OF ANY KIND, EITHER EXPRESSED OR IMPLIED, AS TO ANY MATTER INCLUDING, BUT NOT LIMITED TO, WARRANTY OF FITNESS FOR PURPOSE OR MERCHANTABILITY, EXCLUSIVITY, OR RESULTS OBTAINED FROM USE OF THE MATERIAL. CARNEGIE MELLON UNIVERSITY DOES NOT MAKE ANY WARRANTY OF ANY KIND WITH RESPECT TO FREEDOM FROM PATENT, TRADEMARK, OR COPYRIGHT INFRINGEMENT. A list of Microsoft Corporation trademarks can be found at http://www.microsoft.com/library/toolbar/3.0/trademarks /en-us.mspx. Other trademarks or trade names mentioned herein are the property of their respective owners. All copyright and other intellectual property rights in the material contained in the Journal belong, or are licensed to, Microsoft Corporation. You may not copy, reproduce, transmit, store, adapt or modify the layout or content of this Journal without the prior written consent of Microsoft Corporation and the individual authors. Copyright 2009 Microsoft Corporation. All rights reserved.
Summary The purpose of this article is to share our thoughts about the design patterns for a new generation of applications that are referred to as Software plus Services, cloud computing, or hybrid computing. The article provides a view into S+S architectural considerations and patterns as they affect common architectural domains such as enterprise, software, andinfrastructure architecture.
Introduction
Many enterprises have IT infrastructures that grew organically to meet immediate requirements, instead of following a systematic master plan. Organically grown enterprise systems have a tendency to develop into large, monolithic structures that consist of many subsystems that are either tightly coupled or completely segregated (sometimes referred to as a siloed system). Typically, these systems have arcane and inconsistent interfaces. Their complexity and inefficiency slows down business innovation and can force IT managers to focus on operational and firefighting processes instead of on how information technology can support the core business. Furthermore, some enterprise IT systems have partially duplicated functions that lead to fragmented and inconsistent views of business information, which affects the ability of an enterprise to make sound financial decisions. Software plus Services (S+S) is an extension of Software as a Service (SaaS) that offers organizations more options for outsourcing development, management, deployment, and operational aspects of the technologies that run their businesses. S+S works in conjunction with principles of service-oriented architecture (SOA). S+S helps an SOA-enabled enterprise increase its technology choices by providing multiple modes of sourcing, financing, and deploying application software and services. To make informed decisions and take full advantage of the potential benefits of adopting an S+S model, IT architects and decision makers should weigh the business drivers and technical requirements against the economic, regulatory, political, and financial forces that are at work from both inside and outside the company. This article is based on practical experience that was gained by the Microsoft Worldwide Services consulting organization during the design and delivery of S+S and Cloud-based applications. It provides aview into S+S architectural considerations and patterns as they affect common architectural domains, such as enterprise, software, and infrastructure architecture.
2
Management
Software Architecture
Integration design Application design Information design
Operations
Security
Infrastructure Architecture
Client On premises Off premises
Design Considerations
This section provides a summary of the business and technical challenges that a business should consider during the design or adoption of an S+Sbased solution. Figure 2 illustrates the frame that is used to organize this document. The frame is organized around specific architectural perspectives and identifies the crosscutting concerns, to provide an end-to-end perspective on the types of scenarios, design considerations, and patterns to consider as part of an S+S strategy. This information provides a basis for evaluating the end-to-end implications of adopting S+S strategies. Enterprise Architecture One of the most demanding aspects of the enterprise-architect role is
to balance the constantly changing business needs with the ability of the IT organization to meet those needs consistently. S+S introduces new technology-deployment patterns that reduce operational expense by consolidatingand, in some cases outsourcingIT platforms, applications, or application (business) services. In addition, S+S can enable organizations to integrate systems across the organization with less friction. Organizations can provide information services to existing business relationships, often by combining existing channels. At the highest level, enterprise architects must establish a means of determining the core competencies of the organization and then establish a process for determining which applications support those core competencies, as well as which should perhaps remain in-house and which should not. The following is a model that is used by several large organizations: Proprietary and mission-critical systems Systems that are proprietary or mission-critical in nature or that provide competitive advantages are often considered too important to risk outsourcing to an off-premises service provider. As a result, these systems are usually designed, developed, operated, and managed by the existing IT department of an organization. Nonproprietary and mission-critical systems Systems that are nonproprietary yet still mission-critical might be developed by another company, but might still be designed, operated, and managed by the existing IT department an organization. Nonproprietary systems Systems that are nonproprietary and deliver standardized functionality and interfaces are often good candidates for outsourcing to a cloud-service provider if appropriate service-level agreements (SLAs) can be established with the service providers. E-mail, calendaring, and contentmanagement tools are examples of such systems. This model provides a starting point to evaluate applications and systems, but organizations should take into account their individual organizational differences. For example, if an organization is unable to manage its core systems effectively because of cost or missing expertise, it might consider outsourcing them. Likewise, putting some mission-critical systems in the Cloud might offer additional capabilities at little cost that can offset the drawbacks that are introduced. An example might be allowing access to the system by trusted partners or company branches, without having to build an in-house dedicated infrastructure.
3
Low Business silos Standardized technology Optimized core Business modularity Dynamic venturing
Enterprise-Architecture Maturity
However, simply identifying opportunities for moving applications off-premises is insufficient. To leverage S+S opportunities, decision makers must have a clear understanding of the IT maturity of the organization. This understanding allows them to determine what changes in IT infrastructure and processes should be made to optimize the return on investment (ROI) or cost savings that can be gained through S+S adoption. Figure 3 illustrates the ease of adoption for S+S at varying levels of IT maturity (maturity model based on Enterprise Architecture as Strategy1) and demonstrates that without determining the organizational maturity, the envisioned ROI might be incorrect. Software Architecture, Integration Design Few enterprise applications exist in isolation. Most are connected to other applications and form complex systems that are interconnected through a variety of techniques such as data integration, functional integration, and presentation integration. In most cases, organizations use a variety of integration techniques, which results in tightly coupled systems that are difficult to separate and replace with off-premises capabilities. Typically, in such cases, the organization either establishes course-grained facades around subsets of functionality within its subsystems or adopts integration technologies that provide a bridge between legacy applications and services that could be hosted locally or off-premises. When it integrates at the data layer and allows off-premises applications to use the same data as on-premises applications, an organization must consider a variety of factors, such as where the master data should reside. If the data is read-only or reference data, it might be possible to use push-or-pull replication techniques to keep the data synchronized. For business or transactional data, the organization must consider other techniques. Organizations that use functional SOA-based business services can consider migrating these services to the Cloud, which is discussed in greater detail in the next section. However, in some cases, business applications cannot easily be partitioned into service contractdriven clients and service-provider components. This might be the case when the system involves complex legacy processes and human-driven workflows. In these cases, it might be possible to move the workflow into the Cloud and support a hybrid mode of operation in which the workflow can span both online and offline scenarios. Traditional atomic approaches to managing transactions might no longer be possible, which would require the organization to examine alternative models that can ensure data consistency. The information4
3. Security of service A security protocol (for example, WS-Security), use of a digital signature (yes/no), use of encryption (yes/no), and the type of security token that is used (for example, X.509 and Kerberos) 4. Technical details A network protocol (for example, HTTP and JMS), a messaging style (RPC or document), an encoding style (SOAP-encoded or literal), and the nature of composition (atomic or composed) 5. Commercials One-time cost, annual ongoing cost, and servicelevel-agreement (SLA) commitments Out of these 19 parameters, the architect classified all parameters from functional contract and security of services categories as eliminating parameters. She noted that for eliminating parameters, the requirements are definitive and rigid and she can simply disqualify those candidate services that dont fulfill these requirements. She then went ahead and collected data for all the parameters for all candidate services. She needed to disqualify two candidate services, one for not using required WS-Security protocol and other one for not using digital certificate (as required). All other three services survived the qualification test against eliminating parameters. Next the application architect assigned weights to evaluation parameters. She used Analytic Hierarchy Process (AHP) to prioritize the parameters from the stakeholders point of view. Following up the last three steps led to one of the web services receiving the highest composite score. That web service was then selected during the development of Global Quality Management System.
Shrikant Mulik ([email protected]) is Lead Consultant at L&T Infotech, Mumbai. Manish Godse ([email protected]) is a Research Scholar at the Indian Institute of Technology (IIT), Bombay.
enterprise can derive immediate infrastructure cost savings by replicating virtual server instances to run on the cloud infrastructure as business needs require. While cloud infrastructurerelated services can bring many benefits that were not previously available to enterprises, the advantages do not come for free. IT architects must continue to weigh design considerations that concern availability, scalability, security, reliability, and manageability while they plan and implement a hybrid S+S infrastructure. Infrastructure service disruptions will ultimately affect the availability of application services. Because a number of higherlevel application services might be running on an outsourced cloud infrastructure, an outage at the service-provider infrastructure could affect more than one business functionresulting in the loss of business productivity or revenue in multiple areas. Therefore, enterprises should know whether their infrastructure-service providers can help mitigate such failures. Alternatively, an enterprise might want to use multiple infrastructure-service providers so that it can activate computing resources at the secondary provider in the event of service failure at the primary provider. When desktop-virtualization technology is delivered through a centralized hosting infrastructure, it is important to consider scalability and performance of the solution. Often, desktopvirtualization services are at peak load during office hours when employees log on and perform office tasks. The load gradually tapers off after-hours. Combining virtualization technology with a dynamically expandable computing infrastructure service can be a good approach when the computational demands of an organization fluctuate. Infrastructure security has always been part of the defensein-depth strategy for securing the enterprise. For example, some enterprises rely on IPSec for protecting machine-to-machine communications within the intranet. This mechanism can add another layer of defense to protect against unauthorized information access by non-corporate-owned computing devices. To continue using existing infrastructure-level security mechanisms with cloudinfrastructure services, an enterprise might need to reconfigure its network, public-key, and name-resolution infrastructure. When the server infrastructure is deployed to run as virtualized instances, IT architects should think about organizing the virtual instances into atomic units, where a service failure is isolated within each unit and does not affect other atomic collections of virtualized services. This infrastructure-design practice enables higher application services to be deployed as atomic units that can be swapped in if the virtualized instances fail to operate in another unit. When higher-level applications are deployed across a hybrid S+S infrastructure, it can be difficult to debug application failures that occur because of infrastructure malfunction. Traditional networkmonitoring and tracing tools that are used within the enterprise might cease to work across the boundaries of corporate and serviceprovider firewalls. Therefore, an enterprise can request that its cloudinfrastructure providers provide diagnostic tools that can help inspect cloud-infrastructure traffic. Security Security has been a key area of enterprise computing focus since the late 1990s, when businesses began using the Internet as a mainstream channel of commerce and customer service. In the current computing era of S+S, the security best practices and the technology developed to serve the business Web not only remain relevant, but are even more important to observe.
Processes
Infrastructure
INTELLIGENZ
Information
Communication
the whole enterprise to guarantee a fully functional and businesssupporting environment. In the course of a research project, we set up a virtualized fictitious company that we used as a prototypical scenario to check out the validity of our concepts. In conclusion, we are now able to demonstrate that the approach to induce the IEA design paradigm works perfectly. These consolidated findings might be seen as a successful, pioneering step in the area of cutting-edge EA design principles.
Mario Frai ([email protected], http://www.mariofraiss.com) is the founder and CEO of FRAISS IT Consulting & Media Design, an Austrian technology-consulting company that specializes in developing innovative business solutions that are based on Microsoft technologies. Erwin Zinser ([email protected], http://www.entology.eu) is Professor of Enterprise Architecture Design at the FH JOANNEUM University of Applied Sciences, Department of Information Management, in Graz, Austria.
S+S security covers a broad spectrum of topics, ranging from the provisioning of identities and their entitlements, to enabling enterprise single sign-on between on-premises systems and cloud services, to protecting data in transit and at rest, to hardening application code deployed on cloud platforms against malware and penetration attacks. User provisioning is a key task in the life-cycle management of user identities. When an enterprise adopts a cloud service, it must consider how its enterprise users are provisioned with the cloudservice providers. In addition, as a users organizational role changes, the identity management processes should ensure that the users application permissions are adjusted accordingly at the cloud service. When a user leaves the enterprise, access to the cloud service should also be deactivated. The user provisioning activities for S+S should be automated as much as possible to reduce manual provisioning errors and prevent loss of employee productivity that is due to serviceaccess issues. Enabling single sign-on (SSO) by using existing corporate identities
is a key requirement and priority for many enterprises that adopt cloud services. The reasons are obvious. SSO provides convenience and better application experiences to end users and can reduce security issues that arise from having to manage multiple security credentials. Rationalizing and consolidating multiple identity systems within the enterprise is usually the first step in meeting the SSO challenge. New identity-federation technology can also improve the portability of existing user credentials and permissions and should definitely be a key part of the SSO strategy with cloud-service providers. Data is critically important in every business. Therefore, an enterprise should set a high bar for ensuring that its business information continues to be secure in the S+S world. The key security issues that concern data are confidentiality and integrity when data is transmitted over the Internet and when information is stored at the cloud-service provider. Security mechanisms such as encryption and signing can help ensure that data is not being viewed or modified by unauthorized personnel.
7
documentation that complies with auditing standards such as SAS70, which can help the enterprise determine if the IT-management practices at the service provider meet their business and industry requirements. Enterprise organizations should plan to deploy IT-management solutions for monitoring services that run in the Cloud. The operation strategy should outline the performance indicators and management rules that are required to gain visibility into the performance and availability of the external services. Operational monitoring systems should raise management notifications and alerts, so that any service anomaly can be detected early. Additionally, both outsourced service providers and enterprises that are developing cloud services should implement operationrelated service interfaces to automate management tasks such as provisioning user accounts, setting user permissions, changing service run state, and initiating data backups. In summary, IT management in the S+S world must continue to embrace the end-to-end strategy of planning, delivering, and operating the IT capabilities that are needed to support business operations. Existing IT-management frameworks are still relevant. Enterprises, however, should consider the impact that arises as they integrate external operation processes, personnel, and tools into the existing IT-management practices. With external service providers taking responsibility for systems, organizations lose much of the direct control that they used to have over the people, processes, and technology. Instead, the organizations must provide effective management through clearly defined and mutually agreed-upon SLAs, policies and procedures, key performance indicators, management rules, and servicecontrol interfaces. Design for operation is the architecture mantra for delivering manageable software and services. Ultimately, the outsourcing of operational details to cloud-service providers should empower existing IT staff to focus on new, high-priority computing projects that deliver greater business value. Operations Operations make up a very specific stage of the IT-management life cycle. It involves the day-to-day activities of monitoring software and services, taking corrective actions when problems arise, managing customer helpdesks to resolve user issues, performing routine tasks such as backing up data, and controlling and maintaining consistent service run states to meet the required quality of service. Operational procedures are governed by IT policies, and the outcome is measured by precise systems and applications health metrics such as availability and response times. For example, the MOF outlines a best-practices model for these activities. As enterprises adopt an S+S strategy, they must consider the business impact of outsourcing IT operational roles and responsibilities. Business continuity, liability, and employee and customer satisfaction are all key concerns that must be addressed by establishing clear SLAs with reliable cloud-service providers. The enterprise should continue to play a proactive role in IT operations for its hybrid software-and-services environment. However, instead of focusing on execution details, enterprises should put monitoring systems in place that enable them to detect technical issues at the outsourced services. Enterprises should also establish operational procedures to ensure that problems are resolved by the service providers as quickly as possible. Both enterprises and cloud-service providers can increase their S+S operational effectiveness by designing for operational best practices.
Acknowledgments
The authors would like to thank the following reviewers: Tim OBrien, Rob Boucher Jr., and Sharon Smith.
Resources
Meier, J.D., Alex Homer, David Hill, Jason Taylor, Prashant Bansode, Lonnie Wall, Rob Boucher Jr, and Akshay Bogawat. Microsoft patterns & practices Application Architecture Guide2.0: Designing Applications on the .NET Platform. January 15, 2008. (SeeChapter13, Service Layer Guidelines, and Chapter 18, Services.)
References
1
Conclusion
S+S brings new opportunities for everyone. It provides new options for optimizing business and IT assets, and enables organizations to save cost, increase productivity, innovate, and reach new markets. There are three main ways to think about extending the current portfolios of on-premises technology with cloud computing: consume the Cloud, use the Cloud, and embrace the Cloud: Consume the Cloud is fundamentally about an enterprise outsourcing applications and IT services to third-party cloud providers. The key business drivers that push enterprises to consume online services are reducing IT expenses and refocusing valuable bandwidth on enabling core business capabilities. Cloud providers can usually offer commodity services cheaper and better because of their economy of scale. They can pass-on the cost savings and efficiency to enterprise customers. For Microsoft customers, some good examples are the Microsoft Business Productivity Online Suite (consisting of Exchange Online and Office SharePoint Online), CRM Online, and Live Meeting services. Use the Cloud enables enterprises to tap into cloud platforms and infrastructure services and get an unlimited amount of compute and storage capacity when they need it, without having to make large, upfront capital investments in hardware and infrastructure software. Such a utility computing model gives enterprises more agility in acquiring IT resources to meet dynamic business demands. In addition, by using cloud services, enterprises can avoid affecting the existing corporate infrastructure and speed up the deployment of Web-based applications to support new business initiatives that seek closer communication with customers and partners. For Microsoft customers, some good examples include Windows Azure and SQL Azure. Embrace the Cloud occurs when enterprises deploy technology that enables them to offer cloud services to their customers and partners. Service-oriented enterprises are best positioned to take advantage of this model by transforming their core business assets into information cloud services that differentiate them from their competitors. For Microsoft customers, on-premises technologies such as the BizTalk Server Enterprise Service Bus Toolkit can integrate data feeds and orchestrate workflows that process information exchange via cloud services.
4
Ross, Jeanne W., Peter Weill, and David Robertson. Enterprise Architecture as Strategy: Creating a Foundation for Business Execution. Boston, MA: Harvard Business School Press, 2006. Skonnard, Aaron. Service Virtualization with the Managed Services Engine. MSDN Magazine, May 2009. Ferguson, Donald F., Dennis Pilarinos, and John Shewchuk. TheInternet Service Bus. The Architecture Journal, MSDN, October 2007. Khalidi, Yousef A. Architecting Services for Windows Azure. Professional Developers Conference 2008 (PDC2008), 2008, Slide15. Bain, Tony. Is the Relational Database Doomed? ReadWriteEnterprise, February 12, 2009. Pritchett, Dan. BASE: An Acid Alternative. ACM Queue, July 28, 2008. ITIL. Information Technology Infrastructure Library. Official ITIL Website, 2009. Microsoft Corporation. Microsoft Operations Framework. Microsoft TechNet, 2009.
Fred Chong is working on new and emerging market solutions, integrating mobile phones, software, and cloud services to improve the standard of living and productivity of citizens in developing countries. Alejandro Miguel, Jason Hogg, and Joshy Joseph are architects inside the Solution Engineering team within the Worldwide Services Office of the CTO. Ulrich Homann is the Chief Architect for WorldWide Enterprise Services. Brant Zwiefel is a business architect in the Microsoft Services Service Lines and Marketing team. Danny Garber is a U.S. Azure Community Lead Architect within the Architecture Team of the CTO. Scott Zimmerman is an SOA and BizTalk Solutions Architect in the U.S. Mid-Atlantic District who advises customers on S+S. Stephen Kaufman is a delivery architect who focuses on middle-tier technologies with Microsoft Consulting Services.
Summary Service-oriented architecture (SOA) must evolve toward a more agile design, development, and deployment process. Most of all, it must close the gap between IT and business. This article presents a map between modeling theory (model-driven development and model-driven SOA) and possible implementations with the next wave of Microsoft modeling technology, codename Oslo.
Introduction
The following article presents a map between modeling theory (model-driven development and model-driven SOA) and possible implementations with the next wave of Microsoft modeling technology, codename Oslo. Microsoft has been disclosing much information about Oslo while delivering several Community Technology Previews (CTPs), or early betas. However, most of the information that is available on Oslo is very technology-focused (internal M-language implementation, SDK, and so on). This is why I want to present a higher-level approach. Basically, I want to discuss Why modeling, instead of only How. Figure 2: SOA architecture, based on ESB as central point
Figure 1, in which each circle would be a Figure 1: Point-to-point services network service and each box a whole application. The problem with this services network is that all of these services are directly connected; we have too many point-topoint connections between services. Using point-to-point services connections is fine, when we have only a few services; however, when the SOA complexity of our organization evolves and grows (having too many services), this approach is unmanageable. Sure, you will say that we can improve the previous model by using an enterprise-service-bus (ESB) approach and service-orchestration platforms, as I show in Figure 2. Even then, however, the complexity
What is the most important problem in IT? Is it languages, tools, programmers? Well, according to researchers and business users, it is software complexity. And this has been the main problem since computers were born. Application development is really a costly process, and software requirements are only increasing. Integration, availability, reliability, scalability, security, and integrity/compliancy are becoming more complicated issues, even as they become more critical. Most solutions today require the use of a collection of technologies, not only one. At the same time, the cost to maintain existing software is rising. In many aspects, enterprise applications have evolved into something that is too complex to be really effective and agile. With regard to service-oriented architecture (SOA), when an organization has many connected services, the logical network can become extremely difficult to manage something that is similar to what is shown in
10
Identity Management
Authentication Single sign-on Authorization Represent.
Requests Processor
Forms Workflow Forms generator Telematic archive
Horizontal Services
Notifications Monitoring Electronic sign Payment gateway
ESB
Services interfaces
BPM/EAI
Legacy applications
Component A
Component B
Interpreter
Interpreter
Objects
Objects
Application Component
Service container
Application Component
Although it is highly appealing to build components in this manner, the major architectural challenge is to ensure reliability of the components and their services so as to prevent inherent failures of the component from percolating upwards. One way to ensure the reliability of programmable component architectures is to model the component in such a way that its SLAs are queried and controlled programmatically. Architectural techniques that ensure this reliability include the design of components that: Offer programmatically controlled SLAs via a scripting engine as the primary API. Are state-aware and measure and publish their health and capacity to deliver against SLAs enabling graceful failover. Have longer design lifetimes, due to design that is centered on variables that change less often. Are resilient to environmental failures. Manage the effects of environmental dynamism by using designtime controls. These architectural techniques are described in detail in ourblog.
Platform Service
Nagaraju Pappu is the founder of and chief technology consultant at Canopus Consulting.
is very high: Implementation is based on a low level (programming in languages such as the Microsoft .NET languages and Java); therefore, its maintenance and evolution costs are quite high, although not as high as having non-SOA applications. SOA foundations are right; however, we must improve the work when we design, build, and maintain.
On the other hand, SOA has been the promised land and a main IT objective for a long time. There are many examples in SOA in which the theory is perfect, but its implementation is not. The reality is that there are too many obstacles and problems in the implementation of SOA. Interoperability among different platforms, as well as real autonomous services being consumed from unknown applications, really are headaches and slow processes. SOA promised a functionality separation of a higher level than object-oriented programming and, especially, a much better decoupled components/services architecture that would
decrease external complexity. Additionally, separation of services implementation from services orchestration results in subsystems being much more interchangeable during orchestration. The SOA theory seems correct. But, what is the reality? In the experiences of most organizations, SOA in its pure essence has not properly worked out. Dont get me wrong; services orientation has been very beneficial, in most situationsfor instance, when using services to connect presentation layers to business layers, or even when connecting different services and applications. However, when we talk about the spirit of SOA (such as the four tenets), the ultimate goals are the following: In SOA, we can have many autonomous services independently evolvingwithout knowing who is going to use my services or how my services are going to be consumedand those services should even be naturally connected.
11
A few months ago, a question arose in many architectural forums. It probably was started by Anne Thomas Manes (Burton Group) in a blog post called SOA Is Dead; Long Live Services. So, is SOA dead? I truly dont think so. SOA foundations are completely necessary, and we have moved forward in aspects such as decoupling and interoperability (when compared with separate worlds such as CORBA and COM). So, dont step back; I am convinced that service orientation is very beneficial to the industry. The question and thoughts from Anne were really very interesting, and she really was shaking the architecture and IT communities. She did not really mean that service orientation is out of order; basically, what she said is the following: SOA was supposed to reduce costs and increase agility on a massive scale. Except in rare situations, SOA has failed to deliver its promised benefits... . Although the word SOA is dead, the requirement for service-oriented architecture is stronger than ever. But perhaps thats the challenge: The acronym got in the way. People forgot what SOA stands for. They were too wrapped up in silly technology debates (e.g., Whats the best ESB? or WS-* vs. REST), and they missed the important stuff: architecture and services. So, I dont believe that SOA is dead at all, nor do I think that Anne meant that. She was speaking out against the misused SOA word. Note that she even said that the requirement for service-oriented architecture is stronger than ever. The important points are the architecture and services. Overall, however, I think that we need something more. SOA must become much more agile. Furthermore, at a business level, companies every day are requiring a shorter time to market (TTM). In theory, SOA was going to be the solution to that problem, as it promised flexibility and quick changes. But the reality is a bit sadprobably not because the SOA
Is SOA Dead?
Runtime/Executables
Ultimately, organizations must close the gap between IT and business by improving the communication and collaboration between them. The question is, Who has to come up? Probably, both. IThas to come closer to business and be much more accessible and friendly. At the same time, business experts have to reach a higher level to be able to leverage their knowledge more directly. They have to manipulate technology, but in a new way, because they still have to focus on the business; they do not have to learn how to program a service or application by using a low-level language such as C#, VB, or Java. Someday, model-driven SOA might solve this problem. I am not talking only about services orchestration (by using platforms such as Microsoft BizTalk Server); I mean something elsea one-step-forward levelin which orchestration has to be managed not by technical people, but by business experts who have only a pinch of technical talent. This is something far from the context of today, in which we need technical people for most application changes. I mean a new point of viewone that is closer to the business side. Model-driven SOA will have many advantages, although we will have to face many challenges. But the goal is to solve most typical SOA problems, because model-driven development (MDD) makes the following claim: The model is the code. (See Figure 3.) The advantages and claims of model-driven SOA are the following: 1. The model is the code. Neither compilation nor translation should be necessary. This is a great advantage over code generators and finally drives us to maintain/update the code directly. 2. Solutions that model-driven SOA creates will be compliant with SOA principles, because we are still relying on SOA. What we are changing is only the way in which we will create and orchestrate services. 3. Models have to be defined by business languages. At that very moment, we will have achieved our goal: to close the gap between IT and business, between CIO and CEO. 4. We will consequently get our desired flexibility and agility (promised by SOA, but rarely achieved nowadays) when most core business processes are transformed to the new model. When we use model-driven SOA, changes in business processes have to be made in the model; then, the software automatically changes its behavior. This is the required alignment between IT and business. This is the new promise, the new challenge.
Application
Business Users
12
Microsoft is actually building the foundations to achieve MDD through a base technology, codename Oslo. Microsoft has been disclosing what Oslo is since its very early stages, so as to get customer and partner feedback and create what organizations need.
Oslo is the code name for the forthcoming Microsoft modeling platform. Modeling is used across a wide range of domains; it allows more people to participate in application design and allows developers to write applications at a much higher level of abstraction. So, what is Oslo? It consists of three main pillars: A language (or set of languages) that helps people create and use textual, domain-specific languages (DSLs) and data models. I am talking about the M language and its sublanguages (M,MGraph, MGrammar, and so on). By using these high-level languages, we can create our own textual DSLs. A tool that helps people define and interact with models in a rich and visual manner. The tool is called Quadrant. By using Quadrant, we will be able to work with any kind of visual DSL. This visual tool, by the way, is based on the M language as its foundation technology. A relational repository that makes models available to both tools and platform components. This is called simply the Oslorepository.
I am sure that Microsoft will be able to close the loop in anything that is related to those key parts (M, Quadrant, and the repository) for modeling metadata. In my opinion, however, the key part in this architecture is the runtime-integration with Oslo, such as integration with the next Microsoft application-server capabilities (codename Dublin), ASP.NET (Web development), Windows Communication Foundation (WCF ), and Workflow Foundation ( WF). This is the key for success in model-driven SOA (and MDD in general). Will we get generated source code (C#/VB.NET) or generated .NET MSIL assemblies? Will it be deployed through Dublin? Will it be so easy that business experts will be able to implement/change services? Those are the key points. And this is really a must, if Oslo Figure 4: Architecture of Oslo and related runtimes wants to be successful in MDE/MDD. In this regard, that is the vision of Bob Muglia (Senior Vice President, Microsoft Server & Tools Business), who promises that Oslo will be deeply integrated in the .NET Quadrant platform (.NET runtimes):
In the context of Oslo, MDD indicates a development process that revolves around the building of applications primarily through metadata. In fact, this is the evolution that we had for all development languages and platforms. In every new version of development platforms, we had more and more developmental metadata and less hardcoded/compiled code (consider WF, WPF, XAML, and even HTML). However, with MDD and Oslo, we are going several steps ahead meaning that we are moving more of the definition of an application out of the world of code and into the world of data. As data, the application definition can be easily viewed and quickly edited in a variety of forms (even queried)making all of the design and implementation details much more accessible. This is what the Oslo modeling technology is all about. On the other hand, models (especially in Oslo) are relevant to the entire application life cycle. The term model-driven implies a level of intent and longevity for the data in questiona level of conscious design that bridges the gaps between design, development, deployment, and versioning. For instance, Douglas Purdy (Product Unit Manager for Oslo) says the following:
Oslo Architecture
[Your visual DSL] Editor framework Composition Generic viewers Data flow
The benefits of modeling have always been clear; but, traditionally, only large enterprises have been able to take advantage of it, and [only] on a limited scale. We are making great strides in extending these benefits to a broader audience by focusing on [certain] areas. First, we are deeply integrating modeling into our core .NET platform. Second, on top of the platform, we build a very rich set of perspectives that help specific persons in the life cycle get involved.
Language framework
Model-driven parser SQL generation Language services XML, custom formats, etc.
Runtimes
[Your runtime] Dublin ASP.NET WF WCF SQL/EDM
ADO.NET
In my opinion, it is good to target high goals. That is why we are trying to use MDD and MD-SOA to close the gap with business.
13
14
Pietro Nick Romano ([email protected]) is an IT Architecture and Planning Advisor with Microsoft Services, Spain.
field. It will be like changing from assembler and native code bits to high-level programming languageseven better, because, as Doug says, every person (business user) will be, in a certain way, a programmer, because business-expert users will be creating applications. Of course, this has to be a slow evolution toward the business, but Oslo could be the start.
Conclusion
SOA has to evolve toward a more agile design, development, and deployment process. Most of all, however, SOA must close the gap between IT and business. Model-driven SOA can be the solution to these problems. So, with regard to MDD implementations, Oslo is Microsofts best bet to reach future paradigms for both MDD and model-driven SOA (asubset of MDD).
Csar de la Torre Llorente is an Architect in the Microsoft Developer and Platform Evangelism organization. Before joining Microsoft two years ago, he cofounded Renacimiento Sistemas S.L. (an IT consultancy company and Microsoft Gold Partner) and spent his career architecting, designing, and developing solutions for big customers in the IT industry. Csar is happy to receive feedback at [email protected].
15
Summary This article discusses key concepts and methods that CIOs, CTOs, architects, and IT leaders can put into action pragmatically to help their organizations evolve their IT architecture and solutions, so as to meet ever-changing business needs and demands through service-oriented architecture (SOA). It presents a robust architecture foundationbuilt on proven and already successful methods and practicesto implement tangible service orientation in a consistent and repeatable fashion, with a focus on business priorities and alignment, to deliver predictable and measurable business value while maximizing value of IT investments.
Introduction
dancing technologies magically adapt to change and deliver out-ofthis-world value. In fact, SOA is an incremental and iterative journey that organizations need to experience through ways and means so as to achieve predictable ends. Industry has done a fairly good job in terms of defining the means: standards-based Web services, advancements in middleware products, and integration patterns. On the other hand, because of the bottom-up and technologycentric perspective that is predominantly driven by software vendors, organizations who want to adapt SOA are left in the dark with poorly defined or no ways methods and practices that will help them navigate though their SOA implementations. The following are key, foundational concepts that organizations can embrace as part of their ways to tackle the key challenges that they face and build a robust enterprise architecture (EA) strategy for successful SOA: 1. Abstracting the stable from the volatile 2. Encapsulating the volatile within services 3. Managing change through services the journey
Where to start and how to begin SOA How to align services to business needs and priorities How to identify and scope services How to enable agility and innovate through services How to implement SOA
These are the fundamental challenges that organizations who demand a realization of business value through SOAwhether it is in the form of innovation, growth, revenue, or operational costsare facing today. An inconsistent understanding of promises, disconnected approaches to implementation (even among departments within the same organization), and unrealistic expectations of products and technologies become impediments on the road from aspiration to success. This article introduces and discusses key concepts, principals, and methods that architects can practically put to work immediately to help their companies overcome these challenges and lead their organizations in their journey through SOA implementation for a better outcome. The concepts and the methods that this article presents are technology-agnostic and compatible with widely available methodologies; they are designed to complement existing capabilities for implementing service orientation on-premise, in the Cloud, or both. Samples are selected from a recent case study for a leading global-recruitment consultancy firm in which these concepts and methods were implemented successfully.
Concepts
SOA is not an end; it is not a production environment that organizations can buy or deploy, after which some singing-and16
Notify Acceptance
What Consultant
<people>
Send notification
<Process>
How
E-mail
<IT>
Develop products and services Generate demand Sales Generate new jobs ... Marketing ... Deliver products and services Recruitment Generate new candidates Manage applications Submit application Process application Parse CV [curriculum vitae, or rsum] Notify candidate Notify candidate of application acceptance Notify candidate of application rejection Assess candidates Manage shortlists ... Plan and manage enterprise Financial management Billing ... Manage collaboration
Notify Acceptance
Consultant
<people>
Notify Acceptance
Consultant
<people>
Send notification
<process>
Send notification
<process>
E-mail
<IT>
Implementation evolves
<IT>
SMS
Time
17
Or
The answer most likely is Match candidate and job. In fact, it delivers high value to business and is a competitive differentiator in the marketplace, when compared to Pay employees. This is how an organization might know where to start and align its SOA initiatives with business needs and priorities. An analytic assessment of capabilities through their attributes provides benefits immediately: It enables better, rational, and defensible decision-making to help prioritize the problem. It provides contextual information to architects for a better understanding of the problem at hand (what is important to business, what is not performing well, why not, and so on).
Because both SOA and EA are usually initiated by IT, the lack of both engagement by business stakeholders and support of top management can foil the success of SOA. SOA does require a large enterprise reengineering effort that has consequences at all EA layers: business, applications, infrastructure, and organization. The enterprise SOA critical-success factors (CSF) should be primarily approached as a business development, a business architecture, a way to structure the enterprise, and a style of target EAand only then as an IT-integration technology. Also, the enterprise SOA CSF might succeed only if it is developed inside an EA development, because SOA does not cover the enterprise transformation process. Driven only by IT, both SOA and EA are prone to fail; the engagement of the business stakeholders and the support of top management are keys to their success. The enterprise SOA CSF will demand business-process reengineering, new governance around services, and (ultimately) reorganization. Enterprise SOA was pronounced dead, because the preceding CSFs were not really met. Application-level or Web-domain SOA, however, are alive and well. When it has been implemented, an enterprise-wide SOA will become a competitive asset that is based on business services that are accessed independently of technology and geography, orchestrated agilely for change, and ready for outsourcing in the Cloud.
Adrian Grigoriu is author of the book An Enterprise Architecture Development Framework: The Business Case, Framework and Best Practices for Building Your Enterprise Architecture (Victoria, BC: Trafford, 2006). For more information, visit his blog.
Qualify
Model
With this example in mind, let us look at how these capabilities might be mapped to services. Qualifying Capabilities for Services Qualification is the process of mapping capabilities to services that are composed of service components that will provide the implementation of the capability. There are no predefined qualification criteria or contexts that fit all organizations that operate in all industries, or a magic formula that satisfies each and every business scenario. However, the characteristics of capabilitiessuch as their clear boundaries, purposes, and other attributescome in really handy when capabilities are being qualified for services. Figure 5 represents a possible mapping between the Manage application capabilities (illustrated as lollipops) and the services that encapsulate their implementation. Moreover, because of the associating behavior between capabilities and their implementation, services also form a logical entity to define the expectationssuch as performance, reliability, service-level expectations, quality-of-service (QoS) requirements, or key performance indicators (KPI)of the capability implementation. Thus, the performance of capabilities can be monitored and assessed against business objectives. Parse CV from our recruitment scenario is a good example, inpoint: Ninety-percent accuracy (in terms of extracting quality information from CV documents) One-thousand CVs per hour Such attributes and requirements provide architects with vital information and context in terms of selecting the right products and technologies, as well as developing successful architectures. In this case, for example, architects should be looking at technologies that can hit 90 percent accuracy, as well as perform (Parse 1,000 CVs per hour) and scale. Qualification is a rational, logical, and purposeful activity. When it has been completed, architects can start thinking about developing service-oriented models.
Manage Applications
Notify candidate Process application Parse CV
Notify candidate of application acceptance
Submit application
Submit application
Applicationsubmission service
Applicationprocessing service
Candidatenotification service
19
Send e-mail
E-mail service
Application-processing service A
Candidatenotification service
Application database
seamlessly apply, regardless of whether a service component is a singleton commercial off-the-shelf (COTS) application, a batch file that runs overnight, a set of Web services that run in the Cloud, or a legacy COBOL application.
Application-notification Web service: Notifies candidates of the status, whether acceptance or rejection of their applications E-mail service: Sends e-mail messages
Service-oriented architecture (SOA) has been adopted successfully across a variety of domains. However, a number of high-profile failures have caused some to say that SOA is dead. In reality, the proponents of this statement are saying that SOA as an architectural style makes a lot of sense but, unfortunately, it has been poorly adopted, and expectations have been poorly managed. The way in which the four pillars of SOA adoption are managed can make the difference between success and failure: 1. Create the right adoption strategy, and set realistic targets. Develop a strategy to meet critical business goals that are related to SOA adoption. For example, creating intuitive portals or services that are related to customer information can satisfy a goal to increase the information that is available to customers. Combine this strategy with a plan for initial pilots, followed by systematic expansion to larger parts of the enterprise. This approach reduces risk, enables gradual education and buy-in, and provides for midcourse corrections. 2. Create effective SOA governance. Develop and enforce organizational policies for identifying, developing, and deploying services. Maintain consistency and standard processes for development of services, SOA infrastructure, and applications that consume services. Develop runtime-governance policies and rules for deploying and managing service-oriented systems, monitoring the use of services, and enforcing rules and constraints.
3. Evaluate standards and technologies for SOA implementation in the appropriate context. An SOA implementation might use common technologies in novel contexts. Evaluate in advance whether a specific technology is appropriate for the task at hand. For example, T-Checks1 provide a hands-on, contextual evaluation for determining the fitness of a technology and associated tools for a specific need. Focused, extremely simple experiments can validate specific technology claims early in the life cycle and at lowcost. 4. Recognize that development in SOA environments requires a mindset that is different from traditional development. In service-oriented systems development: Service consumers, usage patterns, and requirements might potentially be unknown to service providers. Service providers, service consumers, and infrastructure developers might have distinctly different goals and needs; in fact, they might belong to different organizations. As a result, there is no central control over a single system. These differences have a profound impact on the way in which software is funded, managed, and developed. They affect all lifecycle activitiesfrom requirements through architecture and design, development, and system testing.
Dennis Smith is a Senior Member of the Technical Staff and Lead of the System of Systems Practice Initiative (SoSP) at Carnegie Mellon Universitys Software Engineering Institute(SEI). For more information, visit his Web page.
1
Lewis, Grace A., and Lutz Wrage. A Process for Context-Based Technology Evaluation. CMU/SEI-2005-TN-025, June 2005.
Often, it is not an option to ignore existing investments and perform a fresh start; architects must also ensure that existing assets are mapped to capabilities through services, so that they can be managed and reusedin this case, the E-mail-messaging service for sending e-mail messages. Let us build on our recruitment scenario and allow the change to happen. Consider that this organization is looking to transform its Manage applications capability by implementing the following changes that the business requires: Improvement in the accuracy of CV parsing from 90 percent to 95 percent by using a COTS specialist application Use of SMS messaging instead of e-mail messaging Extension of reach by allowing candidates to apply for jobs through a mobile application for smart phones Figure 9 illustrates such a change. Note the boundaries of services; although the implementation has changed to meet the latest business demands, the scope and boundaries that are inherited from the capabilities remained stable helping the organization minimize the impact of change and enable room for evolution and innovation. Mind the Gap! Service-oriented modeling plays a vital role in bridging the gap between the business and ITbetween business architecture (represented as business capabilities) and IT architecture (represented as service components)by connecting an abstract view of what is
Candidate
Application-submission service
Application-processing service
Candidatenotification service
Application database
Third-party CV parser
21
Enterprise Architecture
Business architecture Business model IT architecture Service model Technology model
Service-Oriented Modeling
Capabilities Services Service components
done or what must be done to how it is done or how it must be done (see Figure 10). Also, service-oriented modeling enables loose coupling between business models and technology models to embrace change and, ultimately, enable agility. Service-oriented modeling has three qualities in which architects will be very interested: It connects and loosely couples business models to technology models via services. It hides complexities and transformation for IT behind services. It accelerates the delivery of services with robust service modeling. By now, the immediate benefits and practicality of capability architectures and service-oriented modeling might have become clear. Now, let us put these into context and discuss SOA implementation from a life-cycle perspective, and see how these methods can be performed in interactively and incrementally.
Step #1: Prioritizing Capabilities and Identifying Services Service planning is the key phase in which architects must perform capability architecture and service-oriented modeling to prioritize capabilities and model services. The purpose of this phase is to: 1. Identify and prioritize capabilities that require attention, based on business context. 2. Qualify priority capabilities for services and, thus, for service orientation. 3. Model identified services and their key components. This is the phase in which fundamental challenges are addressed: where and how to start SOA, how to identify and scope services, and so on. The primary deliverables are approved projects to deliver services that represent the desired change. Step #2: Transforming Capabilities via Services Service delivery is concerned with transforming capabilities implementing desired change via services. Attributes and boundaries of capabilities, as well as service-oriented models, provide vital input to architects and into the delivery phase. Because service-oriented modeling connects implementation all the way up to business, it enables end-to-end visibility and traceability throughout the delivery phase, and provides teams with a context for the work that they are carrying out. Obviously, the ultimate purpose of service delivery is to deliver to production reliable services that implement the capability and represent the desired change. Step #3: Monitoring Capabilities via Service Operations Service operations focus on operating and monitoring services and their components that are deployed in production. They are concerned with the operational health and performance of services (composition of service components) that represent capabilities. Organizations must ensure that key service characteristicssuch as SLAs, QoS requirements, and KPIsare continually monitored. The key outcome of service operations is that the health and performance of services representing the success of the service, whatever its shape or formare key inputs and requirements for healthy service planning. Changes in economy and competition, pressures to cut operational costs, the desire to innovate, and an ever-changing
Service Planning
Capability performance 1. Prioritize capabilities and identify services Approved projects and services
Service Operations
3. Monitor capabilities through service operations
Released services
Service Delivery
2. Transform capabilities by services
22
CAPABILITY ARCHITECTURE
Service-Oriented Modeling
Conclusion
Solutions to key challenges in SOA are not hidden in products or technologies; in fact, they can be unlocked by applying a framework of methods and practices (collectively illustrated in Figure 12) to help organizations navigate through SOA implementation. Let us sum up by remembering the key concepts: 1. Abstract the stable from the volatile. Capability architectures help organizations build an abstract and stable view of the business and expose areas that require attention. They help architects abstract and decompose the problem into consumable elements that have clear boundaries. 2. Encapsulate the volatile within services. Service-oriented modeling helps organizations connect and align IT to business. It helps architects map capabilities to services and, then, to IT implementationand model these services, while hiding the complexities of technology. 3. Manage change through services. SOA is the journey, not the end. Organizations must go through incremental and iterative cycles to implement SOA with business needs and priorities in mind. Whether it is a business capability or an IT capabilityand whether solutions are implemented on-premise, in the Cloud, or both capability architecture and service-oriented modeling both greatly help an organization navigate through its SOA journey. They can dramatically increase the chances of an organization to succeed in delivering a required changefrom innovation to optimization to outsourcing to automation to the Cloudvia service orientation, so as to realize the desired value and benefits of SOA. The people, processes, and IT that make up implementation of a capability will change; they always do and always will. Loosely coupling what is done (capability) and how it is done (service components) via logical associations (services) provides a foundation for alignment, innovation, productivity, and agility. Loosely coupling business from IT first and, then, loosely coupling systems will enable innovation and productivity that business desperately demands and which cannot be achieved by loosely coupling systems only.
SERVICE-ORIENTED ARCHITECTURE
Service Planning Service Delivery Service Operations
IT SERVICE MANAGEMENT
Acknowledgements
Special thanks to my brother, Kutay Tuna (Microsoft Services), for tirelessly reviewing everything that I threw at him and helping to bring out the lousy author in me.
Hatay Tuna ([email protected]) is an award-winning Principal Architect in Microsoft in the UK. He specializes in Software + Services, Service Oriented Architectures, Business Architectures, Modeling, Model-Driven Development and Architectures, Software as a Service, Service Bus Architectures, Process Automation and Optimization, Event-Driven Architectures, and Software Factories. Hatay has a strong passion for innovation, cutting-edge technologies, and future industry trends. He has successfully delivered several solutions in the Government/Public Sector, the Financial Services space, and the Health and Retail industries. Also, he regularly presents at internal and external events, such as TechReady, SOA & Business Process, and Architecture Insight Conferences. Besides work, Hatay enjoys spending time with his family, playing on Xbox 360, watching movies, and reading strictly technical books.
Resources
Microsoft Corporation. Michael Page International MPI. Microsoft Case Studies, July 2009. Microsoft Corporation. Service-Oriented Modeling. Architect Insight Conference 2009, May 2009. Microsoft Services Business Architecture (MSBA) and Microsoft Services Service-Oriented Modeling (MS SOM) are solutions that are offered by Microsoft Services.
23
Summary This article describes some methods and technologies for enabling an SOA infrastructure to realize business capabilities and track dependencies between the two, so as to gain increased visibility across the IT landscape.
Introduction
Evolving from business-process reengineering (BPR), business-process management (BPM) was an established discipline well before serviceoriented architecture (SOA). Initially, enterprises viewed the two as distinctoften, establishing separate teams that leveraged disparate technologies. Since then, SOA is less about leveraging Web services for faster integration and more about providing an abstraction of information-technology (IT) resources to support business activities directly. This maturity in SOA thinking brings it closer in alignment with BPM. Enterprises now see this alignment and frequently combine new SOA and BPM projects; however, challenges still exist. While activities in a business process can be implemented as discrete services, there is usually no direct connection between BPM model artifacts and SOA
Consulting Services
TechnologyOptimization Services
Integration
model artifacts, which makes traceability difficult. Enterprise architects have a strong desire to see which processes would be affected if a given service were modified. At the same time, business owners want to see how their investments in SOA are faring. Ideally, they would like to gain visibility into which processes and services support a given business capability such as Order fulfillment. This article explains how Microsoft Services might provide enterprises with the visibility that they want through the application of service offerings (see Figure 1) that would leverage any existing Microsoft Services Business Architecture deliverables. Specifically, the article focuses on how Architecture and Planning services might feed directly into technology-optimization services. By weaving existing Integration (SOA) offeringspart of the Application Platform Optimization (APO) model under Business Technology Optimization (BTO) serviceswith emerging concepts that are used to drive future requirements, we hope to paint a picture of how you can enable business capabilities today on our stack and how this will only get easier over time. The Integration offerings are built around the concept of enterprise layers that describes relationships among processes, services, and data in the enterprise. The specific mesh of enterpriselayer items and their relationships in a given environment is referred to as an enterprise service model (ESM). To deliver this ability, we start by mapping items in our ESM to capabilities in a capability model, as described by Martin Sykes and Brad Clayton in their article titled Surviving Turbulent Times: Prioritizing IT Initiatives Using Business Architecture in the July 2009 issue of The Architecture Journal. When the mapping is complete, we have a dependency map that we can use to identify which IT resources support a given capability and to understand which capabilities are affected by IT resources. In this case, an IT resource might be a Web service; but it might also be a message queue (MQ) or a mainframe Customer Information Control System (CICS) transaction. We will continue to use the fictional retail bank, Contoso Bank, which was previously introduced to provide a context for discussing the concepts of enterprise layers and the ESM. As with all businesses, Contoso Bank must perform employee onboarding. The onboarding of an IT resource begins when a start is established after a candidate has accepted an offer and cleared the background check. Onboarding ends when the employee has a telephone number, e-mail address, laptop computer, and cubicle; has registered for HR benefits; and is set up in payroll. Several multistep processes are executed by different groups in the organization to fulfill employee onboarding. In an effort to reduce costs and increase productivity, Contoso Bank has a strong desire to reduce the time that it takes to onboard employees, so that they can start at their jobs more quickly.
24
Process Layer
Presentation Layer
Service Layer
Business Layer
Integration Layer
Enterprise Layers
Data Layer
The EA team at Contoso Bank structures its decisions and activities around an enterprise-layer model, as shown in Figure 2. The model that Contoso Bank adopted consists of the following: Process layer Service layer Integration layer The model that is shown in Figure 3 is an evolution of the distributed(or n-tier) application architecture paradigm of the following: Presentation layer Business layer Data layer The enterprise-layer concept extends these areas of concern across multiple applications. Whereas the presentation layer in a single application represents the actionable interface to invoke business logic, it is often asynchronous events in a process that invoke business logic in an enterprise ecosystem. The business layer encapsulates business logic to a specific application. Frequently, enterprise services must coordinate interaction with multiple business services to fulfill an activity step in a process. The integration layer is where traditional data jobs and enterprise application-integration (EAI) activities take place. The enterpriselayer concept provides enterprise architects with an opportunity to prescribe policies across a given layer and gives them a framework in which to think about dependencies across the entire IT landscape. After several years, the application portfolio of Contoso Bank is more consistent in approach and architecture, but it still struggles to align with the business. An additional relationship is missing: capabilities.
Business Capabilities
Capabilities do not represent an IT resource or a group of IT resources. They are purely business abstractions that can be accomplished or that are wanted. A given capability might depend on additional capabilities that are to be delivered. The Contoso Bank capability model includes the Onboarding capability. As shown in Figure 4, this capability depends on the following child capabilities: Hardware ProvisioningProcuring and deploying telephone and laptop computer HR RegistrationRegistering with HR benefits and payroll Security AccessCreating badge and network credentials
Jeff Niblack ([email protected]) is a Team Manager and Architect at Ratchet. He has more than 20 years of experience in dealing with application architecture and design.
25
HR
Security access
Onboarding
Hardware provisioning
HR registration
Benefit registration
Payroll setup
The business might choose to model these capabilities at a lower level of granularity and include further child capabilities. In our example, the HR Registration capability could be composed on a Benefit Registration capability and a Payroll Setup capability. These capabilities are pure business abstractions, so that there are no assumptions of how these capabilities are implementedor even assumptions of whether they are implemented at all. Modeling Enterprise Layers and Capabilities Artifacts in each enterprise layer can be modeled independently using existing tools in the disciplines of business-process modeling, capability models, and service modeling. Most enterprises would agree that there is benefit in developing their skills in these disciplines. Mature organizations that have established modeling methodologies face new challenges. Often, the business-process models, capability models, and service models are created by using different notations and tools. More importantly (at this point in the discussion), while the models are intended to represent the business and IT landscape, they drift apart quickly and lose value over time.
HR
Process Layer
Security access
Onboarding
Hardware provisioning
Service Layer
Integration Layer
HR registration
Benefit registration
Payroll setup
27
Conclusion
Services and applications that were developed over a period of months or even years at Contoso Bank and a variety of different technologies have been utilized. The developers at Contoso Bank followed common architectural patterns for distributed- and n-tier application development. However, as their business evolves, they face the continued struggle of aligning applications with their business. By focusing on a business capability such as Onboarding, Contoso Bank is better able to align with the applications and business processes. Instead of having to write new applications or application adapters to map to the business capability, Contoso Bank found that by leveraging an MCS solution (the managed-services engine, or MSE), their developers could create new virtual services that align with the business capability. This is possible by modeling the existing services, endpoints, operations, and data entities with the use of the ESM. This process is quickly performed by importing existing services or applications into the ESM; then, new virtual services that align with the business capabilities are defined. Service virtualization enables the composition of multiple physical resources and operations into a virtual service that aligns with the business capability.
Coming Enhancements
The MSE solution was developed over a four-year period by a team of architects in Microsoft Services who were working with customers and their scenarios. During this period, the MSE solution underwent several releases that added features; updated the ESM; and supported new operating systems, as well as new versions of Microsoft .NET Framework and Microsoft SQL Server. New features were developed in consultation with customers and as a result of frequent meetings with various Microsoft product groups. These meetings helped align the solution with product releases and validate the solution. The result was the filling of a joint patent application that was related to service virtualization and acceptance of the MSE solution into the codename Dublin Microsoft Technology Adoption Program (TAP). Dublin is an extension of Windows Server that will provide enhanced hosting and management for Windows Communication Foundation (WCF) and Windows Workflow (WF) applications. In the coming year, MCS plans to extend the ESM to include capabilities as first-class citizens. The ESM will be extended so that business capabilities can be associated with operations. When this is complete, we will be able to get end-to-end visibilityfrom the business capability all the way to the IT resources that deliver that capability. In our example, we find that the child capabilities are mapped to operations and that those operations are projected as virtual services. Someone who was examining the model would see the need to develop a workflow service to coordinate interaction between the three operations. After the development of that service, it too could be managed by the MSE and mapped to the Onboarding capability for dependency tracking. As Oslo gets closer to release, the MSE will take it as a dependency and leverage the repository to store the metadata that represents the ESM. By utilizing Oslo, we will map the virtualized service and operations to the Contoso Bank Onboarding business capabilities. A business analyst or domain expert can start with
References
An Introduction to Service Virtualization on the Microsoft .NET Platform. Microsoft Download Center. July 14, 2009. Download. Microsoft Corporation, Redmond, WA. Service-Oriented Infrastructure from MCS. Microsoft Services. Data sheet. October 23, 2008. Microsoft Corporation, Redmond, WA.
Chris Madrid ([email protected]) is a Solution Architect in the Microsoft Consulting Services Global Practice. Focused on integration, he spends time with customers to grow their SOA maturity through service virtualization and service life-cycle management. Blair Shaw ([email protected]) is a Senior Program Manager in the Microsoft Consulting Services Application Platform Optimization Service Line. He has spent the past four years working on SOA solutions with Microsoft customers and partners.
28
Summary One of the promises of adopting a service-oriented approach in organizations is the potential cost savings that result from the reuse of existing services. A service registry is one of the fundamental pieces of serviceoriented architecture (SOA) for achieving reuse. It refers to a place in which service providers can impart information about their offered services and potential clients can search for services. In this article, we provide advice for implementing an enterprise-wide service registry. We also discuss open issues in industry and academia that affect the management of servicerepository information.
Introduction
and lookup entries) and publication (publish, delete, and update registryrelated information) are core APIs. Figure 1 illustrates some relationships between a WSDL service description and information that is stored in a UDDI service registry. Originally, UDDI was conceived to cover both publicly exposed services and services that were available within an organization. Currently, most existing implementations are internal to organizations. Service publication, discovery, and (finally) reuse of services is more complicated in an inter-organizational scenario; for example, additional legal and commercial agreements are often needed among parties. Dedicated (public) UDDI service registries were criticized for their limitations (among other reasons) during service inquiry/ discovery. Recently, however, Web search engineswhich could be crawling publicly available WSDL documentshave raised promising expectations for discovering publicly available services.5
The reuse of services greatly depends on the ability to describe Designing an Enterprise Service Repository This section proposes some design guidelines to develop an and publish the offered functionality of the services to potential enhanced enterprise service repository. The focus is on improving consumers (clients). A service registry allows you to organize the reuse of services over time in different IT projects. The aim is to information about services and provide facilities to publish and increase service visibility to domain experts (often, this refers to a discover services.1 Universal Description Discovery and Integration (UDDI) and the business-analyst role) and enhance service descriptions with practical Web Services Description Language (WSDL)together with SOAP information for architects. Business analysts, who have a less technical are standards for describing services and their providers, as well as background but strong knowledge of the business domain, are how services can be consumed: frequently the early designers of new initiatives for incorporating or WSDL2 provides a model and XML format for describing what a Web service offers. Figure 1: Relationships between WSDL and UDDI A service description in WSDL separates abstract-service functionality from details such as how and where the service UDDI 3.0 WSDL 2.0 is offered. While the abstract-service Description description includes types and an abstract Type: schema, targetNamespace, elementFormDefault businessEntity interface, concrete details include bindings, Element: name, type a service element that includes all available type Interface: name bindingTemplate implementations of the abstract interface interface Abstract tModel description description name accessPoint at endpoints. Operation: name description hostingRedirector overviewDoc tModelInstanceDetails UDDI3, 4 provides an infrastructure that Pattern categoryBag identifierBag supports the description, publication, and signature categoryBag signature Message: input, output, etc. discovery of service providers; the services type binding that they offer; and the technical details Binding: name, interface, type, soapBinding, protocol businessService for accessing those services. A core aspect Operation: ref, soapBind:mep, soapBind:action name Concrete description of UDDI is how it organizes information description Service: name, interface bindingTemplate categoryBag about services and the providers of signature Endpoint: name, binding, address services. Information entities (UDDI data) are organized in a data model and stored in a UDDI service registry. Inquiring (search The Architecture Journal 21
29
Client application
Business service
Runtimepolicy enforcement Calls to business functionality
Business Layer
(Business service)
Business Layer
Service intermediary
Orchestration
which can be consumed by a BS to operate on different AS responses and consolidate a single answer that is sent to a BS. In turn, the BS delivers the consolidated response to the client application. The aggregator pattern7 is core to the design of a BSE.
Service implementation
(Business-service extension)
Business-service extension
Businessfunctionality orchestration Calls to business functionality
Application Layer
Service implementation
(Application service)
Provider applications
Provider applications
Figure 3: Main interaction among elements of the service-based solution from Example 1
Business service
Repository
Policy
Business-service extend
lifeInsurance service
Client
TotalSales
BSdefinition() PolicyEnforcement()
Business-Service Layer
modifying the software support at companies; they play a key role with regard to the reuse of services. Enterprise Services Enterprise servicebased solutions involve different types of service. Following the separation of concerns that is addressed by the service-virtualization pattern,6 services can act as an intermediate layer between the client and provider applications of the services. The virtualization pattern focuses on the abstraction of technical detailssuch as service-endpoint location, policy enforcement, service versioning, and dynamic service-management information from service consumerswhich access an intermediate service level. Technical concerns are managed at an implementation level, at which the actual business logic is implemented. Based on SOA initiatives in several companies, we can identify three types of service: A business service (BS), which client applications use for accessing the functionality that is implemented in provider applications. An application service (AS), which can be consumed by a BS to access the functionality of the provider applications.
30
Example 1 Let us consider a simplified BS that is used for calculating the total sales that are related to the life-insurance and group-insurance products of an insurance company. The total sales that are associated with lifeinsurance products are obtained from a life-insurance legacy application. Analogously, the total sales that are associated with group-insurance products are obtained from the groupinsurance legacy application. Each legacy Application-Service Layer application exposes an AS (lifeInsurance and groupInsurance, respectively) that provides the total sales for each type of insurance product. A BS receives requests from clients who are asking for the total sales; afterwards, it calls the service registry that has the information that is required to enforce specific policies on messages and dependencies to an AS and a BSE. A BSE operates on the answers of an AS and provides a single answer to the BS that contains the total sales of the company. The BS, in turn, delivers this response to the client application. Figure 3 illustrates the described interaction.
groupInsurance service
Enterprise Service Registry The enterprise service registry (ESR) is a core element that organizes service information and supports the interaction among enterprise applications that provide and consume services. Basic functionalities of an UDDI-based ESR can be enhanced by using: Service-dependencies management. Runtime-policy enforcement. Service versioning. Service-history data (logs) management.
Application Layer
Application service
Figure 2 illustrates the main static relations among elements of an enterprise servicebased solution, as well as their relationship to elements from the virtualization pattern. A service registry organizes the description of the three different types of service and their relationships. Client and provider applications interchange messages that are mediated by BS, BSE, and AS. The service registry manages (at the configuration level) the information that relates the different types of service. The information is persisted in a service repository and used at runtime by a BS to answer client requests.
different roles. 8 Business analysts and solution architects manage information about business and technical concerns, respectively. Both roles work at a project or business-unit scale. Enterprise and infrastructure architects manage service information from a global (enterprise-wide) perspective. While enterprise architects might be interested in managing (for instance) service versions, infrastructure architects care about providing the required infrastructure support to keep services running with the adequate quality of service (QoS), as defined in service-level agreements(SLAs).
Using the Enterprise Service Registry to Improve Reuse of Services Based on our experience in a range of projects, providing simple descriptions about a BS, facilitating its access, and managing services dependencies have been key to improve reuse. For this purpose, an ESR was a core element. Business analysts who trigger new requirements for software support can improve their communication with solution and enterprise architects by referring to a BS that is described in the ESR. Based on the descriptions of the BS and its dependencies to an AS and a BSE, domain experts become aware of available functionality at back-end applications. From our experience, this has facilitated a shift from requirements that are specified in a vague manner to initial solution blueprints that comprise orchestrated services (created by business analysts). Long meetings between architects and business analysts can be reduced to short meetings or even telephone calls that refer only to information at the service registry.
Solution architect
Service-registry functionalities
- Inquiry - Publish - Subscribe - <standard UDDI functionality>
Enterprise Service Registry
Enterprise architect
1 *
Service
- Dependency management - Runtime-policy enforcement - Versioning management - Historical -data (logs) management - <extensible>
Infrastructure architect
Service Description 1 *
SQL Srv.
Services Intermediary
Services Implementation
Service Description
Information about: Additional in service registry: C F
Virtualization-pattern roles
A
Application Service
B E
Business Service
31
Role
Infrastructure architect Solution architect Enterprise architect
X X X X X
X
Registry information Replicated information
X
Business analyst Infrastructure architect Solution architect
X
Enterprise architect
X X X X
X X X X
X X
X X X
X X X X
X X X
Registry information Replicated information Business analyst
X X X
Infrastructure architect
Solution architect
Enterprise architect
X X
X X X X
X X
X X X X
X X X X
X X X X
in the initial blueprints that are made by business analysts. Subsequently, they can agree with enterprise and infrastructure architects on service versions and infrastructure support. Again, information at the ESR was central during the agreement. Information about service dependencies helped infrastructure architects to analyze the impact of binding new consumers to application services. This is critical for maintaining SLAs. Runtime policyenforcement configurations at the service registry allowed specialized treatment for different client-application requests that were associated with a single BSfor example, applying particular validations with regard to formatting, security, and parameterization. In the case of new requirements triggering modifications to existing services:
Often, extensions or modifications involved changes only at the If an AS or BSE was modified and new versions were deployed,
BSE level. the version of the associated BS remained unchanged. (Service versioning is discussed in more detail in the Impact of Service Versioning on Service Registries section.) Only incompatible changes that modify the business functionality could trigger new BS versions (in general, a BS is designed with forward compatibility in mind). Among the lessons that were learned from different projects, we can emphasize the following: Decoupling of a BS from an actual implementation (by using AS(es) and a BSE) is an effective way of keeping domain experts separated from technical information, which facilitates service discovery at the business level. The Architecture Journal 21
32
a single solution or projectallowing their use across projects and at an enterprise-wide scale. In practice, when only an AS is presented to domain experts, it remains almost untouched; that is, it is rarely reused in further developments. Even when new requirements involved modifications to existing service solutions, the reuse of a BS has still been strong. This was facilitated by addressing the required modifications at the BSE level. During legacy-application migration, client applications kept consuming the same BS. Changes mostly occurred at the AS level. At the ESR, AS descriptions and service dependencies were updated. New projects could reuse a BS independently when a migration had occurred.
implementation itself to parts of the service description in a service repository. Changes might aim to improve reuse: Implemented services that follow a bottom-up approach13 often fulfill particular project requirements within a domain. When any of these services becomes a candidate for reuse in a different context, it usually requires modifications or extensions. Analogously, services that follow a top-down approach often must be changed (specialized) to fit in particular contexts. Different versioning strategies address different requirements. A single solution is not likely to be satisfactory for all situations. WSDL and UDDI do not define guidelines for versioning services. Some authors have proposed strategies for service versioning; most of them relate to backward and forward compatibility:14 A backward-compatible version refers to the ability to support consumers of older versions of a service. A forward-compatible version refers to the ability to adapt to unknown future requests that are intended for newer versions of the service. This type of compatibility involves not only a serviceversioning strategy, but also a service-design strategy that is related to changeability. Often, new service versions are replications of a previous version that have additional or modified elements. New versions are named differently (by using some naming convention), and their description is stored in the registry as a new entry. Juric et al.15 propose extensions to WSDL and UDDI for service versioning. The approach addresses run-time and development-time versioning. Efficiency at the code level is addressed by allowing multiple versions of a service to refer to the same codebase. Additionally, notifications about new and deprecated versions are communicated to consumers. Traceability support is provided to track changes. This academic research promotes the reuse of services and keeps the complexity of a service registry manageable. Service-Usage Information for Enhancing Service Description andDiscovery The history of service usage can be an interesting source of informationnot only to re-create the actual behavior,16 but also for service discovery. Stored service usagehistory (logs) can help to categorize services according to the user or how services have behaved over time. Let us consider a service description that indicates a specific performance level in its contract; however, the actual measured performance in a given timeframe (extracted from logs) is lower. This information could be used during service discovery; a service that had lower-than-expected performance levels would be discarded from the search. Statistically extracted information about how services behave against historic interactions can help to build less biased rankings and make service discovery more precise; however, an infrastructure for the constant monitoring of services and storing of the history information must be provided. Based on the service history, probabilities can be assigned to quantify uncertainty. Clark et al.17 consider uncertainty with regard to the configuration of a servicebased system, the rate parameters of system components, and the duration of events. An uncertainty model is used to predict system performance under increased demand. This type of analysis is fundamental when one is dimensioning the service support infrastructure. Historical data about individual services helps to predict the performance of an entire system. Offer and demand in an inter-organizational scenario are subject to how much parties trust one another. Trust in others is one of several criteria for assigning reputationwitness reputation18
33
This section discusses a number of observations in industry and academia with regard to enhanced service descriptions, organization of service information in a service registry, and the role of such a registry to enhance the reuse of services. Strategies for Organizing and Finding Services in Registries If service information in an enterprise service registry is difficult to distinguish because of inadequate organization or ineffective search mechanisms, the value of that registry is reduced. Services categorization can help to distinguish services and classify them according to one or more categories. UDDI registries support this through the tModel. The categorization schemas of UDDI refer to taxonomic classifications. Taxonomies organize concepts in a hierarchical structure; multiple taxonomies can apply to a single UDDI entity. Standard classification schemas are suggested, such as the United Nations Standard Products and Services Code (UNSPSC9); however, other standards or internally created taxonomies can also be used. The UDDI Inquiry API supports different forms of query, such as browse pattern, drill-down pattern, and invocation pattern. Queries can refer directly to services, as well as to service categories. Similarly to a Web search engine, the browse pattern allows one to find registry elements by matching keywords. Although this mechanism automates part of a service search, the results are limited to the coding systems value set and direct value matching. Services whose description includes similar or related concepts, but different syntax, cannot be retrieved by using this approach. Also, during use of different categorization schemas, the management of overlapping categories can become expensive.10 Taxonomy maintenance is an added load that must be considered during the implementation of a service registry. Classification schemas that are not updated can affect the quality of the discovery results.11 The semantic research community has proposed alternatives to enrich service descriptions semantically and enhance classification schemas in services registries. Basic taxonomies can be enriched or replaced by ontologies. Ontologies structure concepts within a domain and define their meaning. Axioms constrain possible interpretations of concepts and reasoning mechanisms that support inferences from existing data. According to Kster et al.,12 although semantic enrichment of services descriptions can improve service discovery, several issues still must be addressed, such as reducing the computational cost of reasoning, maintaining the ontologies, and refining search results to improve effectiveness. Impact of Service Versioning on Service Registries When a service has been implemented, changes can occurfrom the The Architecture Journal 21
pr?
pr!
pr?
pr!
po!
po?
po!
po?
description is called an operating guideline22and allows the hiding of implementation details, while exposing enough information to find compatible partners. Checking if a service can be composed with others is reduced to checking if a graph-based representation of the potential partner is a subgraph of the operating guideline, which is less expensive than exploring all possible states of the services.
Receive product
p!
p?
Send product
Send payment
pa!
Send payment
pa?
pa!
Receive payment
Receive product
p!
to publicly available services. If company X knows that a service is being used or was positively rated for company Y, whom X trusts, the reputation of that service would increase from the point of view of X. One associated problem is the eventual bias for positive ratings, unfair ratings, and the variations of quality between ratings.19 Sufficiency of WSDL Descriptions to Find Services forCompositionEfficiently Services are reused not only by client applications, but also by other services in a service composition. A service composition can provide a more coarse-grained functionality and be closer to a business need. One problem when finding a service (useful in a service composition) is the need to verify if the services that are involved are able to talk to one anotherthat is, if the associated messageinterchange protocol among them is compatible. A basic requirement for compatibility is deadlock-freeness. Moreover, the message syntax and semantics should be compatible. Figure 5 illustrates a typical example of incompatibility at the protocol level between two parties. In the figure, a Buyer party offers a buyProduct service, and a Seller party offers a sellProduct service. To automate a hypothetical sale process, the message-interchange protocol between buyProduct and sellProduct should be compatible. However, Figure 5 illustrates that the Seller expects a payment before sending the product, and the Buyer expects the product before sending the payment. When more and more services are offered and advertised in repositories, there are more chances of satisfying a service demand by composing existing services. However, mediation at the protocol level might be required. Matchmaking conflicts at the message and/or conversation level(s) can be solvedto a certain degree by a mediator component.20, 21 However, verifying and solving compatibility among services at the behavioral level is expensive; it involves the (expensive) exploration of possible states of the services during interaction. To increase reuse here, we need efficient mechanisms for finding compatible services. For instance, instead of directly publishing the behavior of a service in a repository, a provider can publish a summarized description of the expected behavior of all compatible services to service (compatibility refers to deadlock-freeness). The summarized
34
To improve the reuse of services at the enterprise level, architects must define a strategy for publishing and providing facilities to access services information. For this purpose, an enterprise service p? registry is a key piece. Information about Send product services can be organized at the registry, and basic functionality can be enhanced including, for instance, functionality for service versioning, management of service dependencies, and enforcement of runtime policy. In this article, we have provided some design guidelines for enhancing an enterprise service registry to improve the reuse of enterprise services. We have also discussed some open issues in industry and academia with regard to the design and implementation of service registries and associated aspects that are required to describe and organize services information.
pa? Receive payment
Conclusion
Resources
1
Pijanowski, Keith. Visibility and Control in a Service-Oriented Architecture. MSDN, May 2007. 2 W3C.org. Web Services Description Language (WSDL) Version 2.0 Part 1: Core Language. 3 UDDI.org. UDDI Version 3.0.2. UDDI Spec Technical Committee Draft, October 19, 2004. 4 Microsoft Corporation. UDDI Specification Index Page. MSDN, July 2009. 5 Al-Masri, Eyhab, and Qusay H. Mahmoud. Investigating Web Services on the World Wide Web. WWW 2008: Web Engineering Web Service Deployment Track, April 2125, 2008, 795804. 6 Madrid, Chris. SOA Realization Through Service Virtualization. SOA Magazine, September 3, 2007. (Available also here.) 7 Hohpe, Gregor, and Bobby Woolf. Enterprise Integration Patterns. Boston: Addison-Wesley, 2004. 8 Cardinal, Mario. The Hidden Roles of Software Architects: The Three Types of Architect. MSDN, March 2008. 9 UNSPSC.org. UNSPSC Codeset. (Managed by the Electronic Commerce Code Management Association (ECCMA).) August 2007. 10 Klein, Michel. Combining and Relating Ontologies: An Analysis of Problems and Solutions. Proceedings of Workshop on Ontologies and Information Sharing at 17th International Joint Conference on Artificial Intelligence (IJCAI-01), 2001, 5362. 11 Hepp, Martin, Joerg Leukel, and Volker Schmitz. A Quantitative Analysis of eClass, UNSPSC, eOTD, and RNTD: Content, Coverage, and Maintenance. Proceedings of IEEE International Conference on e-Business Engineering (ICEBE05), 2005, 572581. 12 Kster, Ulrich, Holger Lausen, and Birgitta Knig-Ries. Evaluation of Semantic Service Discovery: A Survey and Directions for Future Research. Emerging Web Services Technology. Volume II, 2008, 4158. The Architecture Journal 21
for Protocol Mediation. Proceedings of the Seventh Working IEEE/IFIP Conference on Software Architecture (WICSA 2008), February 2008, 137146. 22 Kaschner, Kathrin, Peter Massuthe, and Karsten Wolf. Symbolic Representation of Operating Guidelines for Services. Petri Net Newsletter. Vol. 72, April 2007, 2128.
Juan Pablo Garca-Gonzlez is a senior software developer and chief architect at DATCO Chile S.A. He has been involved in numerous software projects involving service-based solutions. In addition to their regular projects, he and his team are working on the creation of innovative service-centric mobile solutions for the banking industry. Veronica Gacitua-Decar is a postgraduate researcher at the School of Computing, Dublin City University, and Lero - the Irish Software Engineering Research Centre. Her research is focused on the development of tools and methods for designing process-centric service architectures. Previously, she worked as an IT architect in a large mining company. Dr. Claus Pahl is a senior lecturer at the School of Computing, Dublin City University, where he is leader of the Software and Systems Engineering group. He is also involved in Lero - the Irish Software Engineering Research Centre, as well as the Centre for Next Generation Location (CNGL).
the result of several months (or years) of work during which all other projects are put on hold. The business people must understand what is going on, all of the way. If the business does not understand the EA/SOA model, it is because you are moving too fast or making things too complex. So, slow down! A Successful Approach We have used this approach with several customers who have had little or no prior experience in developing enterprise architectures. One of these customers is Toyota Denmark, where the approach has provided the following results (among others): Involvement of the businesspeople already has ensured a high level of reuse of messages from the second project, after the implementation of the initial enterprise-architecture model. Having seen the immediate reuse of messages, the businesspeople display further engagement and enthusiasm. A new project model is developed with consistent mapping, which gives shorter and more predictable implementation of projects and better control of IT projects (as well as the vendors). Please stop by our Web site for all of the details.
Tom Hansen works with technical and organizational aspects of SOA and EA. He represents Globeteam in the SOA Partner Expert Team, which is hosted by Microsoft Denmark. Michael Andersen is the manager of the Architecture and Development team at Globeteam. He has delivered presentations at conferences that have been hosted by Microsoft and Computerworld.
35
Summary In this article, we will describe how the enterprise service-oriented-architecture (SOA) scope is stretched with the advent of cloud computing. With the help of the case study of a fictitious global retailer, we will demonstrate the process for identifying cloud scenarios. Also, we will come across an emerging breed of distributed applicationsboth on-premise and in the Cloudand discuss the integration considerations for building them.
Introduction
Over the past few years, enterprises have adopted strategies for both build and build vs. buy for their enterprise-application development. In many business domains, such as retail and manufacturing, packaged application has found favor because of the availability of best-practice business functionality for faster time to market. This has led to a mesh of packaged applications, legacy systems, and bespoke and homegrown line-of-business (LOB) applications that are connected by point-to-point integration and mature enterpriseapplication integration (EAI) and SOA-based enterprise service bus (ESB) implementations, in some cases. Enterprises are now extending the existing enterprise SOA investments into the Cloud to address new business pressures and opportunities. In this article, we use the example of a fictitious global retailer (Globtron) to demonstrate how an enterprise should explore the newbusiness opportunities and map application requirements to theCloud. Also, with the help of this case study, we describe the integration requirements and solution options for on-premise and cloud integration in the context of Microsoft Windows Azure Platform .NETServices.
organizationnot as a big-bang SOA-definition project. Enterprisearchitecture (EA) efforts in the organizations helped identify the reusable services and leverage existing investments. Because SOA facilitated interoperability, many legacy applications were exposed asservices and the business functionality was reused. SOA became the EAI driver across the company, and EAs have evolved into on-premise distributed systems that are enabled by SOA.
SOA in an Enterprise
As an architectural principle for delivering agility and flexibility by decoupling interface and implementation (business logic), SOA has been in existence for decades. Object-oriented and component-based development models were established by using the same principle. Recent technological developments and Web-service standards have made SOA easier by replacing method calls on object references to message passing. In large enterprises, services have been delivered in isolation by several projects that are based on LOB requirements in the
36
Model Business Drivers How do cloud opportunities for enterprise IT derive from the business pressures and enterprise requirements? Let us take the example of Globtron, a fictitious retailer of electronics goods. Globtron is a U.S.-based retailer that has a strong brand presence. Recently, it has increased its global footprint with rollouts in a few European countries through acquisitions. Over several decades of running the business, Globtron management knows that its core business value that drives
Centralized applications
37
Figure 2: Case of a global retail enterpriseapplication scenarios for each business driver
system internationally Globtron has established a successful online e-commerce and is able to drive huge traffic and sales in the United States. Globtron wants to utilize the existing e-commerce application for rollouts across the world, but without the hassles and costs to set up multiple instances for similar functionality across different countries. Readiness to address peak workload for e-commerce Globtron has asked its IT department to assess the readiness of its infrastructure to address peak workload demand for the upcoming holiday season. New store portal for store staff Globtron also wants a store portal for the store staff, so that the retailer can improve communication and collaboration with the store staff and share information, such as operation manuals and best practices. Centralization of store systems One of the pain points for retail IT is the overhead and cost that are involved in distributing software upgrades and patches across stores. Globtron wants to re-architect and deploy a central store system that is accessible from the store devices via the Internet and the smart-client model.
Rapid Global Expansion Global usage of service-based loyalty application Globtron has built a loyalty application in the United States that exposes Web Services for consumption from multiple channels, including store and e-commerce systems. The services allow store and e-commerce systems to validate the loyalty card number of a customer, compute loyalty points, and set up the customer in the loyalty system. The Globtron business team believes that it should leverage this system globally for all other countries.
Branding and Superior Customer Service New social-media applications for customer feedback and collaboration Globtron wants to engage better with its customers, obtain feedback on its product and services, and drive sales and customer loyalty through collaborative applications such as reviews and ratings, videos sharing, and discussion forums. New CRM system for repairs service Globtron wants to improve the customer service for its electronics-repairs business that is available in several stores through automation. Launch of online marketing campaign Although Globtron does not have a significance presence in online selling outside the United States, it wants to launch an online marketing campaign to provide offers for its customers, build loyalty, and drive more sales. Driving Growth via New Business Services and Channels New venture into online home-appliance retailing As part of its strategy to venture into new services, Globtron decided to retail large home appliances in Germany. However, because its store format is small and medium, Globtron does not have much real estate for this business. The business team decided to experiment with online selling for this segment. However, the success of this business will require some initial pilots and feedback before it can go full steam into it. Mapping Application Scenarios to Cloud-Application Characteristics We have mapped each of the previous application scenarios to the cloud-application characteristics that we discussed earlier. Figure 3 illustrates the mapping.
Non-core business services High computing workloads over a short time span New business services and pilots Web 2.0 collaborative applications Centralized applications
New portal for store staff New CRM system for repairs service
Readiness to address peak workload for e-commerce Launch of online marketing campaign
New social-media applications for customer feedback and collaboration Global usage of service-based loyalty application Extension of homegrown e-commerce system internationally Centralization of store systems
38
New portal for store staff New CRM system for repairs service
New venture into online home-appliance retailing New social-media applications for customer feedback and collaboration Centralization of store systems Launch of online marketing campaign Global usage of service-based loyalty application (PaaS integration service) Readiness to address peak workload for e-commerce Extension of homegrown e-commerce system internationally
Considerations for Cloud Integration Existing SOA investments There is commonality in the business objectives of SOA and cloud computing with regard to delivering agility, flexibility, and reuse. The existing core business services that have been developed within the enterprises will coexist and integrate with the services in the Cloud. This is again enabled by SOA. The integration solution should address the challenge in letting outside organizations find endpoints of the on-premise services. Enterprise-architecture practices SOA is part of the EA mainstream. The scope of existing organization practices for EA, such as governance, are preserved and extended to the Cloud (the Cloud is also part of the enterprise technology portfolio). Access across firewalls To expose existing on-premise services to the Cloud, enterprise IT has to deal with issues such as handling network-address translation and enabling access through a firewall by opening new ports. Security-access control In the distributed application, we need an access-control solution that works across enterprises that holding their own identity information. The solution should address key quality-of-service (QoS) requirements, such as system availability and reliability of integration points. Extension of On-Premise Applications to the Cloud by Using Windows Azure Platform .NET Services The Windows Azure Platform is an Internet-scale, cloud-computing services platform that is hosted in Microsoft data centers. The Windows Azure Platform uses SOA heavily and provides SOA-based access to data. .NET Services is one of the components of the Windows Azure Platform; it provides cloud-based infrastructure services and facilitates creation and deployment of composite applications. .NET Services comprises an access-control service and a service bus. Access control Cloud implementation of claims-based identity federation for performing federated access-control authorization as a Web service across organizations and protocols. This service utilizes standard protocols, such as WS-Trust and WS-Federation. Service bus Allows applications that expose Web-service endpoints to be located and accessed by clients. This service enables communication between two applicationsboth of which might run in the Cloud; one of which might run in the Cloud and the other on-premise; or both of which might run on-premise,
39
Table 1: Cloud integration for the New venture into online home-appliance retailing scenario Integration requirement(s) Expose existing on-premise loyalty, order management, and fulfillment services Periodic import of catalogs from on-premise merchandising systems Customer data to be replicated between the new online database and on-premise customer repositories Inventory updates from the existing fulfillment and warehousing systems of Globtron Applicable integration type Functional integration Data integration Integration pattern Service interface Windows Azure solution .NET Services
Queue in Windows Azure computing resources Replication capability of cloud data platforms, such as SQL Azure Queue in Windows Azure computing resources
Data integration
Data integration
Messageoriented middleware
Table 2: Cloud integration for the Global usage of service-based loyalty application scenario Integration requirement Existing on-premise loyalty service Applicable integration type Functional integration Integration pattern Service interface Windows Azure solution .NET Services
Table 3: Cloud integration for the New social-media applications for customer feedback andcollaboration scenario Integration requirement Aggregate user-generated content such as blogs, reviews and ratings, and videos with the product data in the marketing site to generate interesting product reviews and information Applicable integration type Data integration Integration pattern Mash-up Windows Azure solution Windows Azure computing and storage resources
Conclusion
In this article, we discussed how organizations have evolved EAs into on-premise distributed systems that are enabled by SOA and can identify cloud opportunities, so as to address their new business requirements. We discussed the application characteristics for the Cloud. We also discussed how an enterprise can map its business drivers and application scenarios to cloud characteristics and identify cloud solutions that coexist and integrate with the existing on-premise applications. Finally, we discussed integration scenarios, requirements, and solution considerations.
but across different enterprise data centers. It also addresses the firewall-access issues of enterprise systems and enables connection, without requiring data centers to open new ports for accessthus, providing location transparency to integrating applications. Refer to the Resources section for links to the Windows Azure resources that are available on the Internet. Integration Scenarios for Globtron Let us consider the integration requirements of some of the
Resources
Armbrust, Michael, et al. Above the Clouds: A Berkeley View of Cloud Computing. Technical Report No. UCB/EECS-2009-28, Electrical Engineering and Computer Services, University of California at Berkeley, February 10, 2008. Bertocci, Vittorio. Claims and Identity: On-Premise and Cloud Solutions. The Architecture Journal, MSDN, July 2008. Carr, Nicholas G. The Big Switch: Rewiring the World, from Edison to Google. New York: W. W. Norton & Co., 2008.
40
Lakshmanan G ([email protected]) is an educator and mentor in IT architecture with the Education and Research group of Infosys Technologies Ltd. With more than 20 years of experience in the IT industry, he nurtures a vibrant community of architects at Infosys. Prior to this, he was Head of Engineering, SaaS Practice, and Head of Infosys EMEA Architecture Practice. Manish Pande ([email protected]) is a lead architect with the Product Incubation and Engineering (PIE) unit of Infosys Technologies Ltd. Currently, he is responsible for architecting Infosys SaaSbased product offerings. With 11 years of consulting experience, he has played a variety of roles, including Solution Architect, Engineering Manager, and Technical and Performance Architect.
It is likely that different departments within your organization will employ different cloud strategies. For some, adopting a cloud platform might be better, while the right choice for others might be to invest in building up a departmental computer lab, as well as hiring and managing the personnel to run and manage it. The decision will be based on the focus of the department and its available ITresources. Another factor is the duration for which the resources are needed. Short-term usage and sporadic usage might require a strategy that is different from the one that is used for long-term investments. Processing and/or storage capacity are no longer assets that you must buy and manage, especially if you do not need them most of the time. Consider the ongoing cost of running and maintaining computer resources, given their actual utilization, and determine their real cost and associated ROA. Lastly, remember that cost must be measured on an ongoing basis. The cost structure of on-demand resource utilization is different from the cost structure of on-premises computing. Protect yourself by continually monitoring and managing your resource utilization, as well as the charges that this consumption creates. Whichever approach you choose, prepare a contingency plan in case the cost of that approach becomes too high.
Shy Cohen ([email protected]) is an independent Software Architect, Consultant, Speaker, Coach, and Entrepreneur. Before he started his own practice, Shy worked at Microsoft for many years as Senior Program Manager onprojects such as Oslo, WCF, MSN, Windows, and ISA.
41
Summary This article proposes that event-driven architecture (EDA) and service-oriented architecture (SOA) are really two sides of the same coin.
Introduction
While event-driven architecture (EDA) is a broadly known topic, both ACID is an acronym for giving up ACID integrity guarantees Atomicity, Consistency, and introducing eventual consistency Isolation, Durability: a make many architects uncomfortable. set of properties that Yet it is exactly these properties traditionally describe that can direct architectural efforts database transactions. toward identifying coarsely grained business-service boundaries services that will result in true IT-business alignment. Business events create natural temporal boundaries across which there is no business expectation of immediate consistency or confirmation. When they are mapped to technical solutions, the loosely coupled business domains on either side of business events simply result in autonomous, loosely coupled services whose contracts explicitly reflect the inherent publish/ A temporal boundary is a subscribe nature of the business. boundary on the axis of This article will describe how all of time, where one side of these concepts fit together, as well as the boundary exists in a how they solve thorny issues such as time-frame disjoint from high availability and fault tolerance. the other side.
Consumers do not initiate communication in EDA; instead, they receive events that are produced by emitters. The communication is also inherently unidirectional; emitters do not depend on any response from consumers to continue performing their work. Events are often named in passive, past-tense formfor example, customer updated and order cancelledand can represent state changes in the domain of the emitter. Events can be thought of as mirror images of the commands in a system. However, there might be cases in which the trigger for an event is not an explicit command, but something like a timeout.
Activity Services
Create Order
Entity Services
Create Customer
Create Order
42
T
Order Created
Order ID + Other Data
OrdToCust Customers Orders When we analyze the When an order has been created, if a customer was not provided, create a new customer rule, we can see that a clear temporal boundary splits it up into two parts. In a system that has this rule, what we will see is that at a given point in time, an order might from left to right, nothing is shared. Not only that, but after the event exist that does not have a corresponding customer. The rule also is published, the publisher no longer even needs to be online for the states the action that should be taken in such a scenario: the creation subscriber to process the event, so long as we use a durable transport of a new customer. There might also be a nonfunctional requirement (such as a queue). that states the maximum time that should be allowed for the action to These benefits become even more pronounced when we consider be completed. integration with other systems. Consider the case in which we want From a technical/database perspective, it might appear that we to integrate with a CRM, whether it is onsite or hosted in the cloud. In have allowed our data to get into an inconsistent state; however, that the EDA approach, if the CRM is unavailable (for whatever reason), the is only if we had modeled our database so that the Orders table had a order will still be accepted. Contrasting this with the classic commandnon-nullable column that contained CustomerIda foreign key to the oriented service-composition approach, we would see there that the Customers table. While such an entity-relationship design would be unavailability of the CRM would cause the entire transaction to time considered perfectly acceptable, we should consider how appropriate out and roll back. The same is true during integration of mainframes it really is, given the requirements of business consistency. and other constrained resources: Even when they are online, they The rule itself indicates the business perspective of consistency; can process only N concurrent transactions (see Figure 3). Because an order that has no connection to a customer is valid, for a certain the event publisher does not need to wait for confirmation from any period of time. Eventually, the business would like a customer to be subscriber, any transactions beyond those that are currently being affiliated with that order; however, the time frame around that can be processed by the mainframe wait patiently in the queue, without any strict (to a level of seconds) or quite lax (to a level of hours or days). It adverse impact on the performance of order processing. is also understandable that the business might want to change these If all systems had to wait for confirmation from one anotheras is time frames in cases in which it might provide a strategic advantage. common in the command-oriented approachto bring one system An entity-relationship design that would reflect these realities to a level of 5 nines of availability, all of the systems that it calls would would likely have a separate mapping table that connected Orders need to have the same level of availability (as would the systems to Customersleaving the Orders entity free of any constraint that that they call, recursively). While the investment in infrastructure relates to the Customers entity. might have business justification for one system (for example, That is the important thing to understand about eventual order processing), it can be ruinous to have to multiply that level of consistency: It starts by identifying the business elements that do not investment across the board for nonstrategic systems (for example, have to be 100-percent, up-to-the-millisecond consistent, and then shipping and billing). reflecting those relaxed constraints in the technical design. In this case, we could even go so far as to Figure 3: Load-leveling effect of queues between publishers and subscribers have each of these transactions occur in its own database, as shown in Figure 2.
Order Processing
Mainframe
Load Time
Load Time
43
Uploading of artifacts
Manoj Manuja (Manoj_Manuja@infosys. com), Rajender Kalra (Rajender_Kalra01@ infosys.com), and Ruchi Malhotra ([email protected]) belong to the Education and ResearchDepartment, Infosys Technologies Limited, inChandigarh,India.
bringing the new subscriber online, we can take the recording of all published events from the audit log and play them to the new subscriber, or perform the regular ETL style of data migration from one subscriber to another.
Publisher
Existing Subscriber
Subscribe
New Subscriber
In companies that are undergoing mergers or acquisitions, the ability to add a new subscriber quickly to any number of events from multiple publishers without having to change any code in those publishers is a big win (see Figure 4). This helps maintain stability of the core environment, while iteratively rolling out bridges between the systems of the two companies. When we look practically at
44
Business
Domain A
IT
App 1
Business
Domain A
IT
Service 1
Domain B
App 2
Domain B
Service 2
If SOA is to have any chance of improving IT-business alignment, the connection between the two needs to look more like the one that is shown in Figure 6. One could describe such a connection as a service owning or being responsible for a single business domain, so that anything outside the service could not perform any actions that relate to that domain. Also, any and all data that relates to that domain also would be accessible only within the service. The EDA model that we saw earlier enabled exactly that kind of strict separation and ownership all the while, providing mechanisms for interaction and collaboration. We should consider this strong connection when we look at rules such as: When an order has been created, if a customer was not provided, create a new customer. The creation of the order as an object or a row in a database has no significance in the business domain. From a business perspective, it could be the acceptance or the authorization of an order that matters. What SOA brings to EDA in terms of IT-business alignment is the necessity of events to represent meaningful business occurrences. For example, instead of thinking of an entity that is being deleted as an event, you should look for the business scenario around it for example, a product that is being discontinued, a discount that is being revoked, or a shipment that is being canceled. Consider introducing a meaningful business status to your entities, instead of the technically common deleted column. While the business domain of sales will probably not be very interested in discontinued products and might treat them as deleted, the support domain might need to continue troubleshooting the problems that clients have with those productsfor a while, at least. Modern-day collaborative businessanalysis methodologies such as value networks can help identify these domains and the event flows between them.
When we look at the code in each of the layers in light of the business domain that it addresses, we tend to see fairly tight coupling between a screen, its logic, and the data that it shows. The places in which we see loose coupling is between screens, logic, and data from different business domains; there is hardly any coupling (if at all) between the screen that shows employee details and the one that is used to cancel an order. The fact that both are screens and are categorized in the UI layer appears not to have much technical significance (if any business significance). Much the same can be said for the code that hooks those screens to the data, as well as the data structures themselves. Any consistency concerns that might have arisen by this separation have already been addressed by the business acceptance of eventual consistency. If there are business demands that two pieces of data that have been allocated to different services always be consistent, this indicates that service boundaries are not aligned with business boundaries and must be changed. This is extremely valuable. Architects can explain to the business the ramifications of their architectural decisions in ways that the business can understandThere might be a couple of seconds during which these two bits of data are not in sync. Is that a problem?and the answer to those kinds of question is Figure 7: Services logically connecting code used to iterate the from different layers architecture, so as to bring it into better alignment with the business. As soon as service boundaries reflect business boundaries, there is great flexibility within each service; each can change its own database schema without having to worry about breaking other services, or even choose to change vendors and technology to
UI
BL
DAL
DB
45
Conclusion
EDA is not a technical panacea to Web Servicescentric architectures. In fact, attempting to employ EDA principles on purely technical domains that implement command-centric business analysis will almost certainly fail. The introduction of eventual consistency without the ratification of business stakeholders is poorly advised. However, if in the process of architecture we work collaboratively with the business, map out the natural boundaries that are inherent in the organization and the way in which it works, and align the boundaries of our services to them, we will find that the benefits of EDA bring substantial gains to the business in terms of greater flexibility and shorter times to market, while its apparent disadvantages become addressed in terms of additional entity statuses and finer-grained events.
for a service-delivery platform that fosters the Web-scale adoption of service technologies. This architecture extends SOA with essential principles upon which the Web builds, such as openness and the support for communication that is based on persistent publication. Additionally, we adopt semantic technologies as a means to lift services and their descriptions to a level of abstraction that deals with computer-understandable conceptualizations, so as to increase the level of automation that can be achieved while carrying out common tasks during the life cycle of services, such as their discovery, composition, and invocation. Further extensions come from the adoption of Web 2.0 principlesnotably, the tight integration of people as service prosumers and RESTful services as a technology that complements traditional Web services. Finally, automated context-adaptation capabilities are embedded within the architecture to support the use of services in unforeseen contextsthus, increasing the versatility of services while retaining their manageability. For more information, visit the Service Oriented Architectures for All (SOA4All) Web site.
Dr. Carlos Pedrinaci is a research fellow at the Knowledge Media Institute (KMi) at the Open University, United Kingdom. Dr. Elena Simperl works as senior researcher and vice-director of the Semantic Technology Institute at the University of Innsbruck. Reto Krummenacher is researcher and project assistant for the Semantic Technology Institute at the University of Innsbruck. Barry Norton is a senior researcher for STI Innsbruck at the University of Innsbruck.
Udi Dahan is an internationally renowned expert on software architecture and design. Recognized four years in a row with the coveted Most Valuable Professional (MVP) award by Microsoft Corporation for solutions architecture and connected systems, he is also on the advisory board of Microsofts next-generation technology platforms: WCF/WF/OSLO, the Software Factories Initiative, and the Composite Application Library & Guidance. He provides clients all
Validation Service, SMS Provisioning Service, SMS Billing service, and MMS Provisioning Service. Event-Driven SOA Events can be of two types: internal system events and business events. A business event is any meaningful activity that alters the flow of a business process or triggers a new process. The subscription to a service by an end user is an example of a business event, which will invoke the orchestration. Usually, services follow a simple requestreply communication, which might not be appropriate in the current scenario. For instance, the generic provisioning service will interact with the appropriate provisioning system; the time that is taken to provision a service can vary, depending on the type of service that is requested. In this case, the provisioning service should be able to signal the orchestration when the provisioning is complete. This is an example of an internal event and can be addressed by using asynchronous Web services, such as WaitHandlers or the WCF Callback mechanism. Managing SOA with ESB ESB is the infrastructure for managing an SOA. The Microsoft ESB Toolkit 2.0 itinerary Web service can be used for service discovery and invocation. The resolver mechanism can provide a generic service resolution. An itinerary service can be an entry point to this system; it will read the message and use the ESB resolver mechanism internally to invoke the business-process orchestration. Addition of a new service in the system means registering the service in a service registry and using the appropriate discovery mechanism. Message transformations between services can be performed by using Microsoft BizTalk mapsthus, each service will complete its processing, and the ESB will take care of the message transformations and routing. Conclusion SOA is a very powerful architectural concept; but, all by itself, it might not address the dynamics of a real-time business. Managing SOA with ESB offers many potential benefits; also, events and services complement each other and cannot be looked at in isolation from one other. In combination with event processing and ESB, SOA can provide an optimal solution that not only addresses the business complexities, but also adds value to it.
Aarti Kaur is a solution architect with Mahindra Satyam, India. The complete article is available on her blog.
47
Summary This article presents some of the characteristics of future service-oriented systems. Also, it focuses on a set of architecture and design drivers for these future serviceoriented systems that can help meet new expectations without sacrificing the loosely coupled, stateless, standards-based characteristics that have driven SOA adoption in many contexts. The article concludes with thoughts on the key role of the architect in the serviceoriented systems-development process
Introduction
It is clear that service-oriented architecture (SOA) is having a substantial impact on the way in which software systems are developed. According to a 2007 Gartner Group report, 50 percent of new mission-critical operational applications and business processes were designed in 2007 around SOA, and that number will be more than 80 percent by 2010. Despite recent news that SOA adoption rates are falling and that SOA is dead, Forrester Group recently reported that SOA adoption is increasing across all of its vertical-industry groups. The reality is that SOA is currently the best option available for systems integration and leverage of legacy systems. SOA is a way of designing, developing, deploying, and managing systems, and it is characterized by coarse-grained services that represent reusable business functionality. Service consumers compose applications or systems by using the functionality that services provide through standard interfaces. At a high level: Services provide reusable business functionality. Service consumers are built by using the functionality from available services. Service-interface definitions are first-class artifacts. There is a clear separation between the service interface and service implementation that come from the legacy systems, external systems, or code that was built specifically for this purpose. An SOA infrastructure enables the discovery, composition, and invocation of services. Protocols are predominantly, but not exclusively, message-based document exchanges. From a more technical point of view, SOA is an architectural style or design paradigm; it is neither a system architecture nor a
48
complete system. As an architectural style, it is characterized by a set of components and connectors, situations in which the style is applicable, and benefits that are associated with implementing thestyle. If it is implemented correctly, SOA adoption can provide business agility, reuse of business functionality, and leverage of legacy systems for an organization. Many organizations recognize these potential benefits and are adopting SOAsome more successfully than others. SOA has indeed crossed the chasm,1 according to a recent Software AG user survey in which 90 percent of the respondents claim to have made some commitment to SOA adoption.2 However, as with any technology, as SOA is adopted within organizations and becomes a mainstream paradigm for systems development, the requirements and expectations that are placed on service orientation increase. What was initially an approach for asynchronous document-based message exchanges now has performance, availability, reliability, security and other expectations of traditional distributed systems. As a result, the loosely coupled, stateless, standards-based nature of the relationship between service consumers and service providers in service-oriented systems is changing, so as to meet these new requirements. In addition, global enterprises and the emerging market of third-party services that are being made available through the Cloud are also placing expectations on service-oriented system architecture and design. The first part of this article presents some of the characteristics of future service-oriented systems. The second part focuses on a set of architecture and design drivers for these future service-oriented systems that can help meet new expectations without sacrificing the loosely coupled, stateless, standards-based characteristics that have driven SOA adoption in many contexts. Finally, the article concludes with thoughts on the key role of the architect in the service-oriented systems-development process.
organizational boundaries, the requirements on their systems also expandfrom consumer, provider, and infrastructure perspectives. What follows are some requirements that will be typical of these future (or even current) service-oriented systems. Security The security threats for service-oriented systems are not new or different; it is the level of exposure that is greater. Service-oriented systems have an unknown and dynamic attack surface. Attack surface refers to the set of ways in which an adversary can exploit vulnerabilities and potentially cause damage. An attack surface can be measured in terms of three kinds of resources that are used in attacks on the system: methods (for example, an API), channels (for example, sockets), and data (for example, input parameters). The greater the number of resources that are accessible for attack, the greater the attack surface and, therefore, the more insecure the software environment.4 From a more global perspective of security, issues such as identity management, dynamic secure-service composition, and trust in third-party services become important requirements in this type of system. Runtime Monitoring and Adaptation Runtime monitoring of systems is a common practice for determining the health of a system. SOA infrastructures can be configured to gather certain measures during system execution, and tools can be integrated into the system to produce reports and alerts if measures cross certain thresholds. Runtime adaptation refers to the capability of the system to adjust itself at runtime when these thresholds are crossed, so as to continue to meet quality requirements. For example, a system might start an additional instance of a service under particular load conditions or restrict access to a service if there is a suspicion that the security of the system has been compromised. These actions are possible when there is full control over a system; however, when services belong to third parties, there is much less control, and it becomes difficult to weave and consolidate the different logs that are emitted from different sources to paint an overall picture of the system. Dynamic Binding The word dynamic is often used to describe the binding between service consumers and services. There are various degrees of dynamism. At the lower end of the spectrum is late binding of a proxy service to a specific service instance that depends on user context or load-balancing policies. At the higher end of the spectrum is fully dynamic binding in which service consumers are capable of querying service registries at runtime, selecting the best service from the list of returned services, and invoking the selected serviceall at runtime, and without human intervention. Late binding is a common, out-of-the-box feature of many commercial and open-source SOA infrastructures, such as an enterprise service bus (ESB). Fully dynamic binding, on the other hand, requires semantically described services that use an ontology that is shared between service consumers and service providers. Semantic Web Services represent an active area of research, as well as an unsolved problem that is not yet ready for large-scale deployment. Multiple Consumers and Consumer Devices As service-oriented systems start crossing organizational boundaries, the variety of service consumers will increase. Services will have to deal with heterogeneous service-consumer development and computing platforms. The proliferation and increasing power of
Dirk Becker ([email protected]) is a member of the global-integration architecture team at Credit Suisse.
49
oriented systems. In a multiorganizational environment, governance has to be extended to include policies and procedures for the identification and binding to external services and the establishment and monitoring of service-level agreements (SLAs) between service providers and consumers.
living Business Process Execution Language (BPEL) process that represents the business-process models. -> BPM Recognize that there is a natural demarcation between business process and business decision points. This will allow for the externalization of decision-making. -> EDM Agree that all architectural interaction will occur via state-defining asynchronous events. -> ESP/CEP Normalize system messages and develop standard patterns, so that your architecture will scale and remain relevant. -> ESB Virtualize and componentize services to increase business-level awareness, better infrastructure utilization, reuse of services, and reduction of complexity. -> SCA Approach security as a federation of services in which endpoints are policy-driven and become responsive to changing needs. -> IDLMS Benefits of Developing an Enhanced SOA From a Business Perspective As the business owner, you now have the tools that will give you far greater insight into and control of the systems that run your business. From a Technical Perspective 1. As the information architect, you will have the ability to conceptualize, simulate, and monitor compliance against the desired business outcomes. 2. As the developer, you now have a clearer separation of concerns and should see a more constrained system that offers higher levels of productivity and maintainability.
Alex Cameron ([email protected]) is a Fellow and Enterprise Architect within the HP TSG Group. Visit the Web site.
between providers and consumers. For example, architectural constructs would have to be put in place to support advanced service registries, multiple messaging options, test instances, SLA monitoring, and any other characteristic that contributes to the perception of service usability. Federation As service-oriented systems grow in size, the centralization of certain aspects might become a bottleneck. Federation can be a solution to this problem. In this context, federation refers to predefined agreements on aspects of the system that allow the autonomy of individual components. Some aspects of service-oriented systems that might require federation in large-scale settings are: Identity management. This is the aspect that is most commonly associated with federation in SOA environments. Federated identity management means that there is a cooperative contract that has been set up among multiple identity providers and uses a decentralized approach, so that an identity in one of the identity providers is recognized by other identity providers in the federation.12 From a consumer perspective, this means not having to log in to every single system that is involved in the execution of a particular business process or workflow. Some of the challenges
of federated identity management include trust, translation among multiple standards, and synchronization. SOA infrastructures. In large-scale service-oriented systems that span multiple organizations, it is unlikely that all organizations will have the same SOA infrastructure. In this case, federation would allow participating organizations to maintain their SOA infrastructures, while shared aspects such as policy management and governance mechanisms are agreed upon, propagated throughout the system, and implemented locally. Service registries. Federated service registries allow registries to appear as a single, virtual registry and individual organizations to retain local control over their own registries. Regardless of the aspect of the system that is federated, there will need to be architectural constructs for establishing agreements, virtualization, and synchronization upon changes. Automated Governance The key to governance implementation is adding control to a system without creating a lot of extra work to its developers and users. The approach is governance automation. The burden of ensuring compliance and enforcement gets pushed to the SOA infrastructure. There are tools and SOA infrastructures in which some governance
51
for external integration and the expectations of SOA adoption increase, many promises will be made on the benefits of SOA in these scenarios that will probably not be validated until implementation. The role of the architect is to perform early, contextual technology evaluation and continuous technology scouting that can lead to more informed decisions on what parts of the system will benefit from SOA technologies.13 Architecture trade-off analysis It is well known that tradeoffs must be made in systems, because the accomplishment of a certain quality is often at the expense of another quality. Common examples of trade-offs are performance versus modifiability, availability versus safety, security versus performance, and interoperability versus cost.14 The use of service orientation in systems that have high system-quality requirements will require architectural trade-offs at the expense of loose coupling and flexibility. If the added overhead for a service-oriented system to meet quality requirements comes at the expense of the characteristics for which SOA is known, the decision to use serviceoriented concepts should be reevaluated. An architecture analysis and evaluation method that is guided by business drivers and performed via scenarios in which the usage of SOA technologies is key can also help an architect make better, early, and informed decisions. Finally, as service-oriented systems start to cross organizational boundaries, architects will have to reevaluate the use of SOA as an architectural style in these systems or to architect their systems in such a way that qualities are met without having to sacrifice the characteristics that have made SOA a worthwhile technology to adopt.
References
1
Conclusion
SOA is potentially being stretched beyond its limits. What was initially an approach for asynchronous document-based message exchanges now has performance, availability, reliability, security, and other expectations of traditional distributed systems. To solve this problem, multiple specifications and standards have been proposed and created, middleware products are becoming more robust, and the community has started to embrace terms such as event-driven SOA and real-time SOA. Therefore, the loosely coupled, stateless, standards-based nature of the relationship between service consumers and service providers in service-oriented systems is changing, so as to meet these new requirements. In addition, global enterprises and the emerging market of third-party services that are being made available through the Cloud are placing new expectations on service-oriented system architecture and design. SOA is not a one-size-fits-all solution. As an architectural style, SOA is an appropriate solution in some situations; however, there are situations in which it is not appropriate or it has to be used in conjunction with other technologies to meet service qualities. The architect of future service-oriented systems is going to play a crucial role in determining what expectations can or cannot be met by SOA adoption, and where trade-offs can be made for the benefit of the organization and the accomplishment of system qualities.
2
This term was coined by Geoffrey A. Moore in his book Crossing the Chasm: Marketing and Selling High-Tech Products to Mainstream Customers (Rev. ed. New York: Collins Business Essentials, 2006) and refers to the chasm that exists between visionaries (early adopters) and pragmatists (early majority) from atechnology-adoption perspective. Softwareag.com. Best Practices for SOA Governance User Survey. Software AG, Summer 2008. [Accessed July 13, 2009.] Forrester.com. Enterprise and SMB Software Survey, North America and Europe, Q4 2008. Forrester, February 2009. Manadhata, Pratyusa K., Kamie M. C. Tan, Roy A. Maxion, and Jeannette M. Wing. An Approach to Measuring a Systems Attack Surface. CMU Technical Report CMU-CS-07-146, August 2007. Pardo-Castellote, Gerardo. SOA Feature Story: Real-Time SOA Starts with the Messaging Bus! SOA World Magazine, November2007. [Accessed July 13, 2009.] Joshi, Rajive. Data-Oriented Architecture: A Loosely Coupled RealTime SOA. Real-Time Innovations, Inc., August 2007. Simanta, Soumya, Ed Morris, Grace A. Lewis, Sriram Balasubramaniam, and Dennis B. Smith. A Scenario-Based Technique for Developing SOA Technical Governance. Software Engineering Institute, June 2009. [Accessed July 13, 2009.] Kontogiannnis, Kostas, Grace A. Lewis, and Dennis B. Smith. AProposed Taxonomy for SOA Research. In Proceedings of the International Workshop on the Foundations of ServiceOriented Architecture (FSOA 2007). Software Engineering Institute, June2008. [Accessed July 13, 2009.] Brun, Yuriy, Giovanna Di Marzo Serugendo, Cristina Gacek, Holger Giese, Holger Kienle, Marin Litoiu, Hausi Mller, Mauro Pezz,
52
Grace Lewis is a Senior Member of the Technical Staff at the Software Engineering Institute (SEI) in Pittsburgh, PA. Currently, she is the lead for the System of Systems Engineering team within the Systems of Systems Practice (SoSP) initiative in the Research, Technology, and Systems Solutions (RTSS) program. Her current interests and projects are in service-oriented architecture (SOA), technologies for systems interoperability, characterization of software-development life-cycle activities in systems of systems environments, and establishing an SOA research agenda. Graces latest publications include multiple reports and articles on these subjects, as well as a book in the SEI Series in Software Engineering. She is also a member of the technical faculty for the Master in Software Engineering program at Carnegie Mellon University (CMU). Grace holds a B.Sc. in Systems Engineering; an Executive MBA from Icesi University in Cali, Colombia; and a Masters in Software Engineering from CMU.
With data centers considering infrastructure resources as service-ondemand resources and amalgamating them with application services, we redefine an SOA service as comprising the server-side application and the infrastructure that hosts it. The business benefits of this type of architecture are twofolda reduced infrastructure footprint and a lower cost of service delivery and are enunciated as follows: Commoditization of services enables true SOA-service rationalization that takes place at the service level, instead of separately at the software and hardware boundaries. This reduces the Figure 1: Infrastructure scope data-center (or infrastructure) footprint for the business and, thus, provides a more Article Scope environmentally friendly SOA service at lowered Demand Provisioning forecasting software costs. Encapsulation of an SOA service across the hardware-software stack provides a complete measure of return on investment (ROI)from application to infrastructure. This gives a more Control Bus transparent, accurate, and rationalized view of the cost of a service and its time-to-market (TTM) cost. These factors drive down the cost of an SOA service, because it allows the business to measure reusability of the SOA service across all tiers of VM VM IT and, thereby, create a framework for more (SOA Service) (SOA service) efficient use of services. Problem Statement Generally, an SOA service comprises applications that serve client (or consumer) applications. The management of infrastructure resources that the service consumesspecifically, in relation to capacity and performanceis left to the engineer to plan and model manually. This means that service demands might not always be met within the required time constraints, if the capacity
management is reactive instead of proactive; thus, there is a danger that the infrastructure will act as a bottleneck. Conversely, infrastructure resources can remain idle should the service demand be below that which is provisioned by capacity planning. This represents a waste of money on unused infrastructure. Furthermore, capacity planning and modeling, when performed properly, can itself be a very costly and time-consuming exercise and would need to be performed regularly in order to track changing SOA-service workloads over time. Therefore, a more efficient and cost-effective solution is needed for designing and implementing SOAservices.
Producers
ESB
Client application
Consumers
53
LAN
SAN/NAS
In this context, we refer to infrastructure resources as the hardware platform and its related services that are required for hosting the SOA applications, not the ESB, as Figure 1 shows. Proposed Solution We propose an SOA service that makes up the entire hardwaresoftware stack. We term such an SOA service a composite service element that would be provided in a virtualized environment, as Figure 2 shows, in order to better provision the resources of the service and measure its resource usage. The host platform (comprising hardware computing resources such as CPU, RAM, and local storage) as well as the operating system would be provisioned as a virtual machine (VM). The VM, as much as the software that it hosts, can be considered a service; we denote this VM as an SOA service element. Furthermore, the entire hardware-software stack can be virtualized to include the storage (such as SAN and NAS devices), the LAN, and the interfaces to both of these. This means that all of the layers in the architecture that comprise the logic, dataaccess, and data-storage layers can be virtualized and provisioned as part of a single SOA service, as Table 1 shows.
Table 1: Components of a proposed composite service element Layer Presentation Logic Data access Data storage Component Client-side applications Server-side applications Database/ middleware SAN, NAS, local storage Virtualized? Possibly On a VM On a VM Possibly Part of composite service element? No Yes Yes Yes Nayan B. Ruparelia ([email protected]) is an IT Architect at EDS, part of Hewlett Packards TSG group. Salim Sheikh ([email protected]) is an Independent Consultant and Enterprise Architect, certified in a number of frameworks including TOGAF, Zachman, COBIT and ITIL. He is also a Certified Process Professional and LEAN/Six Sigma practitioner.
54
21
subscribe at www.architecturejournal.net