Talend ESB GettingStarted UG 51b en
Talend ESB GettingStarted UG 51b en
Talend ESB GettingStarted UG 51b en
5.1_b
Copyleft
This documentation is provided under the terms of the Creative Commons Public License (CCPL). For more information about what you can and cannot do with this documentation in accordance with the CCPL, please read: http://creativecommons.org/licenses/by-nc-sa/2.0/ This document may include documentation produced at The Apache Software Foundation which is licensed under The Apache License 2.0.
Notices
Talend and Talend ESB are trademarks of Talend, Inc. Apache CXF, CXF, Apache Karaf, Karaf, Apache Cellar, Cellar, Apache Camel, Camel, Apache Maven, Maven, Apache Archiva, Archiva are trademarks of The Apache Foundation. Eclipse Equinox is a trademark of the Eclipse Foundation, Inc. SoapUI is a trademark of SmartBear Software. Hyperic is a trademark of VMware, Inc. Nagios is a trademark of Nagios Enterprises, LLC. All other brands, product names, company names, trademarks and service marks are the properties of their respective owners.
Table of Contents
1. Overview of this document ......................................................................................... 1 2. Talend ESB Components and Architecture ..................................................................... 3 2.1. Overview ....................................................................................................... 3 2.2. Advanced Features .......................................................................................... 4 2.3. Web Services Support ..................................................................................... 4 2.4. Standard OSGi Runtime ................................................................................... 5 2.5. Messaging ..................................................................................................... 6 2.6. Talend Studio and Talend ESB packages ............................................................. 6 2.7. Talend Studio ................................................................................................. 9 2.8. List of resources ........................................................................................... 13 3. Downloading and installing Talend ESB software .......................................................... 15 3.1. Prerequisites to using Talend ESB .................................................................... 15 3.2. Downloading, extracting the software ............................................................... 15 3.3. Installing Talend Studio ................................................................................. 16 3.4. Installing Talend Runtime - quick start .............................................................. 18 3.5. Software prerequisites for the Demos ................................................................ 20 3.6. Starting a Talend Runtime container ................................................................. 21 3.7. Creating an alternate container ......................................................................... 22 4. A simple SayHello example for Talend Studio .............................................................. 25 4.1. Overview ..................................................................................................... 25 4.2. Building a simple "SayHello" data service ......................................................... 26 4.3. Creating a "SayHello" provider ........................................................................ 26 4.4. SayHello consumer ........................................................................................ 32 4.5. Exporting the service and running it in an Talend Runtime container ....................... 36 4.6. SayHelloRoute example ................................................................................. 38 5. A real-world Rent-a-Car demo example ....................................................................... 45 5.1. Overview of Rent-a-Car demo example ............................................................. 45 5.2. Building the demo ......................................................................................... 47 5.3. Deploying the demo ...................................................................................... 49 5.4. Running the demo ......................................................................................... 51 5.5. Uninstalling the demo .................................................................................... 54 6. Importing the demo into Eclipse ................................................................................. 57 6.1. Building the project files ................................................................................ 57 6.2. Importing into Eclipse .................................................................................... 57 6.3. Running the examples .................................................................................... 58 7. Enabling the Service Locator and Service Activity Monitoring for the demo ........................ 59 7.1. Technical overview of the Service Locator ......................................................... 59 7.2. Technical overview of Service Activity Monitoring ............................................. 60 7.3. Overview of the demo ................................................................................... 62 7.4. Building the demo with functionality enabled ..................................................... 62 7.5. Uninstalling previous features .......................................................................... 63 7.6. Installing the Service Locator and Service Activity Monitoring .............................. 63 7.7. Installing the Rent-a-Car examples features ........................................................ 67 7.8. Running the demo in one container .................................................................. 69 7.9. Running the demo in two containers - simulating server failover ............................. 70 7.10. Uninstalling the demo features ....................................................................... 72 7.11. Manual configuration issues .......................................................................... 73 8. Enabling WS-Security for the demo ............................................................................ 75 8.1. Technical overview of the Security Token Service (STS) ...................................... 75 8.2. Installation and settings .................................................................................. 77 8.3. Building the demo with WS-Security enabled ..................................................... 78 8.4. Uninstalling the demo features ......................................................................... 79 8.5. Manual configuration of the STS based Authentication in the demo ........................ 79 9. Enabling the Talend Artifact Repository for the demo .................................................... 83 9.1. Overview ..................................................................................................... 83 9.2. More information .......................................................................................... 83
9.3. Chapter structure ........................................................................................... 9.4. Downloading and installing Archiva ................................................................. 9.5. Uploading the demo to a local Archiva repository using Maven files ....................... 9.6. Deploying the demo manually in Talend ESB Standard Edition with Nexus .............. 9.7. Preparing Talend Administration Center to deploy the demo to Talend Runtime ......... 9.8. Deploying the demo with Talend Administration Center to Talend Runtime .............. Glossary ....................................................................................................................
84 84 85 86 88 89 91
iv
List of Figures
2.1. Overview of Karaf components ................................................................................. 5 2.2. Talend Open Studio for ESB components .................................................................... 7 2.3. Talend Enterprise ESB components ............................................................................ 8 2.4. Talend ESB Integration perspective with a Service design ............................................. 10 2.5. Talend ESB Integration perspective with a Job design .................................................. 11 2.6. Talend ESB Mediation perspective ........................................................................... 12 3.1. Talend Runtime container startup ............................................................................. 19 5.1. Rent-a-Car client application and services .................................................................. 46 5.2. Rental Client service calls ...................................................................................... 47 5.3. Rental Client GUI ................................................................................................. 52 5.4. List of available cars ............................................................................................. 52 5.5. Confirmation of booking ........................................................................................ 53 7.1. Architecture of Service Activity Monitoring ............................................................... 61 7.2. Demonstrating server failover .................................................................................. 71 9.1. Talend Artifact Repository provisioning repositories .................................................... 85
how the Service Activity Monitoring (SAM) can be enabled in the Rent-a-Car demo to monitor service calls, typically for collecting usage statistics and fault monitoring. Chapter 8, Enabling WS-Security for the demo shows how to enable WS-Security to authenticate access to the demo. Finally, Chapter 9, Enabling the Talend Artifact Repository for the demo gives information on enabling the Talend Artifact Repository. Note: the Rent-a-Car example (complete with variations) and this documentation may be downloaded from http:// www.talend.com/download.php
Advanced Features
Integration and mediation user interface - see Section 2.7, Talend Studio Service Enablement - support for REST, SOAP web services Mediation and Routing
your standalone Java application should be represented in a Web services context. Two types of Web services development with CXF are possible: 1. Contract-first development: WSDLs define what operations and types a Web service provides. This is often referred to as a Web services contract, and in order to communicate with a Web service, you must satisfy the contract. Contract-first development means that you start out by writing a WSDL file (either by hand or with the help of tooling), and then generating stub Java class implementations from the WSDL file by using a tool like CXF. 2. Code-first development: The other way to develop Web services is by starting out with a Java class and then letting the Web service framework handle the job of generating a WSDL contract for you. This is the easiest mode of development, but it also means that the tool (CXF in this case) is in control of what the contract will be. When you want to fine-tune your WSDL file, it may be better to go for contract-first approach. Apache has been certified and tested to work with the broadest set of vendors Web services implementations. Users benefit from this interoperability testing, which reduces the overall cost and complexity for application integration. Talend ESB supports the creation of SOAP and REST Web services and offers the best WS-* functionality in the market, including support for WS- Addressing, WS-Reliable Messaging, and WS-Security over both HTTP and JMS transports. In addition, the Web services stack in Talend ESB distribution goes well beyond Apache CXF, with support for: OSGi containers along with illustrative examples and documentation Development tools, like Maven plug-ins WSDL document creation Apache Spring configuration generation Management and monitoring of services
Messaging
Karaf supports the following features: Hot deployment: Karaf monitors jar files inside the [home]/deploy directory. So if a jar is copied into this directory, it is automatically installed inside the runtime; subsequently this can be updated or deleted, and Karaf will act correspondingly. Dynamic configuration: Services are usually configured through a standard OSGi service, using property files, which are monitored; changes are propogated to the service. Logging: using a centralized logging back end supported by Log4J. Managing instances: Karaf provides simple console commands for managing multiple instances. See Chapter 5, A real-world Rent-a-Car demo example for an example of using Karaf. For further information, please see Talend ESB Container Administration Guide and http://karaf.apache.org/.
2.5. Messaging
Enterprise messaging software has been used for integration projects more than 30 years now. Not only is messaging a style of communication between applications, its also a widely adopted style of integration between applications. Messaging fulfills the need for both notification and asynchronous, loosely coupled interoperation between applications. Talend ESB embeds Apache ActiveMQ message broker to support a number of different messaging options. Although ActiveMQ is written in Java and implements the JMS specification, APIs for many languages other than Java are provided, including C/C++, .NET, Perl, PHP, etc. In addition to that many other features are added on top to provide high availability, performance, scalability, reliability, and security for enterprise messaging. The job of the message broker is to mediate events between distributed applications, guaranteeing that they reach their intended recipients. The broker therefore must be highly available, performant, and scalable for this goal.
Talend Administration Center also provides a graphical interface for Service Locator and Service Activity Monitoring (SAM) functionalities and gives the ability to remotely deploy the data services and mediation routes developed from the Talend Enterprise ESB Studio. Note that while the Service Locator and Service Activity Monitoring are in both Talend Open Studio for ESB and Talend Enterprise ESB editions, the Web-based user interfaces to these are part of the Web-based Talend Administration Center, and are thus in the Talend Enterprise edition only. This Talend Enterprise edition software is licensed under the Talend License, and requires a license file to be installed to enable these features. We now look at each of these in more detail:
Three different types of functional block are defined: The blue block represents a Talend Studio API where you can carry out data integration or data service processes, mediation routes and services. For more information, see their corresponding chapters in the Talend Open Studio for ESB User Guide. The red blocks represent one or more Talend Runtimes (execution container) deployed inside your information system. Talend Runtime enables you to deploy and execute the Jobs, Routes, and Services created in the Talend Studio. For more information on how to deploy items in Talend Runtime, see the Talend Open Studio for ESB User Guide and for more information about Talend Runtime itself, see the Talend ESB Infrastructure Services Configuration Guide.
If you have several Talend Runtimes on which to deploy the Services and Routes, you will be able to load balance their execution according to your needs. All instances of Talend Runtime will communicate between each other via the Service Locator to identify the one more likely to deploy and execute them. The orange block represents a monitoring database gathering log information of the execution of your data processes and service activity. Data processes log information can be captured with the use of the tFlowMeterCatcher, tStatCatcher, tLogCatcher components. For more information, see the Talend Open Studio Components Reference Guide. And to automate the functionalities of the tFlowMeterCatcher, tStatCatcher, tLogCatcher components without using them, you can use the Stats & Logs tab. For more information regarding Stats & Logs, see the Talend Open Studio for ESB User Guide. The Service Activity Monitoring allows the end-users to monitor service calls. It provides monitoring and consolidated event information that the end-user can use to understand the underlying requests and replies that compose the event, monitor faults that may be unexpectly generated and support the system management decisions. For more information on the Service Activity Monitoring, see its corresponding chapter in Talend ESB Infrastructure Services Configuration Guide. For more information on the installation of all these components, see the Chapter 3, Downloading and installing Talend ESB software of the present guide or see the link to the Installation Guide on the Talend ESB download page http://www.talend.com/download.php.
Five different types of functional block are defined: The light blue block includes one or more Talend Studio APIs and Web browsers that could be on the same or on different machines. From the Studio API, you can carry out data integration or data service processes, mediation routes and services, and publish them on the Artifact Repository. Talend Enterprise ESB Studio allows the user (such as a project manager, a developer, or an administrator) to work on any project for which he has authorization to build Web,
Talend Studio
REST and data services, and mediation routes. For more information, see their corresponding chapters in the Talend Enterprise Studio User Guide. From the Web browser, end-users connect to the remotely based Talend Administration Center through a secured HTTP protocol. The violet block includes a web-based Talend Administration Center (application server) with two shared repositories: one based on an SVN server and one based on a database server. The Talend Administration Center enables you to set up the execution of the tasks that handle routes or services execution into the Talend Runtime. Through the Talend Administration Center, you can access and manage Routes or Services created from Talend Studio and published into the Artifact Repository, and set and monitor their deployment and execution in the Talend Runtime. For more information, see the Talend Administration Center User Guide The dark blue block represents the Artifact Repository that stores all the Routes and Services that are published from the Talend Studio and are ready to be deployed in Talend Runtime. The red block represents one or more Talend Runtimes (execution container) deployed inside your information system. The Talend Runtime deploys and executes the routes and services retrieved from the Artifact Repository according to the set up defined in the Talend Administration Center via the web application. For more information on how to manage deployment, see the Talend Administration Center User Guide and for more information about Talend Runtime itself, see the Talend ESB Infrastructure Services Configuration Guide. If you have several Talend Runtimes in which to deploy the Service and Route artifacts, you will be able to load balance their execution according to your needs. All instances of Talend Runtime will communicate between each other via the Service Locator to identify the one more likely to deploy and execute the artifact(s) set to deployment in Talend Administration Center. The Talend Runtime elected for the deployment will request for the artifact(s) to deploy and execute from the Artifact Repository and the Artifact Repository will thus send the artifact(s) requested along with all the dependencies needed for its/their execution to the Talend Runtime, Talend Runtime that will deploy and execute them. The orange block represents the Dashboard: the Activity Monitoring Console and the Service Activity Monitoring. The Activity Monitoring Console allows end-users to monitor the execution of technical processes. It provides detailed monitoring capabilities that can be used to consolidate collected log information, understand the underlying data flows interaction, prevent faults that could be unexpectedly generated and support the system management decisions. The Service Activity Monitoring allows the end-users to monitor service calls. It provides monitoring and consolidated event information that the end-user can use to understand the underlying requests and replies that compose the event, monitor faults that may be unexpectly generated and support the system management decisions. For more information on the installation of all these components, see the Talend Enterprise ESB Installation Guide or the Talend Enterprise Installer User Guide.
Integration perspective
a Java perspective (Talend Enterprise ESB Studio including m2eclipse Plugin) a soapUI perspective (Talend Enterprise ESB Studio only) These are discussed in more detail in the rest of this section. We will use the term "Talend Studio" for both versions unless there is a specific difference involved.
A Service in the 'Services' node is a Web-Service defined by a WSDL. The WSDL can be just imported, created from scratch in the tooling using the embedded graphical WSDL editor or an existing WSDL can be imported and then edited within the studio. In this case the Service is based on this WSDL information and each service operation can then be implemented in the Job Design node.
10
Mediation perspective
A data service Job is a graphical design, of one or more components connected together, that allows you to set up and run data service operations. Jobs address all of the different sources and targets that you need for data integration processes and combine it with Web services. Additionally, in Talend Enterprise ESB Studio you can use the shared repository feature to work in larger teams, and share resources. It has the facility of team collaboration - team members can store and share their business models, integration and service jobs, integration services and metadata in an industry-standard source manager (SVN). This promotes reusability of objects and code, as well as facilitating the design of development best practices. There are also more extended productivity features like Wizards (for example, for SAP) and deployment options (Publish to Talend Artifact Repository) available in Talend Enterprise ESB Studio. More information on this can be found in Talend Enterprise Studio User Guide, Talend Open Studio for ESB User Guide (see Section 2.8, List of resources for a complete list of manuals).
11
Mediation perspective
EIP and hence Camel are focused on asynchronous messaging patterns because asynchronous patterns are necessary for scalability. But asynchronous messaging is more complex. Camel wraps that complexity in a common framework which is widely understood by integration developers. Because of the high-level abstractions in Camel (for example, Message, Exchange, Component, Endpoint), you can use the same APIs to interact with various systems. For example, every route implements Component and Endpoint APIs and works with the same basic Message and Exchange structure. Each component is implemented to work with specific protocols and data types. So you can wire together very different protocols and system very easily while using the same patterns and routing semantics. Camel mediates both formats and transports between different endpoints. Typically, this allows Services to communicate with each other, via Camel mediation, even if using different message formats, and written using different languages. It also makes adding functionality like logging and tracking easy. Camel is also very lightweight and can be embedded or deployed anywhere you like, such as in a standalone Java application, web application, Java EE application or an OSGi bundle. You can use the Java Domain Specific Language (DSL) to specify a route. For example: From(file:directory).to(jms:queuename) This simple statement polls files from a directory and sends the content of each file a JMS message to a JMS queue. This can be done because Camel provides a uniform interface based on the Exchange object. Talend ESB Mediation perspective On top of Apache Camel, and integrated with the Talend Studio, the Route Builder (Mediation perspective) is a GUI that allows a developer to build these Routes in a visual way. You can run the Routes in a standalone mode or export them as OSGi Bundles which can be easily deployed within the Talend Runtime container. Additionally in Talend Enterprise ESB Studio you can use the shared repository feature to work in larger teams.
Out of the box, Talend ESB supports over 80 protocols and data types, through its extensive library of components. These components enable a Route to connect over transports, use APIs, and understand data formats. You can also create your own components very easily; the component API is very concise. Talend ESB has a modular
12
List of resources
architecture, which allows any component to be loaded into Camel, regardless of whether the component ships with Camel, is from a third party, or is your own custom creation. The creation of your own components (for example, for processing, connectivity, routing, and so on) is simple and well documented.
http://activemq.apache.org/ Security Token Service WSS4J (WS-Security) http://ws.apache.org/wss4j Service Locator ZooKeeper http://zookeeper.apache.org/ Talend Runtime container Karaf (OSGi) http://karaf.apache.org/ Talend ESB Getting Started User Guide, Talend ESB Infrastructure Services Configuration Guide, Talend ESB STS User Guide Talend ESB Getting Started User Guide, Talend ESB Infrastructure Services Configuration Guide Talend ESB Getting Started User Guide, Talend ESB Container Administration Guide, Talend ESB Development Guide
13
Talend ESB STS User Guide - detailed description of configuring and using Security Token Service Talend Mediation Developer Guide - describes Camel mediation components Talend Service Factory User Guide - describes service enablement using CXF
14
Customers who subscribe to the Enterprise Edition of Talend software are sent an email with download instructions and a license file. Download the relevant file(s) and when the download is complete, extract the archive files on your hard drive.
16
2.
5.
Click the [] button next to the connection field. Talend ESB Getting Started User Guide 17
2.
In the Email field, type in a valid email address. Also specify the path of your workspace, or accept the default. Note that here we assume that you are using a local environment. In Talend Enterprise ESB Studio, you can collaborate with other users with a shared environment, and this is described in Talend Enterprise Studio User Guide. 3. Click OK to validate and go back to the login window. In the login window, the connection information you just gave displays in the Connection area.
You are now ready to start using Talend Studio. As first time user, you need to set up a new project - as an example, we set up a very simple "SayHello" example in Chapter 4, A simple SayHello example for Talend Studio . You can also import the Demo project which contains job samples (these are discussed in Talend Open Studio for ESB User Guide).
18
For further information, see Section 3.6, Starting a Talend Runtime container.
19
The Service Activity Monitoring Server can be installed into the Servlet Container (such as Apache Tomcat) or Talend Runtime container. It supports Apache Derby, MySQL, Oracle, SQL Server, IBM DB2 and H2 Database Engine to store Events data. To install the Service Activity Monitoring server within the Talend Runtime container please type: tesb:start-sam in the console. This will also automatically start an Apache Derby database. Note: the Service Activity Monitoring within the Talend Runtime can only be used with Apache Derby - if you would like use one of the other supported databases, use the Tomcat deployment of the Service Activity Monitoring server. For further information and examples, see Chapter 7, Enabling the Service Locator and Service Activity Monitoring for the demo.
Start-all
Instead of these three individual start commands you can also use: tesb:start-all in the container, which starts the Service Locator, Service Activity Monitoring server and the Security Token Service.
The OSGi commands explained in the following sections will be entered at this prompt. In order to check that the container has started up and activated all components, enter the command: list You should get a listing like the following fragment (note that the details of the list can vary, depending on what is installed): karaf@trun> list START LEVEL 100 , List Threshold: 50 ID State Blueprint Spring [ 49] [Active ] [ ] [ [ 50] [Active ] [ ] [ .2) [ 51] [Active ] [ ] [ [ 52] [Active ] [ ] [ [ 53] [Active ] [ ] [ 10725) [ 54] [Active ] [ ] [ 110725) ....
Level Name ] [ 60] Activation 1.1 (1.1) ] [ 60] geronimo-servlet_2.5_spec (1.1 ] [ 60] JavaMail API (compat) (1.4.4) ] [ 60] geronimo-jta_1.1_spec (1.1.1) ] [ 60] Jetty :: Utilities (7.4.5.v201 ] [ 60] Jetty :: IO Utility (7.4.5.v20
Make sure all the listed components show state "Active" before you initiate any further action. If any component remains in state "Resolved", after waiting for 30-60 seconds, you can manually activate it entering "start <ID>"
21
where "<ID>" refers to the number in the first column, for example, "start 100" activates the component with ID "100".
Under Linux, go to directory <Talend.runtime.dir>/alternate-container and execute the following command: ./bin/trun Under Windows, go to directory <Talend.runtime.dir>\alternate-container and execute the following command: .\bin\trun.bat 4. Wait until the initialisation phase has finished, then execute the Karaf configuration adaption script. Input this command at the alternate-container console: source scripts/configureC1.sh The settings are now updated and saved locally. The next time you run the alternate container, the new settings will be used, and both container and alternate-container can be running at the same time. 5. It is not required for the following demos, but if you would like to create a third or fourth container, copy the original container directory to a third location (see above). Again, start running the third container and use this call at the Karaf prompt to set another unique group of settings: source scripts/configureC2.sh Similarly, a fourth container should use the call: source scripts/configureC3.sh You can reconfigure the default values in any container using: source scripts/configureC0.sh
23
3.
Additional information
This chapter gives enough information to create and run the demo. For a comprehensive look at the Talend Studio UI, please see Talend Open Studio for ESB User Guide or Talend Enterprise Studio User Guide, depending on the version of software that you are using. For more details on specific components mentioned in this demo, please see Talend Enterprise Components Reference Guide or Talend Open Studio Components Reference Guide.
A screen opens for the project details. Give the Project name "SayHello" and Project description "simple demo".
26
Creating a service
Click Finish. This action returns us to login screen. Then select the project from the project drop down menu, and click Open. 2. If you haven't registered previously, then please enter your details to create an account to access TalendForge (tutorials, support, and so on), or else click Skip to skip this step for the moment. If you have registered and logged in previously, this screen does not appear. Now, the components are loaded, which takes a few seconds. This is followed by a message about "Generation Engine Initialization in progress" (you can optionally run this in the background, and tick the box "always run this in the background" for future use). The Welcome screen in Talend Studio is now displayed.
3.
4.
For the purposes of this demo, click "X" to the right of Welcome, to exit to the main Talend Studio screen.
27
2.
Enter the name ("SayHelloService") and purpose ("Demo") of the service, and click Next.
3.
The next screen is the "Assign WSDL" screen. WSDL defines the external contract to the service that clients can use. (Please see Section 2.3, Web Services Support for more on this). Select Create new WSDL and click Finish to return to the main screen. Now the main screen has a SayHelloService_0.1.wsdl tab displayed. This WSDL now contains a new port (SayHelloServicePort) for the service, and default request and response operations (SayHelloServiceOperationRequest and SayHelloServiceOperationResponse respectively). Hover over the grey arrows to the right of the operations to display their parameters:
4.
5.
It is possible to make changes to the operations - add new operations and edit existing ones. However, the default operations are enough for this demo example.
28
2.
Next, we implement the operation - expand the elements displayed in SayHelloService 0.1 and right click on SayHelloServiceOperation 0.1 and select "Create New Job".
Keep the default name and prefix "SayHelloServicePortType_SayHelloServiceOperation", and click Finish. This creates a new job, which allows us implement the operations using the components in the main grid. 3. Now we start with the default template of the tESBProviderRequest and tESBProviderResponse. Separate the operations on the grid by clicking the top left of tESBProviderResponse_1 square, and drag it to the right.
29
Click on the border of a square for moving the shape itself around the grid. Undo or edit the changes these changes by right clicking the square and selecting the appropriate actions. Click on the center of a square for actions particular to the component (for example, examining details of the component). 4. We now add some business logic. tXMLMap is a component that transforms and routes data from single or multiple sources to single or multiple destinations. Perform a search for the tXMLMap component in the Palette on the right hand side. There may be two instances found under different sections, but they are both the same, so choose either. Drag and drop it between the two operations. Right click the center of tESBProviderRequest_1 and select "Row", then "Main" and drop the end of the line on tXMLMap_1.
5.
6.
Next, right click on tXMLMap_1 and select "Row", then "Main" and drop the end of the line on tESBProviderResponse_1. Give it the name "Response", and click Ok. Click the default "Yes" when asked if you wish to import the schemas.
2.
3.
30
Click Ok. So, the request and response operations are implemented from the existing schemas. 4. Next we simply link the input from one to the output of the other. Left click the in on the left hand side and drag it to the out expression in the response on the right hand side. Next, we modify the default expression that is sent. On the right hand side, under Expression, click the "HelloServiceRequest" value, and double click the "..." button beside it. Edit the expression (which will evaluate to a name) by clicking in the field, and add "Hello " + before what is there, and +"!" after it.
5.
Click Ok. You will see the updated expression now on the right hand side. Click Ok to return to the main job design screen. 6. Finally, in order to see more as the job executes, we add some logging information. This is done simply by searching for "tLog" in the palette, and dragging tLogRow from the Palette on the right hand side and dropping it between the tXMLMap_1 and tESBProviderResponse_1. That is the implementation of the SayHelloServiceOperation complete.
31
This builds the job, and the log output shows that the web_service has been assigned a port 8090 and has been published for other services to use. Select and copy http://localhost:8090/services/SayHelloService for later use. In Talend Studio you can test each Service operation which you implement in a Job Design individually, to quickly test the implementation. If you would like to test the entire service with multiple operations, you need to export the service and deploy in the runtime. It is not possible to test a service with multiple operations in Talend Studio.
2.
32
Note: tFixedFlowInput generates as many lines and columns as you want using context variables, and tESBConsumer calls a specified method from the invoked Web service, and returns a class, based on parameters. 3. Now we are ready to start implementing. Click on the center of tFixedFlowInput_1 and in Component tab below, select Use Inline Table. Then click the "..." button next to Edit schema to open the Schema editing window.
In this window, click "+" to add a string argument, and rename "newColumn" to "Name" and click Ok to close this window. 4. Returning to the tFixedFlowInput_1 Component tab, use the + button to add sample rows, and successively replace the newline text with names Peter, Alice and Bob.
33
This is the example data that the consumer will send to the SayHelloService. 5. Now, in the main "Job SayHelloConsumer 0.1" tab, link the outputs as before, by right clicking and dragging tFixedFlowInput_1 > row > main to tXMLMap_1. Then right click tXMLMap_1 > row with a new Output name of "request" (click the default yes to get the schema of the target component), and drop the end on tESBConsumer_1. Now double click on the tXMLMap to implement it as before. On the right hand side, click on payload > Import From Repository. In the Metadatas screen that appears navigate from File XML to SayHelloServiceOperationRequest 0.1, and click on it. This enables us to call the service operation. Click Ok. Now left click and drag the Name on the left hand side to the in parameter on the right hand side and click Ok to return to the main screen.
6.
7.
8.
Finally, take care of the logging information. Right click on center of tESBConsumer_1, drag and select Row > Response and drop the end on tLogRow_1 so that any responses should go there. Similarly, right click tESBConsumer_1, select Row > fault and drop the end on tLogRow_2 so that any faults should go there.
34
In summary, tFixedFlowInput generates Peter, Alice, Bob, these will be passed by tXMLMap to tESBConsumer, which will do three corresponding invocations on the target provider.
2.
Finally, we run the consumer job. Click on the Run (Job SayHelloConsumer) tab, and click the "Run" button. The job builds and executes, and we see in the output the three names in a Hello message.
So, we have successfully created and run a "SayHello" consumer and provider in Talend Studio.
35
2.
Now we export the service. Under Services, right click "SayHelloService 0.1" and select "Export Service". In the "Save As" window, we can specify the deploy folder within the Talend Runtime container deploy subdirectory, so that the service is directly ready to run and click Finish. The service is saved as a file SayHelloService-0.1.kar. We show here an example on Windows:
This process builds and exports the job to the deploy directory. 3. Since this a dynamic loading environment, the service immediately starts running. If we now look at the container window, and type "list", we will see it there.
36
1. 2.
Now we check it is working, by starting the consumer. The port that the service is running at has changed, and now uses the container port, which is by default 8040, so we need to update the port. Under Job Designs, click "Job SayHelloConsumer 0.1". Click on the middle of tESBConsumer_1. Then go to the Component tab. Click on the "..." button next to Service configuration, which opens a WSDL settings window. We need to update the port number to use the Talend Runtime container port - so change 8090 to 8040, and click the refresh button.
3. 4.
5. 6.
Now run the consumer job as before from the Job tab, and we see the same output as before. If we look at the output in the Talend Runtime container, we also now see the Hello messages generated by the server running in the container.
37
SayHelloRoute example
So that is how to create a simple SayHello consumer and service, running them Talend Studio, and also running the service in the Talend Runtime container.
2.
3.
Click on Routes > "SayHelloRoute 0.1" to display the route grid. Now you will notice that the palette has changed from the one in the Integration perspective to the Mediation perspective. Now we will create a typical content-based router, dragging and dropping components from the palette to the route grid. The request message is coming in from the consumer, so we drag and drop the Messaging > cCXF component, which intercepts messages coming into server endpoints. Then, as we are doing a content-based route, we drag and drop the Routing > MessageRouter component, which reroutes messages depending on a set of conditions. Then we get two more instances of Messaging > cCXF for the two target services.
38
Multiple cCXF components with the same label in a Route is not supported
It is recommended to label each component with a unique name to better identify its role in the Route. Having duplicate labels may cause problems for the code generation of some components. For example, when using the cCXF component, if you have two components with the label cCXF_1, this will cause errors in the code generation phase. 4. To implement this, we just need to add some parameters. So, click on cCXF_1, and then click on the Component tab below: In the address component, we paste in the previous service address http://localhost:8042/services/ SayHelloService, and update the port to be 8042, since the new service will be listening on this port. We use the WSDL file from the original service. We use the http:// address to get the live background service information. It's in "http://localhost:8040/services/SayHelloService?WSDL", for all three services. The dataformat is PAYLOAD, that is, we are looking at the message body.
At any point, you can save the current information by selecting File > Save or clicking on the Save icon in the top left hand corner. 5. We repeat the previous step for the other components: cCXF_2 - except that the port number in the Address field is 8040, and for cCXF_3, where the port number in the Address field is 8041. So, in summary, we get a request in on port 8042 (cCXF_1), and we send it to either port 8040 (cCXF_2) or 8041 (cCXF_3), depending on the contents of the message. Talend ESB Getting Started User Guide 39
6. 7.
So now, we start connecting the elements. Connect tCXF_1 to cMessageRouter_1 by right clicking the center of tCXF_1 and selecting Row > Route and dropping the end onto cMessageRouter_1. Then we create a "When" trigger for one service, by right-clicking cMessageRouter_1, selecting Trigger > When and dropping the end on cCXF_2.
8.
Similarly, we add an "Otherwise" trigger from cMessageRouter_1 to cCXF_3 by right-clicking cMessageRouter_1, selecting Trigger > Otherwise and dropping the end on cCXF_3. This results in the following:
9.
We need to add the "When" condition. Right click on the when1 line, which brings up a small dialog. Select Type "simple", and the Condition is "${bodyAs(String)} contains 'Alice'".
That is, any messages with "Alice" in the body will be routed to a service in a Talend Runtime container that listens on port 8040.
2.
40
3.
To do a simple check that all this has been done correctly, before we add the consumer, we will use the route execution in the studio. Click on the "Run (Job SayHelloRoute) tab" and click Run:
This checks that the CXF configuration information is syntactically correct. It also gives an output of "0 rows" on the grid, which reflect the flow of messages, so that you know that the parts are connecting. Leave the job running. If you get syntax errors, then click on each component in turn, and examine it in the Component tab. Check in particular that the double quotes are all there, and that the port numbers are correct. 1. Now, we switch back to our consumer to run the demo for real. Click on the Integration perspective in the top right hand corner. Then click "SayHelloConsumer 0.1" under "Job Designs"; then the consumer information is displayed on the main grid. We need to update the port number, so click the Component tab, and the "..." button next to Service Configuration. Update the port number to be 8040, and click the refresh button to retrieve the WSDL information. Now update the endpoint to be that of the route - http://localhost:8042/ services/SayHelloService.
2.
We can now send a request by running the consumer job by clicking on the tab "Run (Job SayhelloRoute)" and click Run. We see the "Hello Peter", "Hello Alice", "Hello Bob" output in the consumer output as before. The main grid shows "3 rows", that is, the three messages being passed along with none going to fault.
41
Exporting Routes
3.
Now we look at the "Route SayHelloRoute 0.1" tab, we will see that 1 message went to the cCXF_2 provider and 2 messages when to the cCXF_3 provider.
4.
if we look at the output in the two containers, we also see that the messages have been routed correctly: This is the output for one container:
5.
42
Exporting Routes
This screen shows the route being exported to a /container/deploy directory in a Talend Runtime so that it is directly deployed, but it can also be stored elsewhere and copied to the deploy directory later.
43
In this scenario, there are two main services that will be demonstrated, ReservationService and CRMService. You will create and orchestrate these services with the help of a simple graphical Rent-a-Car client application, simply called the Rental Client. The Rental Client provides the basic functionality to search for the cars available within a given time period. To implement its business logic the Rental Client calls operations from both the CRM Service and the Reservation Service as shown in the diagram below. 1. The search takes into account the status (silver, gold, platinum) of the customer you select. This customer profile information is passed to the reservation service and is used to determine the outcome of the search. From the list of results you can select the car that best fulfills the customer requirements and then make a reservation simply by clicking on the Book button. At the end of the submission process, the booking confirmation is displayed along with a Reservation ID.
2.
3.
46
47
Description Reservation Service which implements getAvailableCars, submitCarReservation and getConfirmationOfReservation operations. the feature files which will be used to install the Rent-a-Car demo to an OSGi container. optional project files for the soapUI tool. used by the Maven build process. advanced functionality (NOT used by basic demo - see Section 9.5, Uploading the demo to a local Archiva repository using Maven files for details).
Now we look in detail at the services and client subdirectories: 1. first we look at crmservice: Directory or file name common/ client/ client-sl-sam/ client-sts/ client-all/ service/ service-endpoint-jmx/ service-endpoint-slsam/ service-endpoint-sts/ service-endpoint-all/ Description This directory contains the CRMService.wsdl file which is used to generate the initial code. This is an example client application that shows a CXF client invoking the CXF endpoint. This directory contains the CRMService client enabled to be used in Service Locator and Service Activity Monitoring scenario. This directory contains the CRMService client enabled to be used in Security Token Service scenario. This directory contains the CRMService client enabled to be used with all features. CRMService service implementation. the basic CRMService endpoint the CRMService endpoint enabled to be used in Service Locator and Service Activity Monitoring scenario. the CRMService endpoint enabled to be used in Security Token Service scenario. CRMService endpoint enabled to be used with all features.
2. reservationservice has the same structure, except that the parts relate to the ReservationService 3. app-reservation has code to build the commands and simple UI client: Directory or file name src/ target/ pom.xml Description source code target directory for the build used by the Maven build process
48
mvn clean install Running this command, the demo is built, and OSGi bundles are created for deployment into an OSGi container.
Version number
Note: Commands in this document refer to version "5.1.1" of the Talend ESB. If you are using another version, replace the version sub-string .../5.1.1/... by the number of your version. 1. Add the Rent-a-Car feature location into the primary (<Talend.runtime.dir>/container/) OSGi container, not in the alternate-container (see warning above). At the OSGi console prompt (karaf@trun>), type the following command: features:addurl xml mvn:org.talend.esb.examples.rent-a-car/features/5.1.1/
This command allows you to use the relevant bundles directly from the Maven repository where they have been installed by the Maven build. 2. Install the Rent-a-Car services feature into the OSGi container. Type the following console command: features:install tesb-rac-services 3. Check feature installation success.
49
Type the following console command: list You will see the demo bundles in the list of installed bundles - as you can see, both CRMService and ReservationService bundles are installed:
ID State Blueprint [137] [Active ] [ ] [138] [Active ] [ ] (5.1.1) [139] [Active ] [ ] [140] [Active ] [ ] (5.1.1) [141] [Active ] [ ] (5.1.1) [142] [Active ] [ ] Endpoint (5.1.1)
Spring [ [ [ [ [ [ ] [ ] [ ] [ ] [ ] [ ] [
Level Name 60] CRMService Common (5.1.1) 60] ReservationService Common 60] CRMService Service (5.1.1) 60] CRMService Service Endpoint 60] ReservationService Service 60] ReservationService Service
If you have already run a previous example, you may need to run the following command in order to ensure that internal bundle caches are updated: features:refreshurl 2. Install the Rent-a-Car application feature into the OSGi container. Type the following console command: features:install tesb-rac-app 3. Check feature installation success. Type the following console command: list You will see the demo bundles in the list of installed bundles (note you will only see the service bundles installed if you are running both in the same container):
50
ID State [137] [Active [138] [Active (5.1.1) [139] [Active [140] [Active (5.1.1) [141] [Active
Blueprint Spring Level Name ] [ ] [ ] [ 60] CRMService Common (5.1.1) ] [ ] [ ] [ 60] ReservationService Common ] [ ] [ ] [ ] [ ] [ ] [ ] [ 60] CRMService Client (5.1.1) 60] ReservationService Client 60] App Reservation (5.1.1)
] [Created] [
51
Click Find to see the results. You will see the list of available cars.
Click the highlighted line or select any other option and click Reserve. You will now see the confirmation page:
52
53
1 2 3 4 5
VW Golf Standard 50.00 75.00 40.00 BMW 320i Sport 60.00 90.00 45.00 Mazda MX5 Sport 65.00 95.00 50.00 Lexus LX400 SUV 85.00 120.00 100.00 Mercedes E320 Delux 95.00 140.00 100.00
Reservation ID SGA-686277 Customer details ---------------Name: Andrea Ebert eMail: [email protected] City: Munich Status: PLATINUM Car details ----------Brand: BMW Model: 320i Reservation details ------------------Pick up date: 2011/04/27 Return date: 2011/04/27 Daily rate: 60.00 Weekend rate: 90.00 Credits: 210 Thank you for renting a car with Talend :-)
54
If you want to uninstall all the demo features, for example, because you want to try other scenarios from this guide, execute following commands at the OSGi console (which includes the previous commands - not all of these will be installed in both containers if you are using two containers): features:uninstall tesb-rac-services features:uninstall tesb-rac-app features:uninstall tesb-rac-common Then enter features:list to ensure that all features have actually been uninstalled. You should see this in the console: [uninstalled] [5.1.1 [uninstalled] [5.1.1 [uninstalled] [5.1.1 ] tesb-rac-common ] tesb-rac-services ] tesb-rac-app repo-0 repo-0 repo-0
You can also remove Rent-a-Car feature location from the OSGi container(s). At the OSGi console prompt, type the following command: features:removeurl xml mvn:org.talend.esb.examples.rent-a-car/features/5.1.1/
55
3.
Choose Select root directory and click the associated Browse to locate the examples/talend/tesb/ rent-a-car directory. Back in the Import Projects window, under Projects select the project or projects that you would like to import, and then click Finish.
4.
The Talend ESB example projects you selected will be imported into your Eclipse workspace. At this stage, you can now see the source code, modify it, and take advantage of all other Eclipse features.
58
Chapter 7. Enabling the Service Locator and Service Activity Monitoring for the demo
This chapter describes how to enable both of these for one Rent-a-Car demo: the Service Locator component, to publicly register the service endpoints the Service Activity Monitoring (SAM) to monitor service calls, typically for collecting usage statistics and fault monitoring First we do a technical overview of both of these components, and then we do a step-by-step description of running the Rent-a-Car demo with this functionality enabled.
Additionally, it allows service providers to automatically register and unregister their service endpoints. In this way, the providers actively advertise the availability of their service endpoints to consumers.
More information
For more detailed information on the Service Locator, beyond this manual, please see Talend ESB Infrastructure Services Configuration Guide. The Service Locator consists of two parts: 1. The Service Locator server hosting an endpoint repository 2. the CXF feature used to enable usage of the locator for CXF service consumers and providers. Like any standard CXF feature, it has separate functionality for service and consumer: when the provider becomes available or unavailable, a provider-side Locator Feature extension registers and deregisters service endpoints respectively in the endpoint repository. when a service call to a provider is about to be made, a consumer-side Locator Feature extension transparently retrieves service endpoint addresses from the endpoint repository. Note: the Service Locator server implementation is based on proven open source technology - Apache ZooKeeper. To learn more about Apache ZooKeeper, see http://zookeeper.apache.org.
More information
For more detailed information on Service Activity Monitoring, beyond this manual, please see Talend ESB Infrastructure Services Configuration Guide. Service Activity Monitoring consists of two parts: Agents (sam-agent) which gathers and sends monitoring data A Service Activity Monitoring Server which processes and stores the data The sequence of how these are used is as follows: 1. The Agent creates events out of requests and replies from both the service consumer and provider side. 2. The events are first collected locally and then sent to the Service Activity Monitoring Server periodically (so as not to disturb the normal message flow). 3. When the Service Activity Monitoring Server receives events from the Agent, it optionally uses filters and/or handlers on those events and stores them into a database.
60
The Agent and Service Activity Monitoring Server are made available as follows: The Agent is packaged as a JAR that needs to be on the classpath of the service consumer and provider. The Service Activity Monitoring Server is deployed as a WAR in a servlet container and needs access to a database. One service call can generate four events: For example: A consumer is sending a request (REQ_OUT), the service receives request (REQ_IN), the service sends response (RESP_OUT) and the consumer receives response (RESP_IN). An Agent can be configured to collect all four events in this service call, on both the consumer and provider side. For further event processing all of these events will get the same "flow id".
The FlowId Producer is a component used to generate the FlowId (a UUID) for the Message Header and pass it to subsequent messages. For each message exchange, the flow id is created if there is no flow id present. So, for the first client the flow id is created for each service call. When you have an intermediary this receives a service call, but also calls other services; then the flow id is carried from the incoming call to all calls that follow this call. Then on the server side the flow id is taken from the request and also set on the response. Filters or handlers can be set up on both the Agent side and Service Activity Monitoring Server side, and can subsequently be used to filter events and manipulate the event's content. There are some built-in filters and handlers (for example, StringContentFilter, PasswordHandler) and you can develop your own filters and handlers by extending the EventFilter or EventHandler Service Provider Interface (SPI).
61
5.
7.
<bean class="org.talend.esb.servicelocator.cxf.LocatorFeature"> <property name="selectionStrategy" value="randomSelectionStrategy"/> </bean> <ref bean="eventFeature"/> </jaxws:features> The MANIFEST.MF file of related client and service-endpoint will be updated by adding Require-Bundle: samcommon,sam-agent,locator.
This command removes features for the simple scenario from the container.
63
# Configured zookeeper endpoints (divided by a comma if several instances # uses). The service locator client will one by one pick an endpoint to # connect to the service locator until a connection is established. locator.endpoints=localhost:2181 # Endpoint prefix property is needed because we run services in # a container where the endpoints is only relative to the container. endpoint.http.prefix=http://localhost:8040/services endpoint.https.prefix=https://localhost:9001/services locator.strategy=defaultSelectionStrategy locator.reloadAdressesCount=10 connection.timeout=5000 session.timeout=5000
For further information about Service Locator properties configuration please read the Talend ESB Infrastructure Services Configuration Guide.
64
The dataDir value may need to be updated (it is the location to store the in-memory database snapshots - see Talend ESB Infrastructure Services Configuration Guide for more details). 2. Now, the Service Locator server can be started and stopped with the scripts from the zookeeper/bin directory. Under Linux, ensure execution rights for the Service Locator startup scripts: chmod a+x zookeeper/bin/*.sh From directory <Talend.runtime.dir>, the Service Locator is started by the following command: Linux: ./zookeeper/bin/zkServer.sh start in Windows command line: zookeeper\bin\zkServer.cmd start 3. It can be stopped by the following command: Linux: ./zookeeper/bin/zkServer.sh stop Windows: zookeeper\bin\zkServer.cmd stop For more information please read Talend ESB Infrastructure Services Configuration Guide.
18:41:37,843 | INFO | ExtenderThread-8 | ZooKeeper | org.apache.zookeeper.ZooKeeper 373 | 36 - org.apache.hadoop.zookeeper 3.3.2 | Initiating client connection, connectString=locator_host1:2181 sessionTimeout =5000 watcher=org.talend.esb.locator.ServiceLocator$WatcherImpl@1e26d9b 18:41:37,843 | INFO | d-8-SendThread() | ClientCnxn | org.zookeeper.ClientCnxn$SendThread 1041 | 36 - org.apache.hadoop.zookeeper - 3.3.2 | Opening socket connection to server locator_host1/192.168.150.201:2181 18:41:37,843 | INFO | d(sop-td57:2181) | ClientCnxn | org.zookeeper.ClientCnxn$SendThread 949 | 36 - org.apache.hadoop.zookeeper - 3.3.2 | Socket connection established to locator_host1/192.168.150.201:2181, initiating session 18:41:37,844 | INFO | d(sop-td57:2181) | ClientCnxn | org.zookeeper.ClientCnxn$SendThread 738 | 36 - org.apache.hadoop.zookeeper - 3.3.2 | Session establishment complete on server locator_host1/192.168.150.201:2181, sessionid = 0x32e71447e6b004c, negotiated timeout = 4000
65
Also, we can judge how well the feature is working by watching the Locator logs in the console window:
18:41:38,765 | INFO | ExtenderThread-8 | ServiceLocator | talend.esb.locator.ServiceLocator 177 | - - | Register endpoint http://localhost:8888/soap/CRMServiceProvider/ for service {http://services.talend.org/CRMService}CRMServiceProvider. 18:44:07,250 | INFO | xtenderThread-12 | ServiceLocator | talend.esb.locator.ServiceLocator 177 | - - | Register endpoint http://localhost:8888/soap/ReservationServiceProvider/ for service {http://services.talend.org/ReservationService}ReservationServiceProvider.
66
logging error please change the db.recreate property in org.talend.esb.sam.server.cfg to false (db.recreate=false) after you install the Service Activity Monitoring Server. tesb:start-sam will start Derby database on localhost with default 1527 port, and install tesb-datasourcederby feature and tesb-sam-server feature. If you are using other Database (for example MySQL, H2, Oracle, etc), please follow the instructions of DataSource and Service Activity Monitoring Server installation/configuration in the Talend ESB Infrastructure Services Configuration Guide.
67
features:addurl mvn:org.talend.esb.examples.rent-a-car/features-slsam/5.1.1/xml 2. If previous examples have already been installed, do: features:refreshurl 3. Then enter the following commands: features:install tesb-rac-services-sl-sam features:install tesb-rac-app-sl-sam 4. Then, you can use the list command to check if everything installed properly: list For example: [ 172] [Active ] [ ] [ ] [ 80] Service Activity Monitoring :: Common (5.1.1) [ 173] [Active ] [ ] [ ] [ 80] Service Activity Monitoring :: Agent (5.1.1) [ 176] [Active ] [ ] [ ] [ 50] Service Locator Client for CXF (5.1.1) [ 186] [Active ] [Created ] [ ] [ 80] Service Activity Monitoring :: Datasource-derby (5.1.1) [ 187] [Active ] [ ] [ ] [ 80] Apache ServiceMix :: Bundles :: derbynet (10.8.1.2_1) [ 188] [Active ] [ ] [ ] [ 80] Service Activity Monitoring :: Derby Starter (5.1.1) [ 194] [Active ] [ ] [ ] [ 80] Service Activity Monitoring :: Server (5.1.1) [ 195] [Active ] [ ] [ ] [ 80] CRMService Common (5.1.1) [ 196] [Active ] [ ] [ ] [ 80] ReservationService Common (5.1.1) [ 197] [Active ] [ ] [ ] [ 80] CRMService Service (5.1.1) [ 198] [Active ] [ ] [ ] [ 80] CRMService Service Endpoint Locator and SAM (5.1.1) [ 199] [Active ] [ ] [ ] [ 80] ReservationService Service (5.1.1) [ 200] [Active ] [ ] [ ] [ 80] ReservationService Service Endpoint SL and SAM (5.1.1) [ 201] [Active ] [ ] [ ] [ 80] CRMService Client Locator and SAM (5.1.1) [ 202] [Active ] [ ] [ ] [ 80] ReservationService Client Locator and SAM (5.1.1) [ 203] [Active ] [ ] [ ] [ 80] App Reservation (5.1.1)
68
features:addurl 5.1.1/xml 3.
mvn:org.talend.esb.examples.rent-a-car/features-sl-sam/
4.
Then enter: features:install tesb-rac-services-sl-sam Note: only install these features, not tesb-rac-app-sl-sam. Then, you can use the list command to check if everything installed properly: list For example: [152][Active][ ][ ][60] [153][Active][ ][ ][60] [154][Active][ ][ ][60] [155][Active][ ][ ][60] [156][Active][ ][ ][60] [157][Active][ ][ ][60] [158][Active][Created][ Service Activity Monitoring :: Common(5.1.1) Service Activity Monitoring :: Agent(5.1.1) CRMService Common (5.1.1) ReservationService Common (5.1.1) CRMService Client SAM (5.1.1) ReservationService Client SAM(5.1.1) ][60] App Reservation (5.1.1)
############################################### getCRMInformation() invoked ... request data: org.talend.services.crm.types.LoginUserType@fc0795 ############################################### ############################################### getCRMInformation() invoked ... response data: org.talend.services.crm.types.CustomerDetailsType@560854 ############################################### ############################################### getAvailableCars() invoked ... request data: org.talend.services.reservation.types.RESProfileType@a14297
69
############################################### ############################################### getAvailableCars() invoked ... response data: org.talend.services.reservation.types.RESCarListType@11ad529 ############################################### which shows the requests and responses for the CRMService and ReservationService.
For more information about the table's schema definition, please look in Appendix A: Event Structure and Appendix B: EVENTS_CUSTOMINFO Structure in Talend ESB Infrastructure Services Configuration Guide.
70
There are a number of steps to this: 1. 2. 3. 4. 5. The app-reservation looks up the CRMService endpoint in the Service Locator. It then connects directly to the CRMService in the default container. Then we stop the CRMService The app-reservation automatically looks up the CRMService endpoint again in the Service Locator. It then connects directly to the CRMService in the alternate-container.
71
[ 142] [Active ] [ ] [ ] [ 60] CRMService Service Endpoint Locator and SAM (5.1.1) (The number, here "142", will probably be different in your console). 3. 4. Stop it with the command stop 142 (using the bundle number you have found). Then, click "Find" in the application GUI to activate the call to the CRMService.
In the first console you will see the expected warning message: 12:30:55,049 | WARN | AWT-EventQueue-0 | ache.cxf.common.logging.LogUtils 369 | - - | Interceptor for {http://services.talend.org/CRMService}CRMServiceProv ider#{http://services.talend.org/CRMService}getCRMInformation has thrown exception, unwinding now org.apache.cxf.interceptor.Fault: Could not send Message. The warning is only shown if the log level is set appropriately. But even if the warning doesn't appear you still can verify from the output which container recieves the service call. The log level can be adjusted in container/ etc/java.util.logging.properties. Afterwards, the Locator will switch to the CRMService in the second container; and in the alternate container console you will see something like: ############################################### getCRMInformation() invoked ... request data: org.talend.services.crm.types.LoginUserType@1142653 ############################################### ############################################### getCRMInformation() invoked ... response data: org.talend.services.crm.types.CustomerDetailsType@1e03fce ############################################### In first console: ############################################### getAvailableCars() invoked ... request data: org.talend.services.reservation.types.RESProfileType@1d510be ############################################### ############################################### getAvailableCars() invoked ... response data: org.talend.services.reservation.types.RESCarListType@13e49a8 ############################################### Now we can see the ReservationService working in the first container, and the CRMService in the second container.
72
1.
In the default container, enter the following commands at the console: features:uninstall tesb-rac-services-sl-sam features:uninstall tesb-rac-common-sl-sam features:uninstall tesb-rac-app-sl-sam
2.
In the alternate container, enter the following commands at the console: features:uninstall tesb-rac-services-sl-sam features:uninstall tesb-rac-common-sl-sam
3.
Enter features:list to ensure that all features have actually been uninstalled. From Talend Runtime container, you will see: [uninstalled] [5.1.1] tesb-rac-common-sl-sam repo-0 [uninstalled] [5.1.1] tesb-rac-services-sl-sam repo-0 [uninstalled] [5.1.1] tesb-rac-app-sl-sam repo-0
From alternate container, you will see: [uninstalled] [5.1.1] tesb-rac-common-sl-sam repo-0
<Talend.runtime.dir>/examples/talend/tesb/rent-a-car/crmservice/serviceendpoint-sl-sam/src/main/resources/META-INF/spring/beans.xml For example: <import resource="classpath:META-INF/tesb/locator/beans-osgi.xml" /> ... <jaxws:endpoint id="CRMService" ... <jaxws:features> <bean class="org.talend.esb.servicelocator.cxf.LocatorFeature"/"> </jaxws:features> </jaxws:endpoint>
For more information about Service Locator properties please read the Talend ESB Infrastructure Services Configuration Guide.
74
1.
The client sends an authentication request to the Security Token Service (Request Security Token - RST message). The Security Token Service validates the client's credentials. The Security Token Service issues a security token to the client (Request Security Token Response - RSTR message). The RSTR contains a security token, such as an XML Security Assertion Markup Language (SAML) token. The client initializes and sends a request message, containing the token, to the Service. The Service attempts to verify that the security token was issued by a trusted Security Token Service by checking the corresponding STS certificate. On success accepts it (essentially as equivalent to a "valid login"), and processes the request. The service initializes and sends a response message to the client.
2. 3.
4. 5.
6.
The Security Assertion Markup Language (SAML) tokens provide cross-platform interoperability and exchange security information between clients and services in different security domains. The receiver of the message with the token only needs to know the corresponding STS certificate in order to verify the token and able to use the authentication information from the token.
76
77
./alternate-container/log/tesb.log Log messages from services are stored in the same directory in the primary container.
When the STS example has finished being installed, and the cursor returns, type: features:refreshurl features:install tesb-rac-services-sts
Again, wait for the cursor to return. 5. In the alternate container Talend Runtime container console, type in: features:addurl 5.1.1/xml mvn:org.talend.esb.examples.rent-a-car/features-sts/
When the examples have been installed, and the cursor returns, type: features:refreshurl features:install tesb-rac-app-sts Again, wait for the cursor to return. Now, the STS sample is installed and ready to run. 6. Please run the Rent-a-Car client as normal (for example, by running car:gui in the default container - see Chapter 5, A real-world Rent-a-Car demo example). You can then observe the interactions with the Security Token Service. Talend ESB Getting Started User Guide
78
79
That listing demonstrates the sample JAX-WS client with: enabled STS client bean (<entry key="ws-security.sts.client">) The STS client includes wsdlLocation for the STS service(property name="wsdlLocation") STS service name (property name="serviceName") STS service endpoint name (property name="endpointName") the list of additional STS client parameters (property name="properties") for example certificate settings The STS service WSDL can be accessed from the running STS service via http://localhost:8040/ SecurityTokenService/UT?wsdl. You can see these changes in the following files: The CRM client is implemented in the <Talend.runtime.dir>/examples/talend/tesb/renta-car/crmservice/client-sts folder. The CRM client configuration is located in the: <Talend.runtime.dir>/examples/talend/tesb/ rent-a-car/crmservice/client-sts/src/main/filtered-resources/META-INF/ spring/beans.xml file. The Reservation client is implemented in the <Talend.runtime.dir>/examples/talend/tesb/ rent-a-car/reservationservice/client-sts folder. The Reservation client configuration is located in the: <Talend.runtime.dir>/examples/talend/ tesb/rent-a-car/reservationservice/client-sts/src/main/filteredresources/META-INF/spring/beans.xml file. The CRM service is implemented in the <Talend.runtime.dir>/examples/talend/tesb/renta-car/crmservice/service-endpoint-sts. The service configuration is located in the: <Talend.runtime.dir>/examples/talend/tesb/ rent-a-car/crmservice/service-endpoint-sts/src/main/resources/META-INF/ spring/beans.xml file. The Reservation service is implemented in the <Talend.runtime.dir>/examples/talend/tesb/ rent-a-car/reservationservice/service-endpoint-sts. The service configuration is located in the: <Talend.runtime.dir>/examples/talend/tesb/ rent-a-car/reservationservice/service-endpoint-sts/src/main/resources/ META-INF/spring/beans.xml file.
80
code on the server needs a way to distinguish between messages intended for the STS and those intended for the service itself; the action used on the request serves that purpose. A major part of the WS-Security performance cost comes from the wide use of asymmetric encryption. Asymmetric encryption is a useful tool because it works with key pairs. Each key in the pair can be used to encrypt messages that the other key can decrypt. The owner of a key pair can make one key publicly available so that anyone can use it to encrypt messages to the owner securely, and also decrypt messages from the owner (thereby verifying the sender's identity). WS-SecureConversation uses WS-Policy and WS-SecurityPolicy configurations that are similar to those used by the basic WS-Security. A big difference is that when WS-SecureConversation is used, the policy must cover two separate exchanges that between the client and the STS, and that between the client and the actual service. This is handled in the policy description by using a nested policy for the exchange with the STS, while the main body of the policy applies to the exchange between the client and the service. Policy configuration of Rent-a-Car STS sample is represented in two different style as included in the WSDL of service and as external policy attachment. The policy attachment technique gives the ability to store the policies in an external attachment without touching the WSDL of the service. The policy attachment usage is represented in CRMservice and files: Endpoint configuration is represented in: <Talend.runtime.dir>/examples/talend/tesb/renta-car/crmservice/client-sts/src/main/filtered-resources/META-INF/spring/ beans.xml External policy attachment is represented in: <Talend.runtime.dir>/examples/talend/tesb/ rent-a-car/crmservice/client-sts/src/main/resources/saml.policy For additional information about policy attachment, read: http://www.w3.org/Submission/WS-PolicyAttachment/ and http://cxf.apache.org/docs/how-it-works.html The other way to store policy is policy inclusion into WSDL, and it is represented in ReservationService WSDL file: <Talend.runtime.dir>/examples/talend/tesb/rent-a-car/reservationservice/ common/src/main/resources/model/ReservationService-sts.wsdl For additional information about policies using in CXF, please read: http://cxf.apache.org/docs/wssecuritypolicy.html and http://docs.oasis-open.org/ws-sx/ws-securitypolicy/200702/ws-securitypolicy-1.2-specos.html.
81
The sample service keystore and configuration for the Reservation service is located in reservationservice/service-endpoint-sts/src/main/resources. It contains two files: servicestore.jks and serviceKeystore.properties. For additional information for keystore creation and usage read Java keytool documentation and OpenSSL tutorial: http://www.openssl.org/ and the Talend ESB STS User Guide.
82
Chapter structure
For information on the Talend Artifact Repository, see Talend Enterprise ESB Installation Guide. For information on the Talend Administration Center, see Talend Enterprise ESB Installation Guide and Talend Administration Center User Guide.
84
The Talend Artifact Repository is now running on http://localhost:8082/archiva/ with user: tadmin pwd: tadmin.
9.5. Uploading the demo to a local Archiva repository using Maven files
The Maven project files can be used to deploy Rent-a-Car example into an Archiva repository.
1.
In both your Maven <Talend.runtime.dir>/examples/talend/tesb/rent-a-car/ settings.xml file and your $user.dir/.m2/settings.xml file add the section: <servers> .... <server> <id>archiva.repo-release </id> <username>tadmin </username> <password>tadmin </password> </server> <server> <id>archiva.repo-snapshot </id> <username>tadmin </username> <password>tadmin </password> </server> .... </servers>
2.
Also check that the <Talend.runtime.dir>/examples/talend/tesb/rent-a-car/ pom.xml file contains sections with the correct Archiva link: <distributionManagement> <repository> <id>archiva.repo-release</id> <name>Internal Release Repository</name> <url>dav:http://localhost:8082/archiva/repository/repo-release</url> </repository>
85
Deploying the demo manually in Talend ESB Standard Edition with Nexus
<snapshotRepository> <id>archiva.repo-snapshot</id> <name>Internal Snapshot Repository</name> <url>dav:http://localhost:8082/archiva/repository/repo-snapshot </url> </snapshotRepository> </distributionManagement> 3. Using Apache Maven with either Linux or Windows, run the following command from the directory <Talend.runtime.dir>/examples/talend/tesb/rent-a-car/: mvn deploy Note: this step assumes that you have built the demo previously. If not, then please run mvn install first. See Section 5.1, Overview of Rent-a-Car demo example for the beginning of instructions on running the demo. 4. Check on the link http://localhost:8082/archiva/browse, that the Rent-a-Car sample: org.talend.esb.examples.rent-a-car/ is successfully deployed.
9.6. Deploying the demo manually in Talend ESB Standard Edition with Nexus
It is possible to deploy the demo manually in Talend ESB Standard Edition with a Maven Repository Manager. Here we use Sonatype Nexus, but you can use Apache Archiva to deploy if you use the pom definitions as shown
86
Deploying the demo manually in Talend ESB Standard Edition with Nexus
in Section 9.5, Uploading the demo to a local Archiva repository using Maven files. Nexus acts as a sort of shared server of Maven artifacts repositories. Download and learn more about Nexus at: http://nexus.sonatype.org/download-nexus.html For more information on the syntax below, please see Talend ESB Infrastructure Services Configuration Guide and http://archiva.apache.org/. 1. 2. Deploy nexus-webapp.war into Tomcat, and start Tomcat. In your Maven settings.xml file add the section: <server> <id>nexus</id> <username>deployment</username> <password>deployment123</password> </server>
3.
In the Maven project file <Talend.runtime.dir>/examples/talend/tesb/rent-a-car/ pom.xml, change <distributionManagement> section to: <distributionManagement> <!-- use the following if you're not using a snapshot version. --> <repository> <id>nexus</id> <name>RepositoryProxy</name> <url> http://localhost:8080/nexus-webapp/content/repositories/releases </url> </repository> <!-- use the following if you ARE using a snapshot version. --> <snapshotRepository> <id>nexus</id> <name>RepositoryProxy</name> <url> http://localhost:8080/nexus-webapp/content/repositories/snapshots </url> </snapshotRepository> </distributionManagement>
4.
5.
Run the following command from directory <Talend.runtime.dir>/examples/talend/tesb/ rent-a-car/ : mvn deploy
6.
the
Rent-a-Car
sample:
87
7.
Now you can install rent-a-car feature with command: features:addurl xml and work with it as usual. mvn:org.talend.esb.examples.rent-a-car/features/5.1.1/
9.7. Preparing Talend Administration Center to deploy the demo to Talend Runtime
Talend Administration Center is available only for Talend Enterprise ESB. For more information on installing and using the Talend Administration Center, including software prerequisites, see Talend Enterprise ESB Installation Guide. For more detailed descriptions of the ESB Conductor feature, see Talend Administration Center User Guide. The Talend Administration Center is a web-based administration application for all aspects from collaborative work and the related code repository management up to the remote deployment of production data services and routes. The Talend Administration Center uses the Talend Artifact Repository to store and to provide the deployment artifacts for the Talend Runtime container, and interoperates with it. This section summarises the installation of the Talend Administration Center, and the Talend Artifact Repository, and how to use them in conjunction with the Rent-a-Car demo. 1. In order to run this example, please follow the instructions in the Talend Enterprise ESB Installation Guide to install and run the following components in the normal order. Note: these are the minimum components needed, and are sufficient for the examples here, but you may install more components if you wish. (In particular, the Commandline component is not needed). So, here is the list of components you need: SVN Server, to store all your project data. (Note: installation and usage of SVN is only required if you like to use a central code repository. It is not mandatory for the examples here and also not required for the next steps). Tomcat application server Talend Administration Center Web application (effectively a war file deployed in Tomcat) Talend Artifact Repository (effectively a zip file containing Archiva) 2. As a quick checklist, after the installation: Talend Runtime and Tomcat are running. The Talend Administration Center is deployed in Tomcat and is running on http://localhost:8080/ org.talend.administrator, and your license is installed. You should also have created the new user in Talend Administration Center, with appropriate roles, and you need to be re-logged in as the new user in order to see the ESB pages (for more information, see Talend Administration Center User Guide). The Talend Artifact Repository (Archiva) is started by extracting the ZIP file in the Talend Administration Center package and running archiva.bat console (Windows) archiva.sh console (Linux). It is running
88
on http://localhost:8082/archiva/. (It includes extra pre-configured repositories compared with standard Archiva). 3. Please ensure that a Talend Runtime container is running (Section 3.6, Starting a Talend Runtime container). Now, on the Servers page (under Job Conductor on the left hand side), create a new server corresponding to this Talend Runtime container: select Add and specify the following parameters: Host: localhost Command port: 8000 File transfer port: 8001 Monitoring port: 8888 Username: tadmin Password: tadmin Select the Talend Runtime check box. And click Save.
9.8. Deploying the demo with Talend Administration Center to Talend Runtime
At this point, we assume that the demo has been uploaded to the Talend Artifact Repository either manually (see Section 9.5, Uploading the demo to a local Archiva repository using Maven files) or from Talend Enterprise ESB Studio, using a menu option (see Talend Enterprise Studio User Guide). Now we deploy the demo with the Talend Administration Center (for more details on the steps here, see Talend Administration Center User Guide). 1. 2. 3. In the Talend Administration Center, from the menu, select Job Conductor, and then ESB Conductor. Then create a new deployment task. Click the Add button and fill all the fields in the right panel of screen: In the Metadata section, fill in the label ("rent-a-car") and description ("demo")
89
4.
Then click on "Select Feature", and fill in the options like these:
Click Ok. In the runtime section, type in talend-runtime for the server name. Leave the rest of the parameters at their default values, as shown below:
5.
The deployment task appears in the list. Now, click on 'Deploy' and the feature will be deployed to the selected Server (Talend OSGi container) and will be started automatically.
The services (the OSGi Feature: tesb-rac-services) are now deployed on the talend-runtime Server and they are automatically started and can now be used. If you type list at the console of the container, you will see that the services are active. You can now, for example, run the Rent-a-Car client application, which uses the services. See Section 5.1, Overview of Rent-a-Car demo example for more details. For more details on the ESB Conductor see Talend Administration Center User Guide.
90
Glossary
This glossary explains some of the terms in use in this manual. Artifacts Generated classes, interfaces and other files, which support communication between pairs of Java client and service endpoints. They can also be stored (and retrieved) using the Talend Artifact Repository, or Apache Archiva. See Java Authentication and Authorization Service. and This is security functionality at a user and application level to provide authentication and authorization services for clients and services. web-based services for application and network management and monitoring. On the application or service side is a JMX MBean instanciation that supplies the information on memory, active threads, logging, and so on. On the client side is jconsole, which is a GUI JMX client displaying the information. See Java Management Extensions. Apache Karaf is lightweight OSGi container onto which various components, clients and services can be dynamically deployed. See Also OSGi. The OSGi technology is a set of specifications that provide a framework so that applications can have components that are dynamically loaded. See Security Assertion Markup Language. Markup is based on XML, and typically uses SOAP over HTTP to transmit information. SAML is an open standard for exchanging authentication and authorization data between between an identity provider (which provides authentication services to a client) and a service provider (which allows and authenticated user to invoke a service for which they are authorized to use). An independent service that provides security functionality between clients and services who don't neccessarily have a trust relationship. A client authenticates itself with the STS based on policies and requirements defined by the STS. The STS then provides a security token (example: a SAML token) that the client then uses to talk to the target service. The service can validate that token to make sure it really came from the trusted STS. is a protocol specification for exchanging structured information (using XML format). The protocol includes the facility to define how messages are encoded and processed. It is structured as a SOAP envelope, which encloses a header and message body. It may be used with a variety of transport protocols, for example, over HTTPS, making it ideal to be used for security purposes. See Security Token Service. A web interface to a service, providing information to client about how to interact with the service. It provides information such as port number and operation signatures (independent of implementation). See Web Services Description Language WSDL.
JMX Karaf
SOAP