SAP Data Services on Demand is a data-migration tool based on the on-premise SAP Data Services 4.2 application. It is a cloud solution used to migrate data from on-premise systems to SAP cloud applications, such as SAP Integrated Business Planning and SuccessFactors, just to name a few.
SAP Data Services on Demand gives users extraction, transformation, and loading (ETL) capabilities to migrate data from on-premise systems, such as SAP ERP Central Component (ECC) and SAP BW systems, non-SAP systems such as Oracle, DB2, and SQL Server, and flat files to the cloud. To use the SAP Data Services on Demand tool, companies have to have the SAP Data Services on Demand agent software installed on their on-premise systems. This agent acts as a connector to this web-based tool and does the tunneling work. It also has the following capabilities:
- It is a bi-directional process and it offers data integration. The data flow can be bi-directional in nature (i.e., not only can data be migrated from on premise to the cloud, but data can be pulled from the cloud and pushed on premise on an as-needed basis).
- It offers graphical flows and mappings.
- It has centralized monitoring and administration capabilities.
- It has pre-built adapters built in.
- It offers lightweight installations.
The following prerequisites are necessary to access and work on SAP Data Services on Demand.
- Prior knowledge about data services data integration.
- Access to the SAP Data Services on Demand web link, which allows users to work on SAP Data Services on Demand.
The Architecture of SAP Data Services on Demand
Figure 1 shows the architecture diagram for the SAP Data Services on Demand landscape. As represented in the diagram, the company landscape and the SAP landscape are separated by a firewall. All communication between the two landscapes is done via HTTPS protocol.
The SAP Data Services on Demand landscape architecture
To use SAP Data Services on Demand, you do not need a VPN, reverse proxy, or any other firewall exception. The data is streamed from the on-premise source to the cloud target securely via HTTPS and is never persisted, which removes the possibility of any data leakage.
The SAP Data Services on Demand agent communicates with the SAP applications via Remote Function Calls, which are encryptable via a secure network connection (SNC).
As pointed out earlier, the data flow is bi-directional in nature. SAP Data Services on Demand Agent uses long polling (e.g., it places requests to the server and waits for a response when the task is ready to execute). The communication direction is always from the SAP Data Services on Demand agent to the cloud.
How to Install and Configure SAP Data Services on Demand
Before you use SAP Data Services on Demand for migration activities, there are two steps that you need to take:
- Create an organization
Install and configure the SAP Data Services on Demand agent
Create an Organization
To use the SAP Data Services on Demand services you first need to have an organization created for your use. Without an organization name assigned, you are not able to log on to the SAP Data Services on Demand services. To create an organization, you need to request one from the SAP HANA Cloud Integration operations team. Once the organization is created, an administrator is created for that organization who will do the administration jobs, such as user creation and monitoring.
SAP has multiple data centers, organized by region; therefore, the SAP Data Services on Demand URL is data-center specific. For example: https://<Data center Specific Address>/DSoD/session/logon.
The administrator logs on to the region-specific URL, and the screen shown in Figure 2 opens and asks for the name of the organization. Enter the name of the organization and click the Log On button.
Log on to the region-specific data center
This opens the screen in Figure 3, where you’re prompted to enter the user ID and password. Enter them and click the Log On button again.
Enter log-on credentials
Next, you need to finish the installation and configuration of the SAP Data Services on Demand agent on the client premises.
Install and Configure the SAP Data Services on Demand Agent
In the next screen that opens (Figure 4), select the Agents tab at the top of the screen. This is because you need to install and configure the agent first. Click the Download Agent Package link and it directs you to the SAP Service Market Place form (not shown) where you can download the appropriate agent, depending on your operating system (the SAP Data Services on Demand installation package is available both for Windows and Linux).
Download the agent package for SAP Data Services on Demand
The next step is to create a new agent instance. Click the New Agent link on the right side of the page (Figure 4). Once you click the New Agent link the screen in Figure 5 opens. Here you give the new agent instance a name and a description and assign it to a group (either new or existing). By default the group name is the organization name. You can create a new group if you wish. When you’ve completed your entries, click the Next button.
Create a name and description for the new agent, and assign it to a group
In the next screen that opens (Figure 6) you can see two steps. You have already completed Step 1 — downloading the agent. Now you have to finish Step 2 — download the configuration file to the local server system. Download the file now and keep it—this configuration file is needed later.
Download the configuration file and store it to use later
Next, you have to install the SAP Data Services on Demand agent you downloaded earlier from the SAP Service Market Place.
It is recommended by SAP that the SAP Data Services on Demand agent should be installed in a separate machine that doesn’t contain any other SAP application. As it is a lightweight program and does not use system resources, in our experience we have found that is OK to install it on a shared system. However, it is very important to note here that the agent should not be installed on a system on which another instance of SAP Data Services is already installed. This is because both the agent and SAP Data Services use the same ports and other resources, and they can interfere in each other’s activities.
If you are installing the agent on a Windows machine, simply run the .exe file you downloaded earlier from the SAP Service Market Place. (Conversely, if you’re using a Unix or Linux operating system, download the appropriate version to use.) During the .exe run the first screen that opens is shown in Figure 7.
Log on to SAP HANA Cloud Integration to configure the Data Services agent
In the Administrator user name field, enter the system administrator’s user ID (on which the agent has been installed), and enter the Administrator password (which is the system administrator’s password). You also need to provide the configuration file path (which you downloaded earlier). When the agent is installed for the first time, instead of the Re-specify the agent configuration file option, you see a field where you can select a file to upload. Once you browse to the directory where you have saved the configuration file and selected it, the Upload button becomes active and you can use it to upload the appropriate file.
Although the communication is via HTTPS protocol, if the company’s landscape is very secure and needs a proxy, you can give the Proxy host and port during configuration, so that all the communication is done via proxy. If the organization is using a proxy server for secure connection then you need to give the credentials of the proxy server to authenticate the agent to communicate.
After you’ve made your entries to the screen in Figure 7, click the Configure Directories option on the left. This opens the screen in Figure 8 where you define and add the directories that will be accessible to SAP Data Services on Demand during run time. Any text files kept in these white-listed directories are readable. You can also create files in these directories when doing a write-back from SAP cloud applications. To add white-listed directories, click the Add button and a pop-up screen opens where you can give the path of the directory. Once you close the pop-up screen you can see the directory you just added.
Register the directories that are accessible to SAP HANA Cloud Integration
Next you want to configure the adapters. Click the Configure Adapters on the left and the screen in Figure 9 opens. As of now, there are two adapters available in the Adapter Type field — SuccessFactors (shown) and another drop-down option, not shown, OData. You can change the configuration of these respective adapters. If you make any changes in this screen, click the Save button; otherwise, just click the Exit button.
Configure the adapters
Select the Configure PGP (Pretty Good Privacy) option on the left, and the screen in Figure 10 opens. Here you leave the entries as they default as, in this case, you are not using a PGP environment. After you’ve checked the entries, click the Exit button.
Check the configuration
Once the configuration is complete, log on to the URL used earlier (https://<Data center Specific Address>/DSoD/session/logon) and, in the screen that opens (Figure 11), check to make sure your agent is properly configured.
Log on to SAP HANA Cloud Integration and check the agent’s configuration
If, under the Status column you see a green square then the agent is properly configured and ready for the job. If the status is a red triangle then the agent has not been properly configured or the agent is down and you have to repeat the above steps.
An Overview of the Functionalities of SAP Data Services on Demand
Once the installation is complete, you are now ready to work on SAP Data Services on Demand.
First, let’s review the different tabs available on the tool. The first tab is GET STARTED (Figure 12). This tab explains the flow of development and execution of the jobs. It also contains help documentation for the tool. If at any point in time developers need any reference for development they can go here to find more information.
The GET STARTED tab details
The next tab is the DASHBOARD tab (shown in Figure 13). It gives a comprehensive picture of the number of jobs that were executed during a particular time period. You can change the time period parameters on the left in the Time Period field.
The DASHBOARD tab details
Next is the PROJECTS tab (shown in Figure 14). This is one of the most important areas in the tool. All the projects and the jobs are created in this tab. Also, you promote the jobs from development to production here.
The PROJECTS tab details
The DATASTORES tab is where data stores are created (shown in Figure 15). Data stores are the logical connections to the source and the targets to the Data Services tool. The data transform is the same in the DATASTORES tab as in the PROJECTS tab of the tool.
The DATASTORES tab details
The AGENTS tab (shown in Figure 16) contains the information about all the SAP HANA Cloud Integration agents that are installed and registered from the company landscape. Also, as mentioned earlier, from this tab you get the link to download the agent and the configuration file.
The AGENTS tab details
Under the ADMINISTRATION tab (shown in Figure 17) you find the functionalities that are used by the administrator. Here the administrator can do administrative activities such as user creation and deletion. The monitoring activities can be done here as well.
The ADMINISTRATION tab details
And, finally, the SETTINGS tab (shown in Figure 18) contains the user’s information (name, email, and so on) and his/her settings, such as language.
The SETTINGS tab details