Lab: Qlik Replicate for

SAP to MS SQL
TABLE OF CONTENTS
Overview ..................................................................................................................................................... 3
Introduction ............................................................................................................................................... 3
Creating Replicate Task ............................................................................................................................. 4
Managing Endpoint Connections .............................................................................................................. 6
Creating Source Connections 6

Creating Target Connections ..................................................................................................................... 9

Assign Endpoint Connections .................................................................................................................. 10
Managing Target Database ...................................................................................................................... 12
Managing SAP Extractors ......................................................................................................................... 13
Managing Replicate Global Transformation ........................................................................................... 15
Managing Data Extraction........................................................................................................................ 17
Advanced Replicate Information ............................................................................................................. 20
Bidirectional 20
Use Case Example 20
Setting up Bidirectional 20
Rules to adhere to 20
The following issues are indicative of incorrectly configured Tasks 21
Using Bi-Directional to Ignore Transactions 21
Downside 21
Overview
This Document covers the process involved in configuring Qlik Replicate to extract data out of SAP
Enterprise Resource Planning (ERP) System and store it into Microsoft SQL Database. The areas covered
include:

o Creating Replicate Tasks

o Configuring Endpoints (Source and Target)

o Activating SAP Extractors

o Managing the data extraction process

Introduction
Before data can be sourced into Qlik’s Data Warehouse Solution, you must define a Replicate Task to
manage Source data change-data/capture from Source System, into the Database Landing Zone. A
Replicate task can only manage one source and one target system, but within a project, multiple Replicate
Tasks can be used – one for each Source System.

In this Lab, you will define and configure a Qlik Replicate task, which will extract data from SAP ERP System,
by way of SAP Extractors, loading them into Microsoft SQL Database. Below are the steps to accomplish this
task.

SAP to MS SQL | Overview 3

Creating Replicate Task
Steps
1. Ensure that the URL for the Replicate Server is available, and access granted.
- This will be provided by your Systems Administrator.

2. Open browser** and enter the URL of the Replication Server.

- This will prompt you to log into Replicate.

3. Enter your Username and Password as provided by provided by your System Administrator.
- This will take you to the Replicate Console.

4. Select +New Task to create a New Task.

The following window will appear:

SAP to MS SQL | CreatingWorking

Replicatewith
Task SAP Data in Qlik Replicate Environment 4
4
 5. Enter a meaningful Task Name.

6. Select Unidirectional radio button.

- This indicates data flows from Source to Target – with no writebacks to source system.
Bidirectional - Choose to synchronize records between two Endpoints (referred to as Endpoint A and Endpoint
B), ensuring that both endpoints contain identical records. The endpoints can either be the same type (ex.
Oracle-to-Oracle) or different types (ex. Microsoft SQL Server-to- Oracle). To implement bidirectional replication,
two bidirectional Replication tasks need to be defined: one that captures changes made to Endpoint A and
replicates them to Endpoint B (Task 1), and another that captures changes made to Endpoint B and replicates
them to Endpoint A (Task 2). See details

- Log Stream - Log Stream Staging enables a dedicated Replicate task to save data changes from the transaction
log of a single source database and apply them to multiple targets, without the overhead of reading the logs for
each target separately.

7. Ensure that Full Load and Apply Changes are shaded.

- Full Load: Select to enable or disable Full Load options for this task. When full load is enabled, Qlik
Replicate loads the initial source data to the target endpoint. By default, a full load is carried out for this task. If
you want to change this setting after you begin working with this task, you make the change in the Task
Settings.

- Apply Changes: Select to enable or disable Apply Changes (Change Processing). When this option is
enabled, Qlik Replicate processes the changes. By default, change processing is carried out for this task. You can
view the change processing in the Monitor view. If you want to change this setting after you begin working with
this task, you make the change in the Task.

8. Select OK.
- This closes the New Task dialog box.
The Replicate Task is now ready for the creation of Endpoints.

SAP to MS SQL | CreatingWorking

Replicatewith
Task SAP Data in Qlik Replicate Environment 5
5
Managing Endpoint Connections
The Replicate task will use the connections you are about to create to perform a full loading of data, followed by
incremental log-based data replication from your source system to your target – managing your data warehouse ODS
layer which will subsequently be used to populate your data warehouse.

Creating Source Connections

Qlik Replicate requires information to connect to the source and target endpoints that you want to use in a
task. In addition to providing a complete list of the source and target endpoints supported by Qlik Replicate,
there is also the ability to create new Endpoints.

Qlik Replicate delivers a number of Endpoints types for various Source and Target Systems.

Steps
1. In the Replication Console, Select Manage Endpoint Connections.

The option to create or use existing Endpoints will be displayed.

Working
SAP to MS SQL | Managing with
Endpoint SAP Data in Qlik Replicate Environment
Connections 6
6
 2. To create Endpoints, Select + New Endpoint Connections.

2a. To use existing Endpoints, go to Assign Endpoint Connections.

3. Enter a meaningful Endpoint Name and Description for the Endpoint Connector.

4. With Source button selected, Select dropdown arrow to select appropriate Source.

5. Enter Source Server information as provided by System/Database Administrator.

- Note: In this example, SAP Extractor is the Source of the data

Name:

Description:

Role: Source

Type: SAP Extractor

Connection Mode: Custom Application Server

Server Name:

Instance Identifier: 0

Client identifier:

Username:

Password:

Number Format:

Working
SAP to MS SQL | Managing with
Endpoint SAP Data in Qlik Replicate Environment
Connections 7
7
 6. Select Test Connection.

Look for the “Test Connection succeeded” message. Any other message means something may be
incorrect with your Server/Database definitions, or the Server/Database is unavailable.

7. Select Save.

Working
SAP to MS SQL | Managing with
Endpoint SAP Data in Qlik Replicate Environment
Connections 8
8
Creating Target Connections
Steps
1. Select New Endpoint Connection.

2. Enter a meaningful Endpoint Name and Description for the Endpoint Connector.

3. Select Target button selected in Role.

4. Select dropdown arrow in type and select Microsoft SQL Server.

5. Enter the following Target Server credentials as provided by System/Database Administrator.

Name:

Description:

Role: Target

Type: Microsoft SQL Server

Server:
• SQL Server Authentication

Username:

Password:

Database name:

6. Select Test Connection.

Again, look for the “Test Connection succeeded” message. Any other message means something may be
incorrect with your Server/Database definitions, or the Server/Database is unavailable.

7. Select Save.

SAP to MS SQL | CreatingWorking with SAP Data in Qlik Replicate Environment

Target Connections 9
9
Assign Endpoint Connections
At this point the Source and Target Endpoints created above will be assigned, in order for Replicate to extract and
load the data as defined.

Steps
1. On the left of the Replicate Console panel, Select Source.

2. Locate the Source Endpoint created above or one which meets your Source definitions.

3. Drag and drop to Source Endpoint on the right, as indicated in diagram.

4. On the left of the Replicate Console panel, Select Target.

5. Locate the Target Endpoint created above or one which meets your Target definitions.

6. Drag and drop to Target Endpoint on the right, as indicated in diagram.

7. Select Save.

WorkingConnections
SAP to MS SQL | Assign Endpoint with SAP Data in Qlik Replicate Environment 10
10
 Here is a “before” and “after” Endpoint assignments:

WorkingConnections
SAP to MS SQL | Assign Endpoint with SAP Data in Qlik Replicate Environment 11
11
Managing Target Database
Steps

1. Log into the target Microsoft SQL Database via Microsoft SQL Server Management Studio.

2. Ensure that login credentials are the same as that defined in the Target Endpoint.

3. Select Connect.

4. Identify the Database where the data will be stored.

- In our example above database demoDB was used:

5. Check whether Database Schema is already created.

6. Create Schema, if not already created.

Working
SAP to MS SQL | Managing with SAP Data in Qlik Replicate Environment
Target Database 12
12
Managing SAP Extractors
Steps
1. Log into the SAP Server as defined in Source Endpoint.

2. Run Transaction Code RSA5 (SAP Transaction codes are not case-sensitive).

- The DataSource Business Content listing will appear.

These are Extractors delivered by SAP as part of the SAP ERP/S4HANA implementation.

3. Expand the DataSource list as follows:

4. Select the DataSource and Select Activate DataSource.

Note: A DataSource only needs to be Activated only once, within this Transaction Code*1.
You can only activate one DataSource at a time, so repeat this step until all the required DataSources are
activated.

1The Activation process is a one-time process. Extractors stay active once activation takes place unless enhancements are done. Reactivation is
needed in this case.

Working
SAP to MS SQL | Managing with SAP Data in Qlik Replicate Environment
SAP Extractors 13
13
 5. Enter Transaction Code /n/qtqvc/extrep.

6. Select Activate Extractors.

7. Enter Target Logical System Name and the DataSource to be activated.

Repeat entering the DataSources and activate, until all required DataSources are activated. These objects
will now be available in Replicate Table.

Working
SAP to MS SQL | Managing with SAP Data in Qlik Replicate Environment
SAP Extractors 14
14
Managing Replicate Global Transformation
Steps
1. In Replicate Console, select Global Transformation.

2. Select New Global Transformation.

3. Enter meaningful Transformation description.

4. Select Rename Schema button.

5. Select Next.

6. Select Next.

7. Select Rename Schema button.

8. Enter the name of the Database Landing Area Schema.

Working
SAP to MS SQL | Managing with
Replicate SAPTransformation
Global Data in Qlik Replicate Environment 15
15
 9. Select Finish.

10. Select OK.

11. Select Task Setting.

12. Enter Table Target Schema.

13. Select OK.

14. Select Save.

Working
SAP to MS SQL | Managing with
Replicate SAPTransformation
Global Data in Qlik Replicate Environment 16
16
Managing Data Extraction
Replicate is ready to extract the data into SQL Database.

Steps
1. In the Qlik Replicate Console, Select Table Selection.

2. Select Search.
A list of available files/tables will appear.

3. Select Files/tables to be transfer into Replicate the Qlik Replicate Console.

Working
SAP to MS SQL | Managing with SAP Data in Qlik Replicate Environment
Data Extraction 17
17
 4. Select > icon to transfer data to the Selected Tables column.

5. Select OK.

6. Select down arrow beside Run.

7. Select Reload Target.

- If this is the first time this Task is being used to extract data, Start Processing must be used – Reload
Target will not be an option.

Working
SAP to MS SQL | Managing with SAP Data in Qlik Replicate Environment
Data Extraction 18
18
 8. Select Yes to reload Data Target.

Replicate switches to Monitor mode to allow for monitoring of Data Loads.

Once data load is completed, log into the Target Microsoft SQL Database and validate data.

Working
SAP to MS SQL | Managing with SAP Data in Qlik Replicate Environment
Data Extraction 19
19
Advanced Replicate Information
Bidirectional

Note: Bidirectional Replication Profile is limited to certain RDBMS (more of an Oracle EBS

or dynamics system in the Microsoft Environment. SAP Environment will not be support).

Use Case Example

I have 2 systems with same schema. System A has a subset of data and System B a completely different subset. (e.g., US
versus European) and you need all data in both places, but System A will never update / change data from System B and
vice-versa.

Setting up Bidirectional
When setting up bi-directional Replication - be aware that schema names, table names and column names are very
case-sensitive.

For Example: if replicating in Oracle and SQL Server Environments

Oracle table - "MYSCHEMA"."TEST"
SQL Server table - [myschema].[TEST]

In Task settings

TASK 1 - Oracle to SQL

Metadata Tab - Target Metadata --> myschema
Bidirectional Tab -
Source Schema --> MYSCHEMA
Target Schema --> myschema

Task 2 - SQL to Oracle

Metadata Tab- Target Metadata --> MYSCHEMA
Bidirectional Tab-
Source Schema --> myschema
Target Schema --> MYSCHEMA
If column names use mixed cases, ensure these are matched in "Table Settings" and the "Transform Tab".

Rules to adhere to
1. Schemas should be specified exactly as they are in the database with the correct case-sensitivity.
2. The Bidirectional Source Schema should be equal to the Target Schema in the second Task,
and the case-sensitivity should be correct.
3. If the metadata is different (field names), they should be mapped correctly taking into account the
"correct" case-sensitivity of both databases.

Working
SAP to MS SQL | Advanced with
Replicate SAP Data in Qlik Replicate Environment
Information 20
20
The following issues are indicative of incorrectly configured Tasks
1. If you have a Primary and you get a warning about the PK.
2. If the loopback_prevention table is added to the UI capture list (attrep_loopback_prevention).
3. If you update Oracle and you see that the SQL to Oracle Task's statistics got updated, or the opposite
direction.

Using Bi-Directional to Ignore Transactions

These are directions for how to use the bi-directional task in a one-way mode to ignore transactions.
You must create the attrep_loopback_prevention manually in the source:

CREATE TABLE "attrep_loopback_prevention"

( "task_name" VARCHAR2(128) NOT NULL ENABLE,
"timestamp" TIMESTAMP (0),
CONSTRAINT "HRattrep_loopback_prevention" PRIMARY KEY ("task_name") ENABLE,
SUPPLEMENTAL LOG DATA (PRIMARY KEY) COLUMNS )

You can insert a dummy row into the new table:

insert into "attrep_loopback_prevention" values ('JUNK', sysdate);

When you do an operation, you want to ignore then include an operation against this table:

delete from job_history where employee_id > 999;

update "attrep_loopback_prevention" set "timestamp" = sysdate;
commit;

Create a bi-directional task.

Downside Every transaction applied to the target will update the attrep_loopback_prevention table.

Working
SAP to MS SQL | Advanced with
Replicate SAP Data in Qlik Replicate Environment
Information 21
21
 About Qlik
Qlik’s vision is a data-literate world, where everyone can use data and analytics to improve decision-making and solve their most challenging
problems. Qlik provides an end-to-end, real-time data integration and analytics cloud platform to close the gaps between data, insights and action.
By transforming data into active intelligence, businesses can drive better decisions, improve revenue and profitability, and optimize customer
relationships. Qlik does business in more than 100 countries and serves over 50,000 customers around the world.
qlik.com
© 2021 QlikTech International AB. All rights reserved. All company and/or product names may be trade names, trademarks and/or registered trademarks of the respective owners with which they are
associated. CODE NEEDED FOR PUBLISHING

Working
SAP to MS SQL | Advanced with
Replicate SAP Data in Qlik Replicate Environment
Information 22
22