SAP BW & Apache Airflow: More flexibility through bilateral system integration
The Apache Airflow workflow management platform orchestrates a large number of systems in the corporate context. Many interfaces can be conveniently controlled via operators in the workflow, only the connection to SAP systems cannot be realized without further ado.
SAP Business Warehouse (BW), on the other hand, lacks the ability to use open-source libraries and to freely choose the programming language for data manipulation. Through a two-way integration, the obstacles are overcome and the strengths of both systems can be combined in operational use cases. Thus the organization of the data processes gains flexibility regarding development language, linking possibilities and usable libraries.
Linking process chains in SAP BW and workflows in Apache Airflow
The Nextlytics Apache Airflow - BW connector enables plug-and-play integration between the Apache Airflow workflow management platform and SAP BW.
- Endpoint in SAP BW to start process chains externally
- User-defined process chain step that enables starting a workflow in Apache Airflow
- Plug-and-play operator for synchronous and asychronous execution of process chains from Airflow
Functional range of the process chain step im SAP BW
Simple system ID transfer
The trigger process optionally sends the system ID during the API call. In Airflow, the system ID is used to recognize whether the process chain came from the development or the production system.
Transfer of any parameters
Additional transfer parameters can be passed on specifically for an individual workflow run. These can be, for example, file locations, target system abbreviations or temporary table names.
Synchronous and asynchronous execution
After a workflow is started in Apache Airflow, the process chain can either proceed with the execution of subsequent process steps (asynchronous) or the process chain waits for the workflow to complete (synchronous).
Connection testing
The integrated connection test improves error handling. Connection errors due to incorrect access data are thus determined before the first execution.
Success-dependent processing of data
In the case of synchronous execution, the process chain can react differently to the incorrect execution of a workflow in Airflow than in the case of success.
SAP BW process chain
Functionality of the operator in Apache Airflow
Secure authentification via Airflow backend
The connection data for SAP BW is encrypted in the Airflow backend during integration and is therefore not visible in the program code.
Synchronous and asynchronous execution
The workflow in Apache Airflow can also wait for the process chain to complete (synchronous mode) or proceed directly with subsequent steps (asynchronous mode). In synchronous mode, the status check is executed within the trigger task.
Can be easily combined with other operators
The connector can be added to the workflow in a modular way as a user-defined operator. Accordingly, it can be placed at the desired position in any workflow.
Automatic connection management
The management of the connection into the BW is controlled within the operator. During workflow creation, no additional code is required to manage X-CSRF tokens, cookies and the settings of proxies. Only the connection information needs to be stored in the backend.
Airflow workflow
Advantages of the connector
‣ Plug-and-play application
Easy deployment via Airflow Operator and BW endpoint makes installation fast without much configuration effort.
‣ Flexibility through synchronous and asynchronous execution
Multiple use cases can be covered by a synchronous mode.
Installation
Do you have questions or are you interested in the connector?
Please do not hesitate to contact us.
We look forward to hearing from you!
Would you like to know more about SAP Connectors?
Find interesting articles about this topic in our Blog
More flexibility for processes: Connecting SAP BW and Apache Airflow
The integration of systems is an interesting and necessary field that no modern IT landscape can do...
Use Google API and ABAP to enrich SAP data with geoinfos
Nowadays, geodata play an increasingly important role. If you want to exploit the full potential of...