Anyone who has ever been involved in an SAP BW migration project knows that the larger and more complex the system, the more important it is to be methodical and standardized as much as possible. Migrating from SAP BW to SAP Datasphere also confronts companies with the challenge of switching from their familiar on-premise solution to a public cloud environment. This transformation process requires not only careful planning, but also a clear vision of how to optimally leverage the advantages of the new cloud technology while preserving proven business logic.
SAP BW Bridge is designed for the latter aspect - it acts as a link between the systems and enables the migration of existing models and business logic. In this joint blog post by NextLytics and bluetelligence, we will use a practical example from a project to show how this approach can be successfully implemented. After all, using the bridge naturally means partially foregoing the advantages promised by a greenfield approach in Datasphere. In contrast, as a “faster” migration option, it offers many advantages that are worth considering. The following reasons encourage the use of BW Bridge:
In our example of a real project, all these factors applied. We were dealing with a large, historically grown BW system of an energy supplier with highly specialized models that reflect the specifics of the complex and highly regulated energy market. The project objective was to preserve the existing models, business logic and data flows and to rebuild only the reporting layer in SAP Datasphere. In this context, the use of bluetelligence's Performer Suite also turned out to be a key success factor for an efficient and well-targeted migration.
Before we present the detailed objective of the project, it is important to explain a special feature of BW Bridge in connection with SAP Datasphere. Although queries can be transferred to the BW Bridge during migration, they cannot be executed as in the classic BW system or used as a source for other data targets. Instead, queries are provided in the BW Bridge as metadata. This means that they can be imported and used as a basis for the entity import into SAP Datasphere. This feature plays a central role in our approach.
The following diagram illustrates the project's main objective:
The project defined several key steps that were crucial for the migration from SAP BW to SAP Datasphere.
1. Tool-supported transfer via Shell Conversion:By strategically using entity import, we were able to automatically generate a large number of the required objects and thus significantly reduce the time required for modeling in Datasphere.
The illustration in the diagram is intended to be simplified and focuses on the core aspects of the migration. The complete concept includes additional important components:
For detailed information, please feel free to take a look at our article on the reference architecture SAP Datasphere: Datasphere Reference Architecture - Overview & Outlook
Master-Query Analysis
One of the key challenges of the project was to create the key figure definitions as templates for the master queries – to do this, we had to analyze the structures of a large number of queries from the customer's BW system. Even in the planning phase, it was clear to us that we would need a powerful, tool-supported solution for this. As a partner of bluetelligence, we knew the strengths of the Performer Suite and therefore decided to use a temporary license in the project.
The goal was to create a master query for each composite provider that contains all global and local calculated and restricted key figures as well as all characteristics relevant for reporting. In order to only take over the key figures relevant for reporting, we first analyzed all queries executed in the last 18 months using the System Scout analysis Data Loads and Usages:
This analysis provided us with a list of all relevant queries for each composite provider. We used the Query SetCard Designer to create a central list of the definitions of all global and local calculated and restricted key figures. We created a SetCard in which we collected all the necessary information on the key figures used in the queries:
With the help of the Docu Performer (a tool in the suite), we were then able to export the queries to Excel according to our SetCard template. It was important to deactivate the “One file per documentation” setting so as not to create a separate Excel file for each query. The result was a consolidated overview of the key figures for each composite provider. We used this overview as a template for creating the master queries.
Planning of the migration and data flow analysis
At the beginning of the migration, we developed a structured wave plan that relied on a step-by-step conversion during the project duration, instead of a single big bang at the end. This approach allowed us to iteratively migrate reporting scenarios from SAP BW to SAP Datasphere, test them, and conduct user acceptance tests (UAT) with the departments.
This strategy offered several advantages:
In a first step, the customer split the wave planning at the BW InfoArea level. Based on these specifications, we identified the affected composite providers using the entity lists in the Performer Suite. We created lists of these providers, enriched with additional attributes, and made them available to the customer. This allowed for a precise readjustment of the planning, since not all composite providers from an InfoArea could necessarily be migrated in the same wave due to dependencies.
Our task was to use the BW conversion tool as a shell conversion to migrate the data flows for each composite provider completely and smoothly. Although the conversion tool can determine dependencies itself using a scope analysis, it proved useful to manually spread the objects across several conversion tasks. The automatic scope analysis often returns very extensive object lists, which increases the risk of terminations in the case of complex dependencies. Furthermore, certain dependencies, such as lookups in ABAP transformations, are not automatically recognized.
Our solution was to manually define the scope for each conversion step to achieve better manageability and reduce migration complexity. Based on this approach, we developed the following prioritization for the migration:
The DDIC objects had to be manually adapted and activated in the BW Bridge's BTP ABAP cloud environment. This environment is more restrictive than ABAP Classic and only allows APIs released by SAP. Most existing ABAP developments therefore required significant refactoring.
To create the scope lists, we needed a complete listing of all objects in the data flow, including transformations and DTPs, for each composite provider. We used the data flow analysis in the Performer Suite for this. This enabled us to assign all objects in the data flow to a scenario (in our case, a migration wave).
After that, we were able to select all objects in the entity list of the Performer Suite by selecting the corresponding scenario and output them as a list.
These lists served as work lists for the migration and were enriched with additional information (e.g. transformation used ABAP coding) and the migration status. This method allowed us to manage the objects to be migrated for each wave in a transparent and structured way.
Further possible uses in the project
The Performer Suite proved to be a powerful tool in our migration project. Particularly valuable were the easy ways to quickly create lists of BW objects for various purposes.
During the project, we did of course also encounter challenges and limitations although we had tool support. These were mainly restrictions on the part of SAP, but there were also possibilities that the Performer Suite does not (yet) offer.
Exporting the query set cards for 3.x query versions proved problematic, which meant that manual checks were necessary. For larger query selections, occasional crashes occurred, so that we had to spread the queries belonging to a composite provider across several export jobs. Consolidating the individually documented queries on a sheet required additional effort, which we managed by developing a macro. A more targeted analysis of all global and local key figures of a composite provider, for example as a system scout analysis, would be helpful in the future.
The major roadblock for our master-query approach was the extensive restrictions on entity import in Datasphere, which are listed in detail in an SAP Note (https://me.sap.com/notes/2932647). The import process was not transparent, as it was not possible to see which problems were occurring and requiring manual corrections. Some features prevented the import completely, others were skipped, resulting in incomplete key figure definitions. Especially critical were the restrictions on the supported formula operators. The limitation to only the basic operations (+, -, *, /) makes the entity import for queries almost unusable in practice. Real queries often have complexities that go far beyond these simple operations. In view of these problems, manual creation of the key figures in Datasphere ultimately proved to be more efficient. We focused the entity import on the composite providers, from which we then created the analytic models, and manually created the key figures in the analytics models. We used the lists of query set cards as a basis, which we had created for the master queries.
When creating the object lists for the migration waves, we encountered another challenge in the Performer Suite: the data flow analysis did not allow a direct assignment of the DTPs to the scenarios. We had to iteratively check the InfoProviders from the scenario in the entity view using the parent column to determine which DTPs belong to which InfoProvider and then subsequently assign them to the scenario. A simpler functionality for the holistic recording of all components of a data flow, including DTPs, without the need for workarounds, would have significantly simplified and speeded up the process.
In summary, we can say that the use of Performer Suite in our project provided enormous value in reducing the manual effort and potential sources of error and in speeding up the entire migration process. The advantage of the project license was that we only had to purchase a temporary license for the specific use case. Our intention was not to integrate the Performer Suite into the daily IT routine, but to use it specifically for our purpose. Therefore, no expert knowledge in all facets of the tool was required.
We are aware that we could have possibly gained even more helpful information from the tool or could have continued working directly with the lists in the tool. Often, we fell back on the traditional method that we often see in customers and that we normally warn of: exporting to Excel, followed by creative further processing of the exported data. But sometimes you just have to be pragmatic! If it was easier for the project participants, who did not all have access to the tool, to work with object lists in the familiar environment of Excel, then that was absolutely fine with us.
In addition to the features already mentioned with regard to key figure and data flow analysis, we still see potential for future enhancements that could further increase the benefits of the Performer Suite in such projects:
Do you have questions about SAP Datasphere or SAP BW Bridge? Are you trying to build up the necessary know-how in your department or do you need support with a specific issue? Please do not hesitate to contact us. We look forward to exchanging ideas with you!