How to manage decommissioned systems within your data lake
ELSA as a starting point
System decommissioning is typically seen as the end of a system. Instead, we’d like to present this as a new start for the information contained in such a system, free from technical debt.
Most of the time, the systems to be decommissioned are first replaced with a new system. The migration usually only moves the information that was being processed at the time of the cutoff. For example, a purchase order that hasn’t been delivered yet. The problem lies in the historical information that is not migrated, or to be more precise, it is not fully migrated with proof of completeness and non-modification. Such information may have been partially moved to a data lake, but a partial view is irrelevant for tax purposes.
Also, the data migrated to the data lake has been using an ELT process, and the ‘transform’ part may have been relevant for yesterday’s usage, but it might not necessarily be relevant for tomorrow’s usage.
There is a need to get a complete, modification-free, lineage-proven view of the legacy information. ETL prevents us from using a classical data lake process. This article will demonstrate why Entreprise Legacy System Application (ELSA) by TJC is a suitable candidate for complementing your data lake with the legacy golden zone (please refer to part 1 of these articles’ series to read about the different zones). ELSA is the legacy single source of truth.
ELSA is a cloud application for decommissioning SAP and non-SAP Legacy systems. In fact, ELSA can be considered as part of your corporate data lake, as it matches the definition of data lakes (see part 1 of this article series). Let’s see why:
- Solid architecture: based on Cloud Foundry open-source platform, ELSA runs on SAP BTP, with on premise or hyperscaler agnostic storage.
- Logically centralized: ELSA application is a multi-tenant, multi-database approach.
- Highly scalable environment: thanks to the hyperscaler usage and the technology involved in it.
- Filled with different types of analytic data: report, documents, and data from SAP and non-SAP legacy system.
ELSA will be the primary go-to destination for your organization’s legacy data. Watch this demo to learn more about it
A repository from many sources
ELSA, as data lakes, is constructed from potentially dozens of sources. The following example shows legacy systems (we call them workspaces) that have been stored in HANA databases (please note other databases, more affordable ones like Postgress or MySQL are also available)
With non-tampered data, fully extracted, your organization is protected from the risk of information becoming a data dump or a data swamp. Legacy applications have highly structured information. Let’s look at an ERP: to manage business processes, the data model needs to be consistent and relevant. This ensures that ‘metadata’ is embedded in the legacy application data model. ELSA has a name for this: the legacy system Data Dictionary.
ELSA’s component-based architecture
ELSA natively supports databases and blob storage (file system in on-premise implementation). You may store data, reports, and documents in many formats. It also connects to many platforms: any ArchiveLink-enabled content server, or NF461 certified platform such as Stratow. ELSA API allows access from SAP S/4HANA systems, and we’d be happy to consider your company’s architecture requirements
ELSA as a replacement for data lake legacy architectures
When considering data warehouse systems being phased out in the near future, such as SAP BW systems, you may consider not migrating all facts and dimensions to new systems, such as Snowflake or SAP Datasphere.
ELSA can be the solution for either storing some reports or keeping the original data from legacy systems.
One single zone: the golden one
There is no need to improve the data quality of a legacy application. It’s already self-sufficient, meaning that the data stored in ELSA already qualifies as golden data. Over time, you may keep adding legacy systems to your ELSA platform. ELSA will be the source of truth for legacy information.
ELSA ease of use for many data lake features
Data lineage is a native feature.
Data retention (deletion) may be managed from ELSA. We will however not focus on deleting information in ELSA, but rather on how to propose a surgical deletion, only relying on a privacy obligation, and directly run from the DPO (Data Privacy Officer) office (not from the IT department). No one wants to massively destroy historical information that may be used for game-changing AI models that your company will need shortly.
Cold information may be kept outside of the main database, and you decide who gets access to static PDFs or actual data with the required granularity.
Impact of ELSA on your data lake
By securing historical information from many legacy applications, ELSA makes sure legacy information does not go away. It provides an easy way of getting metadata, completeness, data lineage, data masking, and data privacy.
ELSA relies on technology that makes legacy information available to modern applications such as Datasphere, Microsoft Power apps, SAP Builder, or even S/4HANA systems. ELSA is cloud application that is SAP Certified as build on SAP BTP, relies on SAP and hyperscaler technology, allowing participation in any modern data fabric pipeline, and will also manage current systems archives.
Making available all legacy information, ELSA enables the decommissioning part of legacy data warehouse systems, making sure you are not left with data that is no longer under compliance management (legacy systems typically generate events to delete data warehouse information when data privacy requests are implemented or deletion request are due).
Final word
As a conclusion, this article explains why system decommissioning should not be seen as the end of systems, but rather as a continuation of the access to information in a cost-effective way. We can state it is efficient because the information is being cleared of technical debt. Additionally, system decommissioning with a legacy application like ELSA can also bring about increased profitability benefits – please refer to the article Where Enterprise Legacy System Application meets AI. Indeed, it’s a great time for protecting future revenues through system decommissioning.
