The same data element is in different places with different labels causing all kinds of project delivery issues! How do you solve this nasty problem? A common solution a lot of organizations, large and small, resort to is the creation of a Data Platform. A Data Platform is a central integrated technical persistence data store designed to distribute data required by strategic business initiatives. A Data Platform can manage fragmented data across multiple touchpoints, provide the capability to track a data element’s behavior across all channels, and make intelligent data driven decisions based on facts.
The diagram below summarizes the core functions of a Data Platform.
A well-architected Data Platform provides many advantages to an enterprise. Let us dive into some of the key benefits:
Common view of the data
Data from source systems are transformed into a common format (common data model) for easy consumption. Moreover, the common data model enforces a common language providing a common view of data from disparate sources. For example, systems A, B and C can use different labels to refer to a client. This is confusing and requires time consuming analysis before projects can ingest this data for their usage. To eliminate this issue, the common data model in Data Platform defines a single data element for client with a good business definition so all three systems can consistently populate the standardized data element. Now, projects wanting this data have a homogeneous common view of a client.
Reusable data
A Data Platform’s common data model follows modelling best practices to group the data by subject area and in third normal form. This means the data is stored at the most granular format as possible facilitating multitude of downstream projects to consume the data and aggregate it to be fit for purpose. As the Data Platform becomes the single source of truth, enterprises benefit from cost savings and efficient project deliveries.
Data Quality
It is much easier to implement data quality checks in a central Data Platform than it is to implement in disparate source systems that have varying technology stacks and source specific labels that are hard to understand. As the quality of the data is monitored and improved in the Data Platform, enterprises are enabled to make strategic decisions and avoid embarrassing data errors.
Metadata
Business and technical metadata collection is a critical part of Data Platform. Metadata answers fundamental questions such as
-
What data is available?
-
Which systems have the data required?
-
How does the data flow through the enterprise?
-
How to get access to the data?
-
What transformations are applied to the data between source and
common data model?
Metadata to explain the data in the Data Platform is foundational and the most important pillar to build trust in the data. Easily accessible metadata promotes self-serve and trust in data.
Collaboration
A Data Platform promotes collaboration between technical, data management and business teams. The process to collect, standardize and distribute data calls for cross departmental participation for the collective benefit. The technical teams contribute to the development, the data management team upholds the agreed upon data policies and the business teams define the data from a business perspective. This powerful three-way collaboration results in data that is fit purpose for the whole enterprise.
Benefit to producers
Source system teams are often challenged when various downstream teams ask for tailored feeds to be created to satisfy their specific projects. Often, they end up building hundreds of pipelines that become harder to maintain as time goes by. A Data Platform eliminates this challenge by allowing teams to focus their time on day-to-day tasks. The producer systems build the pipeline to the Data Platform allowing any project that needs data to obtain it directly from there.
Benefit to consumers
It is a data driven world and most projects initiated at any enterprise need data to achieve their goals. For instance, Regulatory and Reporting projects need good quality data from all the relevant sources to automate their data feeds. Data Platforms with a common data format speed up the integration and analysis timelines so regulatory projects, for instance, can keep up with the ever-changing requirements without reputational loss or incurring hefty fines.
There are many benefits to creating a Data Platform in the corporate world. Often the Chief Data Office focuses its efforts in building an efficient Data Platform so it can get on with governing the data for the enterprise. There is so much to gain from building a Data Platform to centralize the data; we briefly covered some of them in this article. In the next article, we will get into why Data Platforms do not always succeed.