The System of Record in MDM, by Dalton Cervo
Today’s guest article is by Dalton Cervo, the co-author of a great new book titled Master Data Management in Practice – Achieving True Customer MDM.
I’ve been asked this question many times: why is implementing an MDM solution so difficult? The short answer is MDM includes technical and business challenges, and encompasses a set of disciplines that are pervasive to the organization. But in this article, I’d like to focus on one particular aspect: establishing a system of record (SOR).
Let’s first take a look at a typical IT infrastructure in a large organization. Figure 1 depicts a simplified scenario.
The main points I’d like to make here are:
- Each line of business (LOB) has its own data source, which includes transactional and non-transactional (i.e. master data) information.
- The separate data systems typically lead to disparate and fragmented information.
- The need for data sharing soon arises, and organizations start to integrate information either directly (database links, direct data access, and so on) or via an Enterprise Service Bus (ESB) and underlying Service Oriented Architecture (SOA).
I’m sure you noticed my diagram is really simplified. As you probably have experienced in your organization, this becomes a big mess as companies grow and data systems increase in size and complexity.
Also, SOA services typically grow slowly and unevenly. At some point, either IT and/or business realizes the underlying architecture is unsustainable, and decides to implement an MDM hub to avoid data disparity, duplication and fragmentation and improve data quality, data integration, and data distribution.
Figure 2 depicts one of the many ways data can be integrated into an MDM hub. I won’t get into the merits of which use case of MDM is being implemented, such as Analytical, Operational or Enterprise. As I said previously, I want to stay focused on one particular aspect, the System of Record.
Now that master data is in a single place, the many application systems in each LOB can be leveraged to use the hub as the “single source of truth,” as depicted in Figure 3. Needless to say, this is one of many possible configurations.
Unsurprisingly, the above looks simpler than it really is. Let’s talk about some of the challenges.
The data migration effort offers plenty of technical challenges. A data migration project is typically done in phases. Not all systems are migrated at once, and sometimes, even a single system is not migrated entirely in a single phase. Your data hub becomes another data source, and worse, must be kept in sync with the sources still in operation.
But in addition to all technical challenges, we have the business challenge. What data source is the System of Record (SOR)? During the migration itself, that question is important, because if you have disparate information across the many sources being migrated, which one trumps the others?
Each LOB will say their data is better, and will want their information to prevail. That’s when a data governance office is important, and will help solve the many conflicts that are bound to happen.
And a single source is not necessarily the SOR for all data being converted, which increases the challenge.
Even if a SOR is clearly established during data migration, timing may be an issue as well. At some point, you obviously want to make the MDM hub the System of Record. But how do you manage the intermediate phases? An intermediate state may include many sources still operational, and in need of a live interface to maintain data consistency.
Remember, not everything is migrated at once. That’s clearly a challenge that can get quite complex, especially if the interface is bi-directional.
Once migration is completed, rewriting all your existing systems to use the hub can be expensive and unfeasible, especially if you use COTS products. Custom interfaces are sometimes unavoidable.
Here are some tips to remember:
- Start a data governance program early. You don’t need an MDM initiative to have a data governance body. The earlier you start the cooperation between the many LOB’s, the easier it will be to integrate their data later, and establish which system is the SOR at any given moment.
- The above is also true for data quality. Data quality is just one of many MDM disciplines, but can and should exist even if you’re not doing a full MDM yet. The cleaner your existing data sources are, the easier it will be to integrate them later into a single hub. Plus, data with higher quality facilitates the determination of which source is more appropriate to be the SOR.
- Once integration starts, clearly establish which system is the SOR for a given entity and/or set of information. This definition is critical to avoid confusion and facilitate entity resolution. As data is migrated, clearly define when the SOR transition will occur.
- Data maintenance should be done on the SOR only. Avoid bi-directional interfaces. Synchronizing information that is modified in multiple places is a nightmare, and should be eliminated if possible.
- IT and business must work closely together. Establishing a SOR is a combination of technical feasibility along with properly supporting all business processes. Rarely will one side be able to define the SOR without consulting the other side.
An MDM program requires a lot of planning and coordination, and it is not a single project. It is an ongoing program that needs constant collaboration between the many LOBs and IT. Having a clearly established SOR for every single attribute, at all times, will certainly minimize issues.
Dalton Cervo is a Senior Solutions Consultant at DataFlux, helping organizations in the areas of data quality, data integration, and MDM. Prior to DataFlux, Dalton served as the Data Quality Lead for the customer data domain throughout the planning and implementation of Sun Microsystem’s enterprise customer data hub.