By Prashanta Chandramohan

Over the last few years, Master Data has been recognized as one of the most important types of business information to be managed. Organizations are heading in the right direction by implementing Master Data Management (MDM) systems to take control of critical data like customers, products, employees, suppliers, materials, locations, etc.

There are different architecture and styles that an organization can adopt while building its MDM solution. Gartner defines Consolidated, Registry, Coexistence and Transaction hubs as the top implementation styles. The hub-and-spoke architecture is a very common approach and provides a single consolidated repository of different master data entities. The main advantage of this architecture is its ability to provide an enterprise-wide, complete and accurate view (often referred as the “360 Degree View” or “Single View of the Truth”) for various master data domains.

Implementing MDM requires in-depth changes to the way organizations work, partly because the technologies adopted here are fairly new, and mainly due to the cultural challenges MDM poses to most organizations. When a company is implementing an MDM hub, by definition, it is building a system that will have a footprint across all the departments and lines of businesses in the organization.

Statistics show that MDM implementations either take a long time to complete or in some cases, yield less than the expected return. While the key reasons are the failure to assess data quality and to establish data governance bodies, maturity of the organizations can also be a significant issue.

Over the past 8 years of providing MDM consultation across different industries, one other aspect I keep stumbling upon is the absence of teams having an end-to-end vision. An important role during MDM inception is that of solution architect – the one who should have the complete picture of the solution involving applications and technologies for profiling, data cleansing, consolidation, enrichment, de-duplication and synchronization which are key ingredients of the MDM recipe.

More than anything else, the solution architect will know the complete information system landscape, what integration mechanisms should be applied, and can estimate the hardware and infrastructure requirements based on the volume of data. Technical constraints and lack of expertise are common pitfalls and knowing about those is a key aspect.

One can’t build a house without a blueprint. The same holds true for MDM implementations. Setting the right foundation and architecting the solution while keeping long-term goals in mind is crucial.

The next section earmarks the five key factors that a solution architect should look into during MDM hub design. A carefully designed architecture covering each of these points helps create a sustainable and scalable solution. This is also a balanced, holistic approach that accelerates the implementation, thus helping to realize MDM benefits more rapidly.

1. Data profiling to understand the current state of data quality
2. Data integration mechanisms to consolidate the data
3. Designing an extensible master data repository
4. Robust data matching & survivorship functionality
5. Seamless synchronization of master data

We’ll elaborate each of these factors and note some of the key takeaways that help to architect a robust MDM solution.

Data profiling to understand current state of data quality

The reason most often cited for implementing MDM is to reduce dirty data in the organization. Cleansing data is a difficult, repetitive and cumbersome process. The politics and the viral nature of data and the silos that exist in the organization can all add to the muddle to make it even more challenging.

Given that, we need someone who can tell us how much of the data is dirty in first place. Data profiling tools do that for you. Not only does data profiling provide extremely useful information about the quality of data, it will also help in discovering the underlying characteristics and discrepancies associated with data.

Position the data profiling tool at the forefront of your MDM solution. The tool you use should be able to generate easy-to-interpret reports for pre- and post- data cleansing activities. As an architect, one has to put in place a well-designed, easy-to-use and effective data profiling tool.

The tool should be placed relative to both initial data migration as well as ongoing data integration tasks. For example, data profiling can provide useful insights about data coming from a specific source at the beginning of the project. The generated report may include information such as the percentage of customers whose birth date is defaulted to 01-01-1901, or Tax Id’s set as 111-111-1111. These errors need to be fixed either in the source or on the way into the MDM repository.

Although profiling is a key aspect, don’t let this become a major hindrance to the overall architecture. So be cautious about the time you take here. The biggest benefit of profiling comes in the form of providing accurate estimates for the data integration effort, which is the next part of our solution.

Data integration mechanisms to consolidate the data

While data profiling helps us determine the rules required, data integration tools actually transform the data. They help in applying numerous rules so that an intermediate clean state of data can be reached.

Historically, data integration tools have been helping business to efficiently and effectively gather, transform and load data during mergers and acquisitions, or during the streamlining of different departments. When it comes to MDM implementations, these tools play a crucial role as they can pretty much “make or break” the establishment of an MDM hub.

A well-architected, sustainable and fast batch ETL solution may be appropriate for integrating the source and target systems with a Master Data hub. Careful consideration should be given to the timely availability of the data by designing the optimal number of staging areas. The architect should know exactly what goes through batch ETL versus real-time data integration. It’s very common to integrate source data via batch processing but as the hub evolves, there will be more need for real-time availability of data.

For real-time data integration, it is better to leverage integration tools based on an Enterprise Service Bus (ESB) and / or Service Oriented Architecture (SOA).

Define clear steps to prepare the data, the processing that needs to happen, and the volume of data flowing though the solution on a daily basis. Design the ETL solution as generically as possible so that new sources can be easily integrated or with minimal changes. If the groundwork is flawed, your future integration tasks will become difficult and time-consuming.

Designing an extensible master data repository

I listed some of the key master data management functionalities on my blog sometime ago. The following points should be closely looked at by the architect while designing the master data tools and technologies:

• A flexible data model is a key feature of MDM product you choose. Depending on the domain implemented, there will be changes to the model. Many vendors provide a pre-defined model but the chances are you will have to modify it to meet specific needs. Choose these modifications carefully, keeping in mind the long term usage of the data elements.
• Multi-domain implementations have become very common recently. Usually, you’ll be implementing at least two or three domains together (customer-location-account, customer-location-product, product-location etc). Design an approach that supports easier management of these domains and the complex relationships between them.
• Security is an important aspect of the solution. As MDM is an enterprise-wide solution, it has to support integration with the existing security registries of the organization.
• Audit functionality is a very important requirement both for capturing historical values of master data as well as regulatory compliances. Design your solution to capture WHO – WHEN – WHY changed the data. Track changes coming from source systems or individual users and build cross references to each connected system.
• Implement maintenance processes to control the quality of data flowing into and out of MDM. Data stewards play a key role here in identifying incorrect data, and for authorization and correction so as to ensure that high quality data is delivered to outbound channels. Your solution should provide interfaces and tools to support the stewards in effectively identifying issues and correcting them easily. This is a challenging task and often depends on the flexibility offered by the underlying hub product.
• The architect should carefully look into the non-functional requirements (NFR) such as scalability, performance, security, maintenance, and upgradability of the product. These NFRs will always co-exist with the functional requirements of the organization to help achieve the desired end state. If the hub is not engineered well to handle increasing workload requirements, it’s not going to live long in the organization.

Robust data matching and survivorship functionality

A powerful and effective fuzzy matching engine is a must-have to eliminate duplicate or redundant data. If you look at the matching capabilities of the MDM tools available in the market, most of them do a great job of identifying duplicate records. However, the tools are not simple to use because of the complex matching criteria used in every implementation.

Although certain commonalities exist among organizations, there are always variations to the rules used to compare data elements.

The first challenge is to determine if there are matches. Figure out the business definition of ‘a match’ in your organization. To define this, you will have to come up with a list of critical elements which are necessary for matching. Next, you have to assign weights to these elements. For example a phone number match is given more weight than a name match or a date of birth match. Many MDM products come with certain pre-defined matching algorithms and your best bet is to start from that and modify it to your specific scenario. Careful consideration of algorithm, testing and sampling are the keys here.

One of the questions I often face is the choice of using deterministic matching versus probabilistic (fuzzy) matching. Believe it or not, there are many organizations that feel more comfortable using deterministic, although it’s not as efficient and realistic as probabilistic matching is.

Choose the right option that fits into the customer requirements. For example: if the customer says “we want to make sure that Tax ID is exactly the same to ensure we have a duplicate record”, you need to design your matching rules to accommodate this.

Data survivorship is another major puzzle. You will have to find the best answers for the following questions when you merge two or more duplicate records:

• What data elements survive when you do the merge?
• How to synchronize this data with the source systems so that everyone producing the data knows about this change.
• How to handle scenarios when two non-duplicates are merged. In other words, the false positive matches which result in a merge. False positives in reality are almost un-acceptable while false negatives, while unfortunate, are acceptable, providing they seldom occur. The solution has to address a way to ‘un-do’ the merges that happen accidentally.

Seamless synchronization of master data

Architects spend enormous time on system integration. No exceptions here. MDM must seamlessly integrate with a variety of applications. Whether it’s the system that’s responsible for data entry, or the data warehouse or the business intelligence system, MDM should bring the right information to the right person at the right time.

Realistically speaking, no MDM solution will replace all the sources of enterprise information overnight. It’s a continuously evolving process. Keeping this in mind, the architecture you design should allow other sources of master data to leverage the features of MDM as soon as possible.

Here are some of the important architectural concerns that need to be nailed down:

• Automation and quick synchronization of data ensuring higher data quality is maintained between transitions.
• When duplicate records are consolidated, find out a way to synchronize this update with the sources while they still exist. Although the hub architecture leads to an eventual single view of master data entities across the organization, this is a long-term affair. To show the value added by MDM, you’ll have to allow two-way communication between MDM and existing sources of master data.
• The frequency at which data flows into the downstream applications like enterprise data warehouse, business intelligence and data marts should be thought through. Although most of these applications need delta changes on at least a daily basis, some may require real-time updates.
• Availability of key data elements (typically via a web service interface) to the business users and data stewards for deeper analysis.
• Knowing which type of data transfer mechanism fits a given data integration point is critical. Notification mechanisms can be asynchronous (most of the time) which helps in completing MDM transactions faster.

There are many instances where solutions get architected poorly and thus add more time and effort in realizing the full potential of MDM. I hope the points discussed here will help you architect a more robust solution.

If you have any questions, please use the “Comments” feature below, or contact me at prashant@mdmgeek.com, on Twitter at @MDMGeek, or read my blog at http://www.mdmgeek.com.

Today, I attended the keynote speech by Mark Kelly. He is a very impressive person. He faced the toughest challenge I can imagine when his wife, former Arizona Congresswoman Gabrielle Giffords, was nearly assassinated.  He was a Navy combat pilot in Iraq, an astronaut on four space shuttle missions, and commanded the final flight of Space Shuttle Endeavour.

His talk opened with an inspiring, optimistic video, and he received a rousing welcome from the crowd. He was very funny too.

Mark Kelly grew up in New Jersey, with a twin brother who is also an astronaut. His father was a tough Irish policeman, and his mother was a waitress and a secretary. In the 1970′s, his mother decided to become a police officer too. After months of practice, she managed to scale a tall wall as part of the fitness requirement for the police department.

Mark’s first job was driving an ambulance in the tough sections of Newark, before going off to college.

His wife was a classic overachiever, becoming a Fulbright scholar and attending Cornell. They met in China on a trip. She ran her family’s company and was in the state senate in Arizona.

Mark decided in college that he would be the first person to walk on the planet Mars. He didn’t make it to Mars but he did make it into space four times.

After setting his goal, he decided he would get there by becoming a Naval aviator in 1986 (the year that the movie “Top Gun” came out). After starting flight school in Pensacola, he realized he was not as good a pilot as Maverick. His first landing on a carrier was about 100 miles offshore. As a 23 year old landing on a ship, it was “beyond scary”. He barely passed his carrier landing test.

Mark believes that how good you are at the beginning of something is not a good indicator of how good you’ll be at the end.  It’s a matter of practice, persistence and the drive to never, ever give up.

Life tends to be a set of challenges and obstacles to climb over. In August 1990, Mark was posted to Iraq, and in January 1991 (at age 25), he had his first combat missions in Iraq.  His airplane has 12 one thousand pound bombs. The first airplane that went out that night got shot down over southern Iraq. As he flew his mission, he could see the anti-aircraft artillery snaking up from the ground. They flew a few hundred miles into Iraq, to Basra, through an area of missiles. A missile came up and exploded right over their plane, and fortunately there were no holes in the plane. Even worse than seeing the first missile was seeing the second one.

On the way back from the mission, he flew 150 miles into Iran to avoid the missile fire. They were mistaken for an enemy fighter as they headed back to the carrier. Mark failed to communicate with the people he was working with. A lack of timely and accurate communication that night nearly cost him his life.

As it turned out, Mark’s wife Gabby had the riskier job serving her country. On January 8 2011, there was no countdown clock to the life-changing event of his wife being shot. Mark was in Houston at the time, visiting with his 15 year old daughter. He heard by cell phone from his wife’s chief of staff that she had been shot in the head.

He borrowed a friend’s airplane to fly to Tucson. His twin brother was in space at the time, on the International Space Station. Halfway there, CNN and Fox News reported that Gabby had died. Six other people died that day, but 30 minutes later, Mark found that Gabby had not in fact died. As he walked into the hospital, his cell phone rang. President Obama was calling to offer his support.

As Gabby started to recover, she suffered from aphasia, difficulty in speaking. This required a lot of patience in waiting for her to form the words.

Right after Gabby was injured, Mark had to make a decision about his own career. The final mission of the space shuttle Endeavor was coming up only three months after Gabby’s injury. About ten days later, Gabby regained consciousness, and was transferred to a rehab facility in Houston. Mark had to consider whether to go ahead with the space shuttle mission, considering what his wife would want. In Congress, she was the chairperson of the Space and Aeronautics Subcommittee. Each flight on the space shuttle was riskier than storming the beaches of Normandy in World War Two.

The power of the human spirit is an incredible thing – seeing Gabby fighting so hard to survive and then to recover, it reminds all of us to deny the possibility of failure.

Gabby went back to Congress to vote on the bill to raise the Federal debt ceiling last summer. It was a controversial vote, and most members of Congress would love to have an excuse not to go vote. Mark had to pack for his wife for that trip, and they flew to DC together. Apparently he forgot some important things.

Eight months after being shot in the head, Gabby was able to have her voice heard in that vote. “One small woman brought Washington to its feet. We can compromise on how we fund America, but we cannot compromise on how we define America. That definition is Gabrielle Giffords.”

Things can change in an instant. It’s important to have gratitude and faith – faith in God, in our family and in our friends.

Gabrielle wanted to send us all a message that Mark read: “Be passionate, be courageous, be strong, be your best.”

It was a very moving speech, very motivating and inspiring. I’m really glad I attended it today.

Please post this on Twitter: Reading @HubDesigns Magazine: ‘OAUG COLLABORATE 12 Keynote Speakers’ at http://wp.me/p5Tdn-NW (please retweet)

I was fortunate enough to be invited to speak during one of the afternoon breakouts.

The event provided some thought leadership and networking opportunities for the industry, and was organized into tracks such as:

• Innovation in Clinical Development
• The Evolving Landscape of Safety and Pharmacovigilance
• Business and Workspace Transformation
• Enabling a Great Customer Experience
• Simplify IT and Ignite Innovation

Oracle’s mission is to enable its customers to deliver innovative therapies, devices, diagnostics and optimized care processes to prevent and cure illness.

Its vision is to build solutions to support a collaborative health sciences industry delivering patient-centric, value-based health care.

Oracle’s strategy is to provide a complete, open, integrated application suite and related services with industry-leading customer ROI and satisfaction.

There is a “perfect storm” in the global life sciences industry: more demanding regulators, investors wanting growth opportunities, and payers that are requiring a better economic cost-benefit ration and improved health outcomes.

There’s a strong trend towards more collaboration, with more connections between healthcare companies, contract research organizations, sponsors, and academia. These silos are starting to be broken down, as more companies realize the benefits of cooperating rather than competing.

Severin Schwan, the CEO of Roche, believes that “data from clinical trials and data from clinical practice are separate, but it makes sense to bring these together”. It’s a brave new world, leading to innovations that truly weren’t possible before.

These things are important, but are they really that difficult? A PriceWaterhouse survey showed that most companies in the health sciences industry believe increased data integration is either challenging or very difficult.

Access to the clinical data stored in electronic health records would help companies across the board.

Oracle is bringing to bear some new IT solutions areas:

• Cloud services, which enable pay-as-you-go, partnering and data sharing
• Mobile, including data collection, workflows and decision support to lower costs and improve decisions
• Big data, the core of secondary use as data sources proliferate

What makes something “big data”? It’s a combination of a certain amount of volume of data, the overall velocity of data, the variety of data, and the value of data.

Big data is a complex set of problems that Oracle is starting to address in a comprehensive way.

Proteus now is making a computer chip on a pill, offering much more fine-grained monitoring than typically available.

Oracle is building out a “Health Sciences Cloud”, offering performance, scalability and security, in a HIPAA certified environment, enabling 21 CFR Part 11 compliance, optimized for individual trials, scaling from the smallest to the largest enterprises.

Social collaboration in the corporate environment is different from Facebook and Twitter. Oracle is developing social network technology to bring the capabilities of the public social networks into the corporate framework. It can be deployed both on premise or in the cloud, and allows for real-time, multimedia conversations.

People have typically been doing their work through private networks and e-mail, but that’s typically less productive in terms of collaboration. The life sciences participants and healthcare provider and payer participants need:

• near real-time clinical data
• protocol validation and patient recruitment
• safety signaling
• device studies
• disease monitoring
• disease and patient registries

But it’s not all going to happen at once.

Gamification is starting to be applied to changing behaviors – patients, investigators, doctors, etc. Oracle is looking at medication adherence, for example.

Oracle’s health sciences roadmap includes a clinical suite, a healthcare suite, and a safety suite, and it also includes a strong analytics component (predictive, real-time and operational), as well as industry-specific high performance hardware.

Oracle’s R&D investment delivered 27 new releases and upgrades in 2011.

Oracle has proof points on Key Performance Indicators like reducing the number of non-performing study sites, improved CRA productivity, improved Clinical Study Manager productivity, reduced time to complete clinical studies, etc.

Oracle has a number of integrations already in place, to tie together its Clinical Trial Management, Clinical Data Analytics, Integrated Response Technology, InForm schedule management and data capture, Argus Safety, and more.

Oracle has been working to integrate its ClearTrial acquisition with its other health sciences products for clinical trial management.

Going back to 2008, Oracle saw an opportunity to invest in order to differentiate itself from others in this space. The innovation accelerated in 2009 at Oracle, with some of its acquisitions in the health sciences software industry. In 2010, Oracle made further investments in improving its products, and in 2011 it came out with more than 25 new releases or major upgrades.

Oracle will continue to invest in integration, big data, gamification, new products, and bring the ClearTrial integration into its suite of products.

Watch HubDesignsMagazine – hopefully I’ll be able to write another post while I’m here today before I speak at 3:00.

I recently conducted a very non-scientific study of the marketing material provided by the major Master Data Management (MDM) vendors.  My hypothesis for this study was that I don’t believe that MDM product vendors know how to effectively promote (through their web sites) the business value of their products to business leaders; the very ones that ultimately must give their blessing (if not actually sign the check) to any project that has as deep an impact on the enterprise as an MDM initiative does.

The Non-Scientific Process

The first task was to set the baseline for what would be considered “business value”.  For this I went to David Loshin’s excellent book, Master Data Management, which has a nice list of MDM benefits to work from.

From page 10 of that book:

“…master data management can be justified in support of a business initiative that relies on any of the following MDM benefits:

• Comprehensive customer knowledge
• Improved customer service
• Consistent reporting
• Improved competitiveness
• Improved risk management
• Improved operational efficiency and reduced costs
• Improved decision-making
• Better spend analysis and planning
• Regulatory compliance
• Increased information quality
• Quicker results
• Improved business productivity
• Simplified application development”

With this list of benefits in hand, I then set out to make a list of the major players in the MDM space.

For this, I referenced the latest Magic Quadrant for Master Data Management from Gartner (for customer data solutions only) to build that list.  [Quick sidebar - I will not be naming names or pointing fingers in this article, for good or bad.]

However, I did not limit myself to just the vendors that made Gartner’s final cut; I also included most of the “Other Vendors” listed on the report.  I then visited each of those vendor’s web sites to print out the data sheets for their products.  And finally, I sat down and read each of the data sheets, looking for stated opportunities for adding/creating business value, which I cross-referenced with my baseline list of business benefits.

The Non-Scientific Results

The Benefits-to-Vendor matrix that I created for this study was very sparsely filled out by the time I was done, which met the expectation of my original hypothesis.  However, an interesting item worth noting was that even when a benefit was mentioned, frequently, the vendors just seemed to be tossing the business benefits in (sometimes using bulleted lists) just to make sure that they had some basic coverage.  There was very little, if any, depth to the discussion of business value.  However, as shallow as those statements might have been, I do have to say that a few of the vendors at least mentioned well over 50% of the benefits on the list.  Unfortunately, just as many mentioned only 2-4 items from the list.

At this point, let’s revisit my original hypothesis, shall we?  The answer to the question of whether they were actually selling business value to the business leaders would, for me, have to be “No”, at least not through their product data sheets.  This, however, sparked the first of many additional questions.  Do business leaders even read product data sheets?  Based on all of the reading that I just did, I’d have to say that the vendors probably don’t believe that business leaders read their product data sheets.  If they did, you’d think that they would have put more effort into having a deeper conversation about business value.

The next question that came to mind was, “Should I have been reading the vendor’s white papers instead?”  I’ve always been led to believe that business leaders read lots of white papers, yet I don’t personally know many (outside of IT) that actually do (please send me a note in the comments below if you are a business leader that reads lots of white papers. I’m curious.)

I went back to the web sites and downloaded their white papers if they had them (about half of the vendors had one or more).  These made for better “business” reading than the product data sheets, but for the most part, they were still too product focused and didn’t get too deep into discussing value propositions.

Who is the Prospect?

So, if the business leaders are not the vendor’s primary target through their web-based marketing material (as is apparent from their lack of a solid discussion of business value) who are they really marketing to?

Surprise!  It’s your IT Department, of course.  The vendors spend a lot of their time and money getting your IT people excited about their tools.  Then, the hope is that the IT people will start discussing those tools with the business people.  The failure pattern on this is that IT people are just as likely as the vendors themselves (if not more so) to NOT know how to sell business value to the business.

What’s the Big Deal?

“At least the sales guys aren’t calling me,” you say.  Well, a huge part of any sales process is teaching you the things you need to know in order to influence your purchasing decision in a positive way.  This would be where they explain to you, in detail, the “value” that their products bring to your organization.  But, as I’ve just pointed out, they aren’t really trying to sell their products directly to you.  So, by default, you’re going to be behind the learning curve on MDM.

Take a look again at the list of benefits above. These are business benefits (value) that can only be successfully attained through a well thought out and well executed combination of process improvement, effective governance, and appropriate technology decisions.  MDM adds value by solving very specific types of complicated business problems.  An MDM initiative may require the purchase of software tools to support the effort, but then again, it may not.  Unfortunately, you won’t be in a position to make good decisions on the technology until you have a firm understanding of the problem(s) being solved and how each specific MDM tool can be added to the solution to solve them, and you’re not going to get this reading the marketing material on the vendor web sites.

Next Time

In my next article, I’ll start giving you a more detailed explanation of the items in the benefit list from above.  The hope is that, if you have a good grasp of the business benefits/value that can be attained through an MDM implementation, you’ll be more likely to be the person in the driver’s seat for that MDM implementation, not the vendors or the IT folks, which is the way it should be (a widely-accepted best practice).

I’m attending the Gartner MDM Summit this week, and one of the first sessions I dropped in on was Mark Beyer’s “Extreme Information Management”.

Mark started out by talking about the extreme information challenges for the 21st century CIOs, saying it is no longer possible for one person or group to control how an idea like “big data” is used. But Gartner thinks the term Extreme Information Management provides more clarity.

The human brain is a purpose built processor, tuned to recognize threats more than anything else. One of the biggest threats today is stress due to information overload. He quoted Thomas Edison saying “I have not failed, I’ve just learned 10,000 ways that do not work”.

So today’s session is about mitigating the stress that comes from extreme information.

Through 2015, organizations integrating new information types will outperform their industry peers financially by more than 20%.

Unstructured data comprises 60-80% of all enterprise data. Anecdotally, Gartner is seeing 5-15% of analytic efforts starting to use some form of sentiment or social analysis to augment structured data analytics.

Information is exploding, growing at 59% per year. The new mantra is “include all the information assets, deal with the volume, and derive the business value”.

And gut feel probably differs from an answer derived from analytics 79% of the time.

Leaders are able to create new data points to prove that they’re more effective than the competition.

Extreme information management is the concept that your current information infrastructure must be intentionally managed along 12 complementary dimensions to meet the challenges of the 21st century Information Age.

Mark pointed the audience to a research note on Gartner.com: “Big Data is Only The Beginning of 21st Century Extreme Information Management”.

If volume is increasing at the same time as variety, that is a “big data” problem. Does the create case match the use case? If the business reports that volume is an issue, then ask if the variety and the velocity are changing as well.

The 12 dimensions are:

Big Data

Volume

Variety

Velocity

Complexity

Other

Technology

Contracts

Classification

Pervasive Use

Qualifying the Information

Fidelity

Validation

Perishability

Linking

You will always be stretching out on these 12 dimensional aspects – big data initiatives are never finished, and Fidelity is the culmination of the other 11 dimensions.

Traditional application data growth is related to 87% of performance issues, costs are 3-10 times more to store data as to capture it, and this represents 15% of your problem.

The variety of types of information assets is crushing.

Operational technology – small apps, millions of devices, leads to billions of instances.

Governance models in too many places can challenge the contracts around the use of data. Profiling the data in an ERP system allows the IT organization to work proactively with the business, to look for changes in the data that might mean changes in the business process. It’s very common to see four columns crammed into one (overloaded attributes).

There are a lot of opportunities in social media.

A network of related data is far more reliable than any one datapoint, especially if you know when, why and how it was related.

The “cloud” aspect of big data allows you to get more power, more data, and use an alternative deployment.

Big Data MapReduce Hadoop

Recommendations

Introduce a metadata management program

Plan on extending existing repository management environment to use cloud storage

Build at least one pilot analysis that combines text, social and operational data

I found Mark’s session very thought provoking and intellectually stimulating, and it helped me to think about where today’s clients who are doing MDM and data governance might need to go next.

Now that we’ve talked about the role of organizational change and education, readiness assessment, developing a strategic roadmap, creating a business case and designing the Data Governance initiative before trying to execute on it, it’s time to talk about the implementation process itself.

At this point, you’ve gotten everyone up to speed on what master data management is, and how critical Data Governance will be to successfully managing master data. You’ve figured out where the weak spots are, in the organization, in the current MDM and data governance processes, in the technology landscape, and in current information management efforts. You’ve accurately described the “As Is” state, and worked with the organization to envision the “To Be” or future state. And you’ve spent the up front time to design a new Data Governance framework for the enterprise going forward.

So it’s time to execute – to deliver on the vision. The client company has a lot of responsibility here. We’re all familiar with the saying “you can bring a horse to water, but you can’t make him drink.” If the client has problems making decisions and carrying them out, the Data Governance practitioner is going to struggle, no question about it.

I usually start with the organizational side of things, and I start at the top. If the senior executives aren’t bought in yet, back up and find out what it will take to get them there. These initiatives truly cannot be driven “from the bottom up”. It’s true that you’ve got to get the various lines of business on board, and to have the active support of IT. But if the “top of the house” isn’t there yet, you won’t get too far with your Data Governance implementation.

If your executive sponsorship is solid though, and you’ve got the makings of a Data Governance Executive Steering Committee, you’re ready to start worrying about building out the Data Governance Office. This is like a Program Management Office for IT projects, or a “Finance Department for Data”, as I’ve sometimes described it.

You’ll probably want to start with an internal person who already knows the business and who brings some existing political capital and relationships to the role. Knowledge of the business and the right leadership traits and personal characteristics are critical. Functional experience in Data Governance is certainly important, but it’s very hard to find internal candidates that know the business, have the right leadership skills, and have DG experience to boot. So go for the first two, and use your Data Governance consulting partner to bring the new Data Governance Office leader up to speed.

Of course, you may have to go outside the company, but try to avoid that if you can.

Then you can start filling the other positions in the DG Office. Look for people with internal experience managing data, and make sure you’ve involved them in the earlier steps of education, readiness assessment, strategic road mapping, and data governance framework design. That way, they should be bought into your strategy, since they had a hand in shaping it.

After settling the Executive Steering Committee into their role, and filling the necessary roles in the Data Governance Office, you’re ready to get the Data Stewardship community in the business (and the corresponding IT stewards) lined up. These folks should have been involved from the beginning too, taking part in the educational workshops, in defining the Current State and the Future State, and in buying into the Strategic Roadmap and Data Governance Framework / design.

You’ll need to spend a fair amount of time socializing the strategy and the design with these folks, so that they see being involved as a data steward as just the next logical step in their involvement in the Data Governance program.

Set up individual data stewards in the various functional areas, business units and geographies. Almost all of the time, these are going to be existing people in existing roles, but the Data Governance program is going to be formalizing their responsibilities around creating and maintaining corporate master data and reference data.

You’ll need to work with Human Resources in most cases, because job descriptions will have to be revised a bit, and existing compensation structures revisited. This is because the data stewards naturally will tend to do what they’re financially incentivized to do, like most people. So be sure to allow plenty of time for this reshaping with HR, and make sure your Executive Sponsor and Steering Committee will back you up when making these changes.

From a process perspective, a periodic meeting between the Data Governance Office and the Steering Committee is a good idea. And forming a Data Governance Council, consisting of the DG Office plus some or all of the data stewards, will also be useful. This group may need to meet weekly at first, until you reach some type of “steady state”, and can shift to bi-weekly or monthly meetings. Of course, if some big projects like an ERP upgrade or a CRM implementation come up, you may need to go back to weekly meetings.

It’s also a good idea to set up a “data help desk”, which allows anyone in the company to notify the Data Governance Office of a new data related issue. This type of case management capability will help ensure that issues aren’t falling through the cracks.

In terms of technology, make sure you’ve got the Five Essential Elements covered:

(1)  an MDM hub (either already implemented, in the implementation process, or at least planned for the future via the strategic roadmap),

(2)  adequate data integration technology (an ETL tool by itself is probably not enough, you’ll want a service-oriented architecture or service bus based approach to handle real-time or near real-time integration)

(3)  a data quality tool of some type, with strong profiling, standardization, matching, de-duplication, and “golden record” creation functionality

(4)  third party enrichment capability, particularly if you’re dealing with the Customer domain – don’t put this off to a later phase, the information you don’t have which is available from a third party provider like Dun & Bradstreet and Equifax (for information on businesses) or Acxiom (for information on consumers) can be invaluable in answering the questions the business has

(5)  technology to facilitate data governance, including tools for metadata management, data modeling, information access and security management, workflow and business process management, and policy management. You may not need to buy all of this right away, but Data Governance for a large enterprise is a big task, and anything you can do to automate routine tasks will usually have a positive ROI. Don’t try to do everything using Excel spreadsheets, Word documents, and e-mail. That type of “least common denominator” approach is a false economy. Technology won’t ever be a silver bullet for Data Governance (about 80% of the work will be with organizational, cultural, political and business process matters) but don’t ignore it altogether; it can be a powerful “force multiplier”.

I’m alway amazed how many companies try to do master data management and Data Governance without one or more of the above capabilities.

If you’ve got your organization built out, your processes designed and in place, some new technology fitting in to automate labor intensive tasks, and your information is starting to get some dedicated scrutiny from the centralized Data Governance Office and the distributed data stewardship community, you’re going to be well on the way to a solid Data Governance implementation.

Just remember that there are five or six levels of maturity in most Data Governance Maturity models (the National Association of State CIOs has a great review of them). So after everyone agreed on the current level and desired new level during the readiness assessment and strategic roadmap phases, make sure you manage expectations that the company isn’t going to magically go from a Level 1 to a Level 5 in one year.

One company I’ve worked with and stayed in touch with has been at this for ten years, and at a recent conference, I heard their CIO say they only graded themselves a “4″ on a 5-point maturity scale for how they manage their master data.

So be realistic with everyone on how long things are going to take, and the fact that the company is usually embarking on the first year of a continuous improvement process that requires organizational change management, cultural evolution, process redesign, technology adoption, and a new dedication to managing information.

To be a “data driven company” – which I believe is required to succeed in the twenty-first century – is a lot of work. And one of the most important tasks for the Data Governance Office during Year 1 is to be planning and working towards Year 2 and beyond.

You can cut-and-paste the following into Twitter: >> I’m reading “Secrets of Successful Data Governance Implementation” on @HubDesigns Magazine at xxx (please retweet) <<

Like data integration processes, MDM solutions can be seen at times as ‘plumbing’, but in both cases the work to be done has strategic, business-critical purposes.

Master Data is meant to be at the heart of most business transactions, applications and decisions that occur in an enterprise. In today’s volatile business world, it’s important to focus MDM on what matters most for the goals of the business to ensure reliable, unified and timely data for key systems and processes. Such data must conform to how information is used by the business. So while it is tempting for enterprises to think of MDM as a technical or IT initiative, MDM is far more about business purpose and business problems to be solved.

To solve the problem of ensuring high quality ‘cleansed’ and unified master data, interest in MDM practices and solutions is increasing. But in many ways, MDM implementations are still immature and the solution space is still evolving. Our colleague William McKnight notes that often MDM solutions are focused on “simple” issues, rather than on larger, more strategic value, but that he is “starting to see some pioneer MDM implementations explore the full range of benefits”.

Among the benefits are data governance practices that elevate data to a strategic position in the enterprise in support of: operational and productivity improvements, reliability of BI and analytics results, timely customer insights, and ultimately, better decision-making.

Key Challenges for MDM Practices and Solutions

• Determine most beneficial focus for MDM and where to start
• Collaboration between IT and Business – both are needed for MDM to work
• Visualizing and aligning strategic and tactical milestones, while moving forward on the ‘right’ path
• Integrations with business processes and systems
• Securing confidence in the ‘rightness’ of the master data
• Demonstrating on-going and future value of MDM initiatives

Understand Business Processes to Understand Data Priorities

First, it is imperative to understand how the business operates: how the business uses key data and processes, how processes interact with data, and which business processes matter to strategic focal points. From an understanding of business processes, the best approach or approaches can be determined for implementing a master data solution that not only creates trustworthy, consistent master data but is responsive to changes in the business and to new data that continues to be collected / generated by business processes.

Data quality plays such an important part in the MDM solution. Analysis must be done to determine overall quality, where the weak links are, and the impacts on business processes and operations from inadequate data quality. But data quality is a subjective measurement – part of the process and data assessment will be the setting of requirements for acceptable data quality for individual data elements to provide the level of quality needed by the selected business processes. Some thought should also be given to future needs, whenever possible.

The potential “trap” is that data that is “good enough” for one business process may be inadequate for another, particularly when considering future directions.

Data derives its real value from its usability in significant business processes, including those that drive and benefit from analytics, intelligence and decision-making processes. And value attaches to data when it positively impacts business outcomes. Unified, quality data attained through MDM solutions must also be integrated into business processes, which is not an easy task. Such integration is not only at the technology and systems levels, but involves the synchronization of people, practices and ‘super’ processes across the enterprise to reflect how work is accomplished.

MDM Effectiveness Needs Business + IT Collaboration

• Which business processes are the focus — how is data used now and in future planned versions of these processes?
• How are business processes and data used across the enterprise, across business units, functions, roles, partner channels, and geographies?
• What are the relevant business objectives – which processes and data are related to them?
• What are the standards for data quality?
• How will the failure to achieve high quality unified view of data negatively impact business goals and objectives?

Extreme Information Management aka ‘Big Data’

All data management solutions are now challenged by the proliferation of data, content and information in sites that are not owned by the enterprise, such as cloud and mobile services, social sites and partner repositories. It has become clear that such data can be valuable to enterprises, particularly for insights into customers, future trends and markets, and directions for innovation. MDM solutions are also challenged to now pull in and harmonize some very unwieldy data with existing master data repositories.

In particular, integrating customer master data with customer-related content in social media will likely lead MDM solutions into new territory. The difficulty of content from social media is capturing the data ‘clues’ to be passed to analytics for accurately determining sentiment and context. So-called ‘big data’ needs to be aligned with reliable enterprise master data to properly correlate social content in decision-making processes. How the MDM solution maintains an overall master customer ‘profile’ will include continuous integration processes for targeted social media sites.

Enterprises continue to experience massive increases in data assets. Beyond social content, enterprises encounter data explosions from business activities such as constant M&A, outsourcing initiatives, and highly diversified partner and supply chain channels. Industries such as Telecom have enormous stores of machine-generated data related to customer activities as well as operational performance and ‘product’ pricing. GPS or location data relates more and more to customer and market insights for certain industries.

Achieving consistent master data with such disparate, sometimes unwieldy and complex, data sources will continue to be difficult, but will likely lead MDM practices and solutions to new levels of value to the enterprise and vital business process. Again, the focus will have to be on the key business processes that need these new data sources integrated with existing master data.

About the author: Julie Hunt is an accomplished software industry analyst, providing strategic market and competitive insights. Her 25+ years as a software professional range from the very technical side to customer-centric work in solutions consulting, sales and marketing.  Julie shares her take on the software industry via her blog Highly Competitive and on Twitter: @juliebhunt. For more information, please visit Julie Hunt Consulting – Strategic Product & Market Intelligence Services.

You can retweet this by cutting and pasting >> New on @HubDesigns Magazine: “Business Priorities Point the Way for #MDM” by Julie Hunt at http://wp.me/p5Tdn-Nd <<

In our opinion, the Gartner Master Data Management Summit is the best conference in the MDM and Data Governance space, covering all industries, all domains of data, and all major vendors. The quality of the analyst research and education there is top notch.

I’ve attended every year since 2008, and I find the event very helpful in learning new things, networking within the MDM industry, and talking to past, present and future Hub Designs clients.

This year, the event will be held at the JW L.A. LIVE in Los Angeles, CA, on Wed. April 4th through Thu. April 5th, 2012. For more information, go to gartner.com/us/mdm.

We’ll be hanging out in Booth “W”. If you’re attending, please stop by and see us during the Solution Showcase (Wed. April 4th, 12:30-2:30 pm and 6:00-8:00 p.m. plus Thu. April 5th, 11:45-1:45 pm).  Please let us know if you read HubDesignsMagazine.com; we’d like to meet as many readers as possible!  Julie Hunt, our new editor, will be there as well, looking for new writers and story ideas.

We’ll be talking about our Consulting practice, which has been working with large clients on MDM strategy and data governance formation for almost five years. We’ll also be talking about our Thought Leadership practice, which develops high quality content to help MDM software vendors reach their target audiences, through white papers, webinars, blog articles, speaking engagements, and social media.

If you’re like to meet during the conference, please send a message via the Hub Designs’ Contact Us page.

We also have two full conference registrations available – so if you’d like to attend at no charge (you’re responsible for your travel expenses), please contact us via the Hub Designs’ Contact Us page. 

You can retweet this by cutting and pasting >> New on @HubDesigns Magazine: “Hub Designs Sponsoring Gartner MDM Summit for 4th Year” at http://wp.me/p5Tdn-N1 <<

Stan Christiaens, COO at Collibra sat down with the Hub Designs MDM Think Tank in early March to give us a briefing on Collibra’s Data Governance Software.

Collibra focuses on the data governance aspects of MDM.  It addresses the elements of governance in the vision presented below – organization, people/processes, documentation and operationalization.

The product provides a rich, configurable workflow system for routing a data record throughout the organization (and possibly beyond) to the point in which it becomes a master record and is stored in a relational database.

At this point, Collibra integrates with IBM InfoSphere and Informatica Metadata Manager for operational distribution. The API can be leveraged for other integrations.

Collibra also provides a fully functional metadata management repository for the attributes under management.  This includes some really useful features around the stewardship function, which allows a Collibra implementation to be flexible to the organization’s needs.  As Stan said, “you need to understand the company before imposing organizational change.”

With a multi-layered and as-complex-as-necessary hierarchy of stewardship which Collibra calls a “communities hierarchy,” available stewards can be easily found for governance participation, data questions or the other data enrichment functions that stewards provide to their data.

Other cool features include profile pictures associated with the names in the dictionary and the ability to access the dictionary from anywhere.  Within an application like Word or Excel, you can hit a hotkey and see the full dictionary representation of the element.  This includes the source system, fact types, derived fact types, rules, roles, taxonomies, definition and record status.  Data Governance, according to Stan, is “making people do the right thing.”  The more you think about this simple definition, the more accurate it seems to become.  And Collibra is creating information sharing at the business level, circumventing much of the semantic gap between IT and business areas.

Although there is much to data governance beyond the toolset, tools like Collibra can certainly facilitate getting a data governance program off the ground.  Data governance tools like Collibra are on the periphery of a required toolset in organizations employing MDM.  It’s like purchasing a faster utility for a DBMS.  The base DBMS utility has the functionality, but does it have enough for your need?  It will be interesting to watch this niche market to see how it evolves in the MDM ecosytem.

Seven Steps to Data Quality

Many organizations start a Data Quality Program solely as an IT initiative without business involvement, and then try to persuade business teams about the value of data quality. For business stakeholders to fully understand the importance and value of data quality, it’s essential to get them involved from Day One as champions of the effort.

Follow these seven simple and practical steps to jump-start your Data Quality Program and make it quick, effective and able to meet your business needs. These ideas are from my practical experience of having implemented it at a large financial services company.

Step 1: Understand the business need for data quality

Take a “top down” approach by starting with the business and understanding their current pain points and the value to them of improved data quality. They’re the key stakeholders, so it’s best they drive the program and be part of the data quality process.

Case Study Lesson: we identified a small team of data stewards that engaged with business stakeholders to understand the issues they were facing due to lack of data quality. Those discussions led to detailed requirements gathering sessions.

Step 2: Define the scope

Take a “start small, finish big” approach. Identify the first set of systems and attributes that have major data quality issues where fixing them will give the best value for the effort.

Case Study Lesson: we identified source systems and data attributes that contributed to the key performance indicators (KPIs) of the business. We prioritized the data elements into low, medium or high in order to be included in the scope for measuring, monitoring and improving data quality.

Keep the scope narrow and yet effective so you can get back to the business with results sooner (days rather than weeks). Address disconnects or refinements early on.

Step 3: Collect detailed business requirements

Sounds obvious yet many data quality programs don’t have requirements documented, reviewed, and approved by the business stakeholders.

Case Study Lesson: we prepared an inventory of about 100 critical business rules to identify exception conditions in business operations: 

• An “A-Active” loan cannot have unpaid loan amount (UPB) equal to zero. This implies either the loan status or loan amount or both are incorrect.
• Loan to Value (LTV) ratio cannot exceed a predefined limit (Loans with high LTV indicate higher risk)
• A customer with occupation as “Lawyer” or “Professor” cannot qualify for a “Student” discount.
• Identify marketing amount spent on “Do Not Contact” customers or leads.

Step 4: Automate and profile business requirements using a standard tool.

Many data quality tools perform out-of-the-box analysis on rows and columns that could mean little to the business if the data is not validated against THEIR business rules.

Case Study Lesson: we used a combination of home-grown and Commercial Off The Shelf (COTS) tools. The rules were part of the data quality database. The rules were run periodically and results were analyzed by the data stewards and the findings were presented to technology and business stakeholders.

Step 5: Perform root cause analysis on the identified issues

Review findings with both technology and business stakeholders. The findings typically include the need for business process change, application change, user training, and data quality requirements change.

Case Study Lesson: our data stewards, in combination with technology and business stakeholders, performed the root cause analysis of issues.

The findings included:

• Technology issues — application allowing null values in a certain field
• Process issues — compliance documents not submitted for several years, added a process to check for compliance
• Training issues — user did not know where or how to check for the document repository
• Refinement of quality rules — data results initially thought issue prone were found to be valid under certain conditions
• Or a combination of all of the above

Step 6: Remediate the root cause of issues.

Case Study example: this involved technology “fixes”. These were typically scheduled for the next available release. Process changes were communicated to appropriate business stakeholders for remedy and follow up. A very important field, storing percentages, had several versions of precision after the decimal in different systems (e.g. 5.5, 5.555, 5.55, or 5.55555). When these were used in calculations over huge loan amounts, the differences were materially significant.

Step 7: Measure and Monitor Data Quality Metrics on an on-going basis.

Case Study Lesson: we performed Steps 1 through 6 on a monthly basis. Metrics on data quality measurements were reported in a simple spreadsheet with insightful charts and graphs and trends over time.

As a result, business stakeholders were easily able to see how their data was getting better over time. If done right, you should start seeing data quality improvements within the first 90 days.

Additionally, I recommend base lining your data set prior to program initiation to compare and demonstrate the value once the program is in place. Communicate your “aha” moments from data profiling and analysis to business stakeholders. Demonstrate the value add and go for the next increment of the data set.

Can you share your experiences in improving data quality? How effective have they been? What would you add to the above list to make it simpler and more effective?

Kuppusamy Bhaskar, http://www.linkedin.com/in/kbhaskar, kbhaskar@aikya-inc.com

Brian Vile, Program Director, MDM Product Management at IBM, and Vasu Vallurupalli, Solutions Lead and Alliances Business Development Exec, sat down with the Hub Designs MDM Think Tank towards the end of 2011 to give us a briefing on changes in IBM’s Master Data Management portfolio.

The first thing Brian and Vasu wanted to brief us on was a big announcement about the unification of IBM’s three MDM products in InfoSphere Master Data Management V10:

• InfoSphere MDM Server (based on IBM’s acquisition of DWL in 2005)
• InfoSphere MDM Server for PIM (based on IBM’s acquisition of Trigo in 2004)
• Initiate Master Data Service (based on IBM’s acquisition of Initiate Systems in 2010)

All three of these products are now being sold as “IBM InfoSphere MDM V10”, broken out into these “editions”:

Essentially, the Standard Edition is Initiate Master Data Service, the Advanced Edition is both InfoSphere MDM Server and Initiate Master Data Service offered together. The Collaborative Edition is the former IBM InfoSphere MDM Server for Product Information Management (PIM). Finally, the Enterprise Edition includes all three of IBM’s MDM offerings.

What we’re starting to see as well is tighter integration between various other IBM products and its MDM suite.  For example, the workflow capabilities in InfoSphere MDM Server and Initiate are based on IBM’s BPM Express, the leading business process management product formerly known as Lombardi.   IBM’s intent is to focus on improving the master data management experience for its customers. So it includes the Express Edition of BPM, a streamlined version of that product, at no extra charge, and the MDM product is now compatible with the full version of IBM BPM v7.5.

I’ve been a proponent for several years of more tightly integrating MDM solutions with BPM capabilities, and its seems like IBM is providing this to improve master data consistency, to present data in its business process context, to help implement data governance policies and to coordinate multi-step, multi-role workflows for data stewardship and data governance.

So now IBM is selling four SKUs – the Standard Edition, Advanced Edition, Collaborative Edition and Enterprise Edition. While this may confuse some IBM customers initially, there’s no doubt that it’s a powerful story to be able to tell the marketplace. IBM claims to be #1 in market share, with 700+ customers and a lot of experience in helping customers with Information Governance through the team headed up by Steven Adler (http://infogovcommunity.com).

Some of the other themes IBM covered with us include the unified matching engine (based on Initiate’s matching engine), which provides consistent probabilistic matching across all of the editions (except Collaborative), and an emphasis on real-time capabilities, through an application toolkit made up of an Adaptive Service Interface. The service model can also be read through IBM’s Rational software development framework, which allows customers to build flexible, reusable MDM-powered applications on top of trustworthy master data.

All of these things, in IBM’s opinion, simplify the skill set needed to implement InfoSphere MDM, accelerate overall time to value and reduce risk by decreasing the time to go live, as well as lowering the overall total cost of ownership. Given the emphasis recently on rapid, incremental implementations that deliver value to the business quickly, these seem to be moves in the right direction for IBM.

The Adaptive Service Interface works, in part, by creating a “views” layer on top of the tables in the database, thus reducing complexity and simplifying access to the underlying data. IBM can do industry-specific views as well in the ASI layer.

IBM says they are now routinely seeing 3-4 month implementations in their customer base, and can offer the transactional, registry and hybrid styles of MDM deployment, as well as the collaborative authoring style in a single product suite.

Global Name Management is another new feature in V10. It’s based on IBM’s Global Name Recognition product, and is embedded into the probabilistic matching engine in order to help with international data.

Another core InfoSphere MDM Server enhancement is “undo collapse”, which allows a user to fully undo the merger of two entities which seemed like duplicates but later needed to be undone. This is a very sophisticated feature that, in our experience, most companies end up needing at some point in their MDM journey.

IBM also is making it easier for customers to migrate from one edition of the product to another, from Initiate Master Data Service to InfoSphere MDM Server, for example. IBM claims that 50% of what customers invest in the Initiate implementation will carry over to InfoSphere MDM Server; through the tools it provides to ease the move from registry style to transactional style.

In terms of what’s next, IBM foresees one major and one minor release per year. All teams within the Information Governance development product area are using Scrum, an Agile development framework.

And IBM is using simple Themes now for communicating to its customers, the broader market and the analyst community. For example, the “IBM MDM Portfolio” theme in the strategic roadmap is based on the new version of IBM InfoSphere MDM, with its new features for usability, scalability, availability, upgradability, etc. IBM is striving to offer rapid implementations and a seamless MDM journey from one edition of the product to another. The product offers a wide breadth of data models and business services, and supports a variety of platforms, languages and cultures.

The “Information Consumption” theme is all about making the use of master data easier, through business services and service orchestration, the MDM Application Toolkit for building “widgets” that perform useful work for other applications and for building MDM-Powered Solutions on top of the high reliability master data. With event alerts and notifications, an adapter/connector framework, and native support for unstructured data (still over 80% of most enterprise’s information), IBM is rounding out its offering to make it easier to consume master data.

The “Master Data Governance” theme includes a new offering for Reference Data Management that will be “productized”, and a Master Policy Hub to provide policy management capabilities. This will provide a mechanism to build policies, a dashboard to let you know how you’re doing on those policies, and alerts to tell you when things change.

With those refinements to its offerings, and the unification of all of its MDM products in one cohesive suite, IBM seems to have matured its portfolio quite a bit in the last twelve months. I spent a lot of time at the last Gartner MDM Summit talking with key IBM people like George Skaryak and Rick Clements, and saw the early stages then of what IBM has now fully rolled out.  I’m looking forward to spending more time with them at the upcoming Gartner MDM Summit in early April in Los Angeles. It should be interesting to see what IBM is coming up with to build on what they’ve rolled out over the past few months.

The purpose of this article is to discuss how each helps strengthen the others, sharing some of the experiences I’ve had implementing MDM with these technologies.

Based on the size of the organization and how centrally managed enterprise services are managed, MDM, SOA and BPM each require an IT initiative (sponsored by the business) to land if the goal is to establish a foundational IT capability.

Let’s be clear, these three capabilities are cost centers—they themselves don’t generate revenue or profit.  Anyone implementing these solutions knows that pain. Any positive ROI story hinges on what they enable and how effectively we can make that connection.  They are a means to an end, sharing a common vision of helping the business.

I’m not going to delve into the various possible business benefits in this paper (e.g. MDM’s single authoritative version of the truth or 360 view of customer, SOA’s “real-time” enablement and BPM’s business process re-engineering opportunities). I am going to point out that each is to some degree reliant on the other for the successful enablement of business goals.

First, I’d like to clarify this article is written largely from an MDM standpoint. I’d like to give a brief background on the two successful MDM implementations I’ve helped steward in order to give context.  These two MDM solutions are both around “customer master” data but are fairly radically different.

MDM Solution #1 – I had the privilege of being the architect and development lead for implementing a Siperian-based MDM implementation for customer “organizations” (B2B) consisting of roughly 200,000 customer records.

The key points for this MDM solution included:

• MDM core was a central transactional hub, storing the single version of the truth, and syndicating that truth to the rest of the enterprise.
• MDM exposed a real-time web service (WS) interface for all updates (outside of initial data loads) that were managed by a SOA Enterprise Service Bus (ESB).
• We centralized all customer master data updates for the enterprise into a common BPM process that orchestrated human workflow steps (using IBM’s FileNet eForms and a Web user interface) and SOA MDM services (MDM was the only system of origin for organization master data)
• Applications in the eco-system (i.e. SAP ERP) were updated via an asynchronous publish/subscribe model using canonical messaging
• The solution implemented a Party pattern and additional MDM entities were on the strategy map: Vendor, Employee, Product

MDM Solution #2 – As a technical Program Manager Lead I’ve been responsible for a variety of tasks including establishing a multi-year roadmap, and leading a team of program managers to design and manage dozens of MDM features to land a customer MDM platform for “individual” data (B2C), consisting of over 1 billion customer persona records.  This DIY solution is arguably one of the largest scaling MDM solutions in the world.

The key points of this MDM solution included:

• MDM core was a central transactional hub, storing the single version of the truth, and syndicating that truth to the rest of enterprise
• MDM exposed real-time sync, async, batch and pub-sub integration patterns.
• MDM harmonization approach was used to harmonize the many “systems of origin” of this data across the enterprise
• No BPM aspect to the solution
• MDM solution had to handle ~1500 transactions per second in update traffic at peak, with sub 400ms response time, with strong emphasis on engineering excellence around high availability (HA), disaster recovery (DR) and supporting online upgrades (“always on”).
• Solution implemented an MDM platform (scale up and out) designed to store any number of MDM entities (into the billions of records total).

Both of the above were successful: each landed an MDM solution meeting the original aspirations of the initiative—albeit, with considerably different technical solutions.  Both involved real-time SOA, but implemented it a bit differently.  Both involved workflow, but only solution #1 involved BPM in the sense of enabling the business to define process flows externally to MDM, orchestrating MDM and human workflow.

The key question enters my mind: what have I learned from these experiences?  What additional strategies and/or approaches could I promote or evangelize with the business stakeholders for helping solve the business problems being discussed? I feel that having a strong understanding of how MDM, SOA and BPM can work together will really equip you with the tools and approaches to be successful.

MDM Briefly

For the purpose of this article, I’m looking at Operational MDM, and customer data specifically.  I assume you’re familiar with the key capabilities provided by an operational MDM solution, including establishing a single authoritative version of the truth for shared and common enterprise data and having that data then be used across the enterprise.

MDM and SOA

Real-time is the reality of business today; business agility and savings are the battle cry. Whether driven by just-in-time aspirations, quicker time to market, or faster customer service, IT is tasked with making data highly accessible.

MDM operational solutions (whether Commercial Off The Shelf or Do It Yourself) are center stage and a facilitator of this objective. As such, most MDM solutions involve a service layer in the form of Web Services (SOAP, REST, HTTP, etc.) whereby clients send service requests to a real-time interface that receives the request and sends back a response.

A simple example would be MDM providing a basic set of Create-Read-Update-Delete (CRUD) service operations that can be used to lookup or update master entities (with proper security).

Clearly, having a SOA strategy – perhaps involving an Enterprise Service Bus (ESB) and directory services – is of value to MDM. Operational MDM needs it.   If a SOA strategy is not in place, some serious discussions are needed to make sure you don’t go down the wrong path with MDM services.

Does SOA benefit by having MDM?  Yes.

Without MDM, you risk limiting your SOA solution to tactical integrations with one-off point-to-point integrations where individual systems hand-shake on specific needs, or you risk limiting SOA to intra-application scope.

You risk simply paving the cow paths of the past—yes, it is in real time, but you risk implementing a new flavor of spaghetti (we’ve all seen those integration diagrams!).

The opportunity here with merging MDM and SOA is recognizing that the enterprise data definitions in MDM change the game from tactical to strategic. MDM establishes a common identifier/key and a common definition of the attribution of the entity or entities in question.

It may even allow an application to consider enterprise uses beyond their narrow immediate requirement—concepts like creating composite services come to mind, whether combining disparate entities, or with data augmentation from key engagement/transactional data.

In my experience, both of the MDM solutions I’ve helped design and implement had a SOA strategy as the preferred integration approach. Clearly, offering a secure integration interface that allows for synchronous interoperability within an enterprise, or even to external partners, is a key for success in a real-time world.

When working on MDM solution #1 (mentioned above), I appreciated the benefits of an ESB, including:

• Abstracting MDM from the various enterprise applications integrating with MDM. MDM’s service operations were locked down to only the ESB, while access security was implemented on the ESB per a standard security policy
• Providing ability to create targeted listener end-points on the ESB that pointed to a common general service in MDM
• Providing a basic versioning strategy allowing MDM to deprecate older service contracts earlier, and using the ESB to manage deprecation of the old contracts (both old and new mapping to the new contracts).

MDM solution #2 did not include an ESB because it wasn’t present.  We accomplished some of the same benefits by using virtual IPs.  Not quite as elegant architecturally, but it worked nicely.

MDM needs SOA and without MDM, a SOA strategy is at risk of being ineffectual and likely the breeding ground for a future re-factoring in the future.

SOA and BPM

The word complementary comes to mind.  SOA + BPM represents a perfect example of the whole being greater than the sum of the parts.

SOA’s real-time nature and interoperability lend themselves well towards enabling time critical business orchestrations.  Add in human workflow and now the business has the ability to choreograph end-to-end processes.  True business process management can occur, limited only by the maturity of the SOA strategy and the ability of the BPM tooling.

BPM without SOA is possible, but the opportunities are likely constrained, and the results involve cumbersome point-to-point integrations.

While there has been some tension in the past between SOA and BPM camps (I’ve not been in any of those religious wars), the benefits of the two working together are strong in my experience.  In fact, I would argue that to maximize the value of SOA, understanding the larger meta-point around what scenarios the services could facilitate in business processes is a must.

BPM and MDM

Let’s face it, most, if not all, of the core business processes that generate revenue involve the core and common Master Data Entities (Organization and Individual customers, products/materials, etc).

Companies sell products or services, and they sell them to organizations or individuals. The ability to look-up and potentially update MDM data as part of these business processes is a given.

As a side note, in in attempting to keep this document simple, I won’t be introducing or discussing multi-mastering (multiple systems of origin) or the implications of whether MDM is the sole mechanism for updating data or if harmonization is required.  BPM integration with MDM would / could be impacted accordingly.

While MDM services are likely part of many BPM processes, the MDM solution itself should remain agnostic to business processes as much possible.  With that stated, MDM can provide capabilities that support a BPM strategy while largely remaining agnostic.

As an Operational MDM solution, it is assumed that the master data is the single authoritative version of the truth.  Sometimes organizations call this the “golden” data.  Along those lines, an MDM solution can be used to store pre-golden data using terms like “copper” or “silver” to reflect that the data is not yet fully certified.

By implementing a “copper/silver” data strategy to the existing “gold” data strategy in MDM, you can open up the possibility of having MDM store the “in-flight” update data that is being choreographed via BPM workflows.  Using a BPM provided parameter value that asserts the level of the data is a fairly simple approach that keeps MDM agnostic to the many BPM workflows, but allows for delineating between “copper” and “gold” lookups or updates—a lineage capability is a natural fit for storing this in-flight data.

Some form or record locking is likely also needed to eliminate collisions. Note that part of the update business rules in MDM would/should enforce an appropriate level of validation prior to being asserted as gold. (A more in-depth discussion on these concepts would be appropriate as a separate article.)

While SOA and BPM should be loosely coupled, there are MDM design considerations that should be considered ahead of time that can allow for MDM to more strongly support workflow concepts. I strongly recommend that these considerations be evaluated as soon as possible in an MDM high level design.

Pulling It All Together

The following diagram is a high level view on how BPM, SOA and MDM can fit together. This example is very MDM centric, effectively representing updates to master data.  Other examples could be made where MDM data is retrieved as part of order-to-pay processes, etc.  This MDM example best aligns to business processes involving management of the master data by company representative (working on behalf of customers), as opposed to consumer self-service scenarios.

In the example above, many interactions occur, and implementation details may distinguish the BPM from the user interfaces involved.  I do want to call out that I’ve simplified the “BPM” layer and combined the UI element.  Several design options are available that would impact whether BPM and/or a UI element interacts directly with MDM.

Key points include:

Step 1 – A change is initiated by a triggering event—likely a person initiating a process. This initiates a new workflow item.  An e-form or web-based interface then allows the user to search/lookup the record, make necessary changes, and submit.

Step 2 – The BPM manages the workflow items, choreographing the necessary human and automated steps in the business workflow (direct integration with SOA depends on architectural/design approach for how the UIs interact with BPM or if they call the ESB directly).  BPM of course manages the work queues, assignments, creating metrics on age of requests (SLA), status dashboard, etc.

Step 3 – ESB listeners receive requests, verify security credentials, map requests to the appropriate listener, and map responses back to the originating caller.

Step 4 – MDM services process the incoming requests. The validate operation is tasked with ensuring a master record (or object) passes necessary minimum data standards and quality bars. Using a rules engine, a given validation request can be configured to apply rules based on a passed parameter defining the “phase” of the request.  This can allow for defining a ramping level of quality gates.  For example, the initiating validation may be basic rules, while the final approval from the steward could require that all applicable rules fire.

Step 5 – Leveraging the copper/gold concepts mentioned above (see “BPM and MDM”), similar to the Validate operation, the Update operation can use the “phase” parameter to determine lifecycle of the update received.  Updates can then be evaluated and executed accordingly until the appropriate “gold” request occurs.

Step 6 – Both “copper” and “gold” updates can be stored to the MDM store.  Once this data is persisted as “gold”, thus becoming the authoritative truth, it is available for lookup and for syndicating out to subscribing systems.

Step 7 – The majority of MDM solutions require syndicating out updated master data to subscribing systems, even if only to an EDW/BI solution.  In an ideal perfect world, all operational systems in your enterprise will use MDM as their store for master data, but that may be a vision that takes a long time to achieve (if ever).

Step 8 – A Pub/Sub type application (which can be distinct from or part of the ESB platform) receives these syndications and brokers, transforms and routes these changes to subscribers.   Concepts around canonical message forms and come into play here.

Summary

The benefit of experience is that you have firsthand knowledge of what works, what doesn’t work, and what opportunities you might have in the future the next time you find yourself in a similar situation.

Whether I day dream about “if I could do it over again” or if indeed I find myself defining another MDM strategy/solution in the future, I can safely say that I’ll be exposing the opportunities and options to the business on how MDM, SOA and BPM can be leveraged to meet business needs.

Olmstead Associates, Inc. – a leading provider of consulting and professional development to the securities industry – and Hub Designs, a specialist in the development and delivery of high value master data management (MDM) and data governance strategies, have announced a partnership to provide information management solutions to the financial services industry.

“While data has always played an important role within the securities industry, that role has grown significantly and will expand dramatically in the future” states Philip Sindel, President of Olmstead Associates.  “The need for enterprise level information management is being fueled by pending regulatory demands requiring more transparency, customer needs, risk management and the globalization and speed of all markets.  By teaming with Hub Designs, a leader in providing structure and governance for enterprise information management, we can help our clients maintain and access their data across the entire organization.”

Dan Power, founder of Hub Designs, notes that “succeeding at information management requires bringing together many different and complex pieces.  Attention must be paid to putting together or changing a data hub, data integration, cleansing and enrichment tools, data governance, organizational culture, business processes and many other factors in a way that provides measurable and tangible benefits to the business.  By combining our focus on information management with Olmstead Associates recognized expertise in the securities industry, we can provide powerful solutions to industry firms.”

About Olmstead Associates, Inc. 

Olmstead Associates is a Boston-based consulting firm focusing on the securities industry. They bridge the gap between business and technology by providing management consulting, project management, subject matter experts and industry specific professional development to many of the top industry firms. They have worked extensively supporting pre-trade, trade and post-trade operations and systems initiatives.

About Hub Solution Designs, Inc.

Hub Solution Designs, Inc. (Hub Designs) is a global management & technology consulting firm, specializing in the development and delivery of high value master data management (MDM) and data governance strategies.   They deliver value and competitive advantage through experience in management and technology consulting, plus a distinctive approach to working with clients.

Contact:

Olmstead Associates, Inc.                            Hub Designs
185 Devonshire Street, Suite 1000             188 Whiting Street, Suite 6A
Boston, MA  02110                                         Hingham, MA  02043
Phone: (617) 778-0372                                 Phone: (781) 749-8010
http://www.olmst.com                                 http://www.hubdesigns.com

Salah Kamel, the CEO and Founder of Semarchy, and FX Nicolas, Semarchy’s Senior Director of Product Management and Marketing, were gracious enough to do a briefing with the Hub Designs Think Tank in early November.

Salah’s background includes being a senior architect at Oracle for the Oracle Data Integrator product, and being Chief Technology Officer and co-founder of Sunopsis, which Oracle acquired in 2006.

Salah worked closely with the MDM teams at Oracle, and after doing a thorough review of the other MDM hubs, decided to leave Oracle to create Semarchy, a self-funded startup that was founded in 2011 and is based in Lyon, France.

In November 2011, Semarchy released Semarchy Convergence for MDM Release 1.2, as well as new marketing positioning and messaging and a new web site.

Semarchy refers to its approach as “Evolutionary MDM”, and believes this approach accelerates the time to value and reduces project risk versus other MDM products on the market, by allowing customers to converge on a Single Source of Truth.

Salah and FX emphasized three points about Semarchy Convergence for MDM and the Evolutionary MDM approach:

(1)   It’s non-intrusive – the product doesn’t force customers to change their IT architectures or their business processes. Convergence uses an automatically generated convergence hub into which the “golden records” are enriched, validated and consolidated.

(2)   It accelerates project evolution – we all realize how much projects and requirements change over time, so Convergence provides version control over everything. For example, you can have multiple simultaneous versions of the same data model within the same master data instance.

(3)   It controls data evolution – master data also changes, due to updates from the source systems, and because of adding source systems and updating rules used to consolidate source data into “golden records”. So master data can be versioned and then accessed as it existed at specific points in time.

Semarchy crafted its product primarily for IT management and systems integrators, so they can get started tactically and work iteratively.

Salah and FX told us about the main features of the product, which include:

1. Logical Modeling: Semarchy Convergence for MDM provides a rapid, iterative framework that simplifies designing semantically complete master data models. It enables users to collaborate closely with IT to define high-level concepts using business terminology. This single point of definition enables rapid movement towards consensus, and is very useful for modeling things like financial hierarchies and product catalogs.
2. Data Certification Framework: the Data Certification Framework transforms the source records pushed to the hub by source applications (Publishers) into consolidated and certified “Golden Records”.  The framework supports record and field-level consolidation. The certification process is automated and involves several phases, generated from the rules and constraints defined in the model. The framework enables the fastest time to production for MDM implementations.
3. Versioning: the Version Control features allows customers to have their master data evolve along with the enterprise on the three dimensions of the master data lifecycle: Content, Shape and Breadth. These features enable a fast, simple and safe iterative approach to MDM.
4. Non-intrusive Convergence Hub: Semarchy Convergence for MDM generates and deploys a Data Convergence Hub once the logical model has been designed. This Data Convergence Hub uses an architecture that is unobtrusive for master data publishers and consumers, and non-disruptive to existing processes. The hub includes the data certification processes, the storage structures and the access APIs for the deployed models.
5. Data Access: business analysts, data stewards and application users using the generated user interface can collaborate and access the master data as it is now or as it was at any point in time. This interface makes master data access intuitive for all users.
6. Open and Scalable Architecture Based on Web 2.0: Semarchy Convergence for MDM uses an open architecture for enterprise-scale deployments. The core of this architecture reinforces an unobtrusive approach for Evolutionary MDM. The platform is secure, scalable and extensible. It supports ever-growing data volumes, and is built to be Cloud-ready.

Right now, Convergence runs in a Java application server and uses the Oracle database management system for storage and data processing. It provides SQL and Web Services input and output APIs to access the Convergence Hub.

Although the product provides some important data quality features, it can also be used with the major data quality tools on the market.

Semarchy has created its own language – SemQL – that allows the rules driving the data certification process to be expressed. With SemQL, users can create enrichment, validation, matching or consolidation rules using the business terms defined in the data model. The company is also creating plug-ins that allow users to leverage third party products or services such as Google or Yahoo Maps in the data certification process.

With all the consolidation in the MDM market over the last couple of years, it’s very encouraging to see young companies like Semarchy still entering the market and innovating with their product offerings. We look forward to more briefings as the company and its products evolve.

You can cut-and-paste the following into Twitter: >> I’m reading “Semarchy Convergence for MDM” on @HubDesigns Magazine at http://wp.me/p5Tdn-LG #MDM #DataGovernance (please retweet) <<

This position is with a fast growing consulting firm, Hub Solution Designs, Inc.  It’s a challenging position with significant growth potential for an experienced person.

The Oracle TCA Business Analyst / Developer will be responsible for the successful delivery of an Oracle Customer Data Hub implementation, as well as building and managing relationships with client project managers, business owners and project teams.

Qualifications

Hub Designs is looking for a top notch person to join our team for a client project in the Southeast.  We’re looking for someone with strong Oracle Customer Data Hub and E-Business Suite Release 12 experience. This person needs to be “techno-functional” – that is, have strong skills on both the technical and functional side of Oracle.

The project will involve writing functional and technical specifications for customer data related aspects of the project.  Familiarity with Oracle’s Trading Community Architecture (TCA) and the associated Application Programming Interfaces (APIs) is a must. Excellent written and verbal communication skills are also required.

The client is in the high tech industry and will be implementing Oracle E-Business Suite R12 along with Oracle Customer Data Hub.  Significant experience in master data management, data integration, and customer data quality tools is required. Past track record in implementing Dun & Bradstreet (D&B) integration with Oracle is a big plus, as is familiarity with data governance frameworks and processes.

Next Steps

If you meet the above requirements and have the legal right to work permanently in the United States, please send your resume (in Word or PDF format) with a brief introduction including your name, complete address, best phone number and e-mail address to: info@hubdesigns.com.

You can cut-and-paste the following into Twitter: >> Reading ”Looking for Oracle TCA Business Analyst / Developer” on @HubDesigns at http://wp.me/p5Tdn-LO #MDM #DataGovernance (pls retweet) <<

Although building the business case for master data management (MDM) may seem daunting at times, it’s not as challenging as you might think. At one client recently, after we created a strategic roadmap for the first two to three years of their MDM program, we built on that foundation and developed a high level business case, focusing on expected benefits, estimated total cost of ownership and a return on investment (ROI) analysis model built in Excel.

In creating the business case, we started by identifying all the relevant stakeholders, from the business, from IT and from Finance. We met with people from Sales, Customer Care, Credit & Risk Management, Logistics / Warehouse, Marketing, Commercial Financing, Credit Scoring, Dealer Network, Finance, and with senior people in IT. All told, we met with about 35 people over a ten day period.

In each meeting, we brought them up to speed on where we were since their previous involvement in the strategic roadmap process. We shared the MDM vision with them, which they had helped to develop in a few “To Be” workshops. Then we led into the reason for that day’s meeting, which was to gather information for the business case.

In particular, we were interested in selecting metrics relevant to their area of the business. So we shared with them some potential metrics, broken out into categories such as:

• Growth (Increase Revenue / CRM Effectiveness)
• Efficiency (Reduce Costs / Increase Productivity)
• Compliance & Risk Management
• Agility & Decision-Making

In each area, we had three or four goals to prompt their thinking about metrics that might be relevant to that particular area in their part of the business.

We also wanted them to think about potential use cases, places where a Single View of the Customer would streamline their processes or help them to increase revenue or in some other way improve the business.

With each group, we took careful notes. We were up front with them that we wanted them to speak freely, and that we wanted their permission to use their names in the business case. They were so enthusiastic that they agreed readily. In many cases, the improvements they saw coming from the Customer Hub were things they had been working towards for several years. Because of that “pent up demand” factor, they had no problems putting their names next to the improvements they saw the MDM hub bringing.

We also told them that we would come back to them several times to confirm that they were still happy using their names with the use cases and metrics, and we followed up with them over the next week after the meetings via e-mail to make sure they were still comfortable with that.

We ended up with seven use cases, and in one case a Sales group stepped up to increasing their sales targets by 3% once they had the Single View of the Customer in place.

The total estimated value of the benefits, over a five year period, was over $36 million dollars, but that of course was before we looked at the cost side of the equation.

After we estimated the Total Cost of Ownership over the same five year period at about $10.3 million, we plugged the benefits and the costs into our ROI Analysis model, in order to discount the cash flows properly. Without getting into the technical definitions of the terms too much, the basics are that a dollar in the future is worth less than a dollar in the present (that’s referred to as “discounting”).

You also have a “hurdle rate” (or minimum acceptable rate of return) that the company expects to earn on all of its investments.

We ended up with a Net Present Value of more than $14 million, and a Return on Investment of more than 150%.

The methodology we follow isn’t rocket science: work closely with the business to identify metrics and use cases; negotiate improvements that they were comfortable with and would put their names on; estimate the Total Cost of Ownership; then plug those figures into an ROI Analysis model.

We found that most of the business groups needed to be guided through it in a facilitated workshop. Breaking it down for them, asking them questions using the “Five Why’s” approach – a question-asking method used to explore the cause/effect relationships underlying a particular problem and to determine the root cause – can help.

We would ask questions like: how long does it take you to gather the data you need to do your jobs? Does it help if you know who the “expert” is in a given area? What happens after you get the data? Is there a lot of manual data cleansing required? Do you have to correct the data and then re-correct it next month?

Through these types of questions, one group volunteered that 80% of their time was spent finding, downloading, formatting, scrubbing, and reconciling data. With five people in the group, we estimated that 40% of that time could be freed up, saving the company almost $100,000 per year in that one case alone.

That may not sound like a lot, but you can multiply that by a lot of similar groups around the company, and pretty soon it starts to add up to serious money. Another group we talked to employed a full-time person just to perform that same task of finding, downloading, formatting, scrubbing, and reconciling data.

Having a central repository of customer data that is accurate, complete, timely, and consistently unlocks potential revenue gains, is a huge productivity boost, and makes compliance and risk management much easier. We could have gone on further, talking to more groups within the company, but we stopped because we had enough use cases and metrics to meet our ROI target, and because we didn’t have any more time to spend on the business case.

Hopefully this “tale from the field” will give you some ideas on how to approach finding ROI and building an MDM business case.

You can cut-and-paste the following into Twitter: >> I’m reading “Finding ROI – Tales from the Field” on @HubDesigns Magazine at http://wp.me/p5Tdn-Lx (please retweet) <<.

My last two columns in Information Management magazine discussed How to Start a Data Governance Program and recommended working in the areas of people, process, technology – and information – to achieve success by taking a 4-Dimensional Approach to Data Governance.

This article focuses on how to get executive buy-in. Many times, it’s not a pretty picture. You’ll hear people say things like, “we think master data management is a nice-to-have, not a must-have”. Or if they’re really being straight with you, “it’s hard to get people excited about data”.

There are a few things going at once here. First, since I’ve been able to work with many different organizations over the years, internal people do seem to be working at a disadvantage some times. There’s an old saying that “a prophet is without honor in his own country”, which means that the old familiar people, even when they’re saying some terrific things that make a lot of sense for the business, may tend not to be listened to as well as they should.

Secondly, work on your own passion level. One of the main reasons companies bring our firm in to help them convince their management team of the need for a master data management (MDM) solution is because of our passion. It’s very noticeable, and if you can develop that same level of enthusiasm and excitement for your potential project, it will be a huge help in selling the vision to other people, including your company’s executives. Ultimately, you can’t sell what you don’t believe in, and people pay attention to that type of “burning platform” passion.

Thirdly, we can all use a brush up on our communications skills. My friend Tom Carlock wrote a great article titled So You Want to Be a Data Champion? on TDAN.com in April 2008, in which he said “you need to craft your elevator speech (an overview of an idea for a product, service or project)”, and “take time to customize a small portion of your speech to show how your vision can have a direct impact on your audience”. So work on your powers of persuasion. As hard as it is, start selling your ideas to senior management. Practice on your boss first. Then work on that person’s boss. Work from there. Eventually you’ll become comfortable meeting with and presenting to the most senior people in your company.

It can be a little overwhelming. Be as prepared as it is humanly possible to be. Most people at the senior level are incredibly smart – and they’re liable to ask tough questions, so try to anticipate as many of them as you can. Have a positive attitude, and exercise good listening habits, so that you make sure not to cut them off in your excitement to answer the question.

Constantly be trying to build bridges and relationships between yourself and different parts of the business. Executive buy-in will most likely end up coming from a consensus built from several parts of the organization, not a single overwhelming element of the business case.

So play the part of the person who brings together all the disparate parts of the business, someone who weaves together the anecdotes, war stories and proponents from all of the far flung parts of the company. That person, the gatherer of the business pain points from all over the company, plays a unique role. Make sure you quantify the pain points as you gather them, even if it’s at an estimate level only. You’ll need that for your business case later on.

What do you do with the people who refuse to buy in, the naysayers who go on the attack or who quietly sit on the sidelines, bide their time and then start lobbing bombs? For the out-and-out naysayers, try to figure out how strong their political pull is, so you can classify them as strong or weak opponents. This will help you figure out what to do about them, and how to respond to their opposition – whether to respond at all if they’re strong opponents or whether to just ignore them if they’re weak.

If people remain uncommitted, try to find out why – are there other priorities that matter more to them? Do they have a hidden agenda? Sometimes the easiest way to find out why someone is not being supportive is to come right out and ask. Find out what you’d have to change in order to get them on board, and if you can, make those changes.

A couple of things I’ve seen is when the executive wants to run the program himself, or when the person in question wants a senior role in the data governance organization. Then the opposition makes more sense, once you see it in that context.

Enterprise-wide MDM programs or the data governance organizations and processes that go with them are long term programs. The executive buy-in you’ll need is a critical step that cannot be skipped or shortchanged. You’ll only be harming yourselves if you miss getting the buy-in step right. Every subsequent step becomes much harder, to the point where your funding can dry up in mid-stream, and your project resources can disappear on you overnight.

But with this executive buy-in, it’s like having “air cover” at critical stages when you need it. You’ll find everything goes more smoothly. Getting access to critical business people will be easier; calendars will align and meeting rooms will become available. Funding and resources become available. Obstacles are removed. Project deadlines that looked dubious now start to look reasonable. The difference is night and day.

Getting executive buy-in is worth the trouble it takes to get, and in almost every case, I wouldn’t advise proceeding with a project without getting it.

You can cut-and-paste the following into Twitter: >> I’m reading “Wrestling with Getting Executive Buy-In” on @HubDesigns Magazine at http://wp.me/p5Tdn-Lq (please retweet) <<.

The Hub Designs Blog (which was founded in July 2007) evolved into the Hub Designs Magazine in April 2011, and made its debut then as an online magazine with the latest articles, news, best practices and solutions for Master Data Management (MDM) and Data Governance.

The magazine provides a rich resource for consultants and practitioners, with more than 340 articles, through coverage of vendor products, relevant research and white papers, events and conferences, plus advice from real world implementations. Content is provided by practitioners, consultants, business strategists, and industry experts. The blog and magazine were founded by Dan Power, president of Hub Designs, a global consulting firm that specializes in master data management (MDM) and data governance strategies and solutions.

Due to increased interest in Hub Designs Magazine and a fast growing readership, today we’re announcing that Julie Hunt has joined the magazine as Editor. Julie is working with Dan Power to expand subject matter coverage, bring in new writers from different aspects of the MDM and data governance world (including the buyer and end user perspectives), and to continue to provide high quality content for our targeted readership with interests in MDM and data governance.

Julie Hunt is an independent consultant and software industry analyst who works with software vendors in the strategic aspects of creating, marketing and selling software, with a strong focus on customer needs. She has worked in the B2B software industry for over 25 years, in roles from the very technical to her current practice in market intelligence and solution strategies. Julie is a member of the BBBT, Boulder BI Brain Trust, a group of analysts and consultants who are briefed by software vendors for business intelligence, analytics, data management, and enterprise data warehouse technology solutions. In 2011, she was named by Information Management as one of 17 Women in Technology You Should Be Following for data and information management. Julie is based in Austin, TX.

Dan and Julie are also founders of the Hub Designs MDM Think Tank, a group of practitioners and consultants who take briefings from software vendors for MDM, data governance and data quality solutions. Written coverage of the briefings is published in the Hub Designs Magazine. Hub Designs will be sponsoring the upcoming Gartner MDM Summit in early April in Los Angeles, CA, and both Dan and Julie will be together there to cover the event.

Everyone involved with Hub Designs Magazine would like to extend our heartfelt thanks to the many readers, supporters and contributors who have made the magazine possible and successful. Please continue to read, and let us know what we can do to better serve you. Your feedback makes the magazine better for everyone!

You can cut-and-paste the following into Twitter: >> I’m reading “Announcing @JulieBHunt, @HubDesigns Magazine’s New Editor” at http://wp.me/p5Tdn-Lk (please retweet) <<.

by Julie Hunt

It’s always great to get a briefing from a very knowledgeable and flexible founder of a young company: tapping into original vision that has been tempered by the experience of customer implementations can be a rich experience. Recently the MDM Think Tank chatted with Martin Turvey, CEO of Loqate, about location intelligence and data quality. Loqate may be a young company, but in many ways, it is not a “start-up” due to the extensive expertise of founders Turvey and Paul Flew, in the areas of address quality practices and location intelligence — and the technologies that support them. Flew’s strong NLP (natural language processing) background is also an important asset for Loqate.

For most businesses, accurate address data is essential; poor address quality costs companies tons of money and lost customers. Not only is missing address information a problem, but inconsistent address data across repositories impedes data synchronization and calls into question the accuracy of all of the data. Turvey provided an interesting point: over 80% of data has a location element (source: PBBI). This means that verifying and cleansing address data, and then adding geocoding, can greatly enhance the usability of this data.

Use cases for location intelligence include: fraud detection, insurance risk assessments, improving mailing and shipping operations, real time mapping of web content and news feeds. Loqate maintains a repository of reference data comprised of address, geocodes, languages and other data covering 242 countries and territories.

Loqate is one of the few independent software vendors for address quality / location intelligence services, since most of the other vendors have been acquired by the usual suspects. Traditional data management stacks from the large infrastructure vendors are not an option for many companies, especially in emerging markets and the larger international arenas for address quality needs.

So Loqate is positioning its independence as a competitive differentiator that allows it to target solution niches not usually covered by the big infrastructure vendors.  This positioning is strengthened by Loqate solution innovation and future direction, including an upcoming Cloud / SaaS offering. Turvey commented that most MDM solutions fail at what Loqate does well for address quality and location intelligence. As to sales model, the Loqate solution is only available through Loqate partner companies that OEM or sub-license the technology.

The components of Loqate are broken down into:  Verify, Geocode, Register, and Identify. Additional technology includes the Geo-Data Quality Engine and the Global Knowledge Repository.

In Verify, where validation and clean up are performed, powerful differentiation exists in the parsing methodology. Based on lexicon and context, parsing algorithms understand what the data should be, based on context parameters. Loqate parsing capabilities analyze and remediate data in situ, and do the hard work on single strings without breaking the string into components first. In fact, the string’s structural context provides cues for Loqate’s parsing / validation. Most other data quality tools require users to do quite a bit of hands-on mapping and other tedious work as preparation for remediation. This isn’t the case for Loqate Verify. Parsing is also trans-global for processing records and strings with mixed international character and language sets. Data can also be transliterated from one character set to another character set (e.g. Roman to Katakana in Japan)

Turvey cited a use case for Fraud Detection: cheats will use fake addresses made up of components that are valid on an individual basis (real street address, real city), but in combination the components do not add up to a valid address. Most other address quality and data quality solutions will vet the individual components but not the composite, which means they may validate a fraudulent address.

Loqate provides real value by trapping errors at data entry time when remediation is at the cheapest point. It’s also essential to trap inconsistencies at data entry to prevent fraud and block bad data from infecting processes such as shipping that can greatly impact costs.

Geocoding is available for every address data element “out of the box”. The geocode assignment results from high granularity generation of a latitude-longitude coordinate for any address worldwide. From the geocode a map visualization can be created, accurate to delivery point / rooftop levels.

Register provides smart data entry to reduce errors and speed up operational processes. This is particularly useful when working with customers, whether it’s a call center employee talking to a customer or a self-service interface for eCommerce.

Identify:  The Loqate Entity Extraction module enables location, additional geographic and contact information to be identEified and extracted from web content and other unstructured information sources. LocationV extractions enable content to be localized, driving location-based services and geographically targeted advertising.

Loqate has included the ability to quantify address data quality through its Accuracy Code analysis. A verification accuracy match score portrays the similarity between input data and the closest reference data match as a percentage between 0 and 100, with 100 as best match, as well as detailing what changes have been made to each key component during processing and determining validity of those components.

Right now, user tweaking is limited and is mainly accomplished through processing rules. There is no means as yet to tweak algorithms. Loqate is taking a slow approach here, being careful not to lead users astray, since algorithm changes can end in unexpected and harmful consequences.

Next up for Loqate is an impending Cloud / SaaS offering that should be available Spring 2012. Loqate is touting their Cloud services as “developer source for anything location”. It is also a great opportunity for further innovation since the Cloud can take software vendors in some very interesting directions.

About the author: Julie Hunt is a software industry strategist and analyst, providing market and competitive insights. Her 25+ years as a software professional range from the very technical side to customer-centric work in solutions consulting, sales and marketing.  Julie shares her takes on the software industry via her blog Highly Competitive and on Twitter at @juliebhunt   For more information, please visit Julie Hunt Consulting – Strategic Product & Market Intelligence Services. 

Liaison provides a cloud-based platform, which is a horizontally marketed solution with a strong focus on Life Sciences. The solution emphasizes a Data Integration component, which handles the movement, transformation and delivery of data. There’s a Data Management component, which handles data quality improvement through semantic data and metadata management, as well as data modeling, data cleansing and  enrichment. Finally, there’s a Data Security component, which protects data at rest and in motion, and provides compliance with  mandates.

Liaison defines MDM as “an ongoing process of continuous data improvement to better meet the needs of the business”, and approaches MDM via a “software as a service” (SaaS) approach through a lifecycle of:

• Data Integration
• Data Consolidation
• Data Management
• Data Automation

Liaison approaches MDM from a history in the product domain, but at this point, many of its customers are mastering product data and business-to-business customer data as well.

The main product component of the Liaison Technologies solution is Enterprise Content Director, which I found a little confusing, since it brought to mind a content management system (CMS). But from the discussion with Jonathan Razza, it was apparent that Liaison was doing the classic MDM functions, such as:

• Extracting data from source systems
• Audit and cleansing
• Matching and merging (survivorship)
• Conversion and loading data
• Maintaining a master repository
• Synchronizing additions and updates to the master data back to the source systems

Additionally, Liaison can do data enrichment in the cloud, focused on the customer domain. Using web services, Liaison’s product can invoke any web service to validate or enrich customer data.

Since Liaison’s product is delivered via the cloud, it makes it easier for Liaison to offer outsourced data stewardship, using an onshore model. This has become a differentiator for Liaison, as it offers managed services (in blocks of hours) to assist clients who may find it difficult to get access to their own internal IT resources when they need something done quickly.

Liaison Technologies brought its customer, xpedx, to tell us about their experiences in an end user case study. xpedx is a $7 billion North American paper, graphic supplies and equipment distributor that has tapped into Liaison’s Cloud MDM solution. xpedx was a finalist in the 2011 Gartner MDM Excellence Awards.

Scott Dickerson, who is the National E-Business Manager for Data Quality and Catalog Services at xpedx, talked to us about their experience with Liaison Technologies. xpedx had over a dozen ERP systems at one point, and over 700,000 – 800,000 SKUs (products).

xpedx uses Liaison to manage 600 product attributes, as well as to integrate data coming from its 13,000+ suppliers. Liaison built a custom web-based “front end” to their cloud offering for xpedx, and uses about 65 ontologies to classify and categorize its products.

Datapoint, the name for the custom web portal created jointly by xpedx and Liaison, turns “dumb applications” into applications capable of producing “smart data” – where the data is in a “ready to go” format for consumption within xpedx.

Liaison has about 9,000 customers across all their services and solutions and almost 40 cloud MDM clients, and like the rest of the MDM industry, is experiencing rapid growth in the interest in and demand for its product. One thing that was interesting about this briefing was that Liaison was the first MDM vendor to bring a customer voice into the conversation at the briefing, and I thought it added a lot of credibility to their message.

In the first and second segments of this series, we discussed situations where implicit values are woven into business processes or IT systems unknowingly or on purpose. The examples provided may seem trivial on the surface if one considers that everybody in the business had the implicit knowledge needed to make sense of the information. It is not so trivial when time passes, employees retire or leave, and there are implicit values scattered around the enterprise that nobody remembers anymore. Add to that the fact that implicit values, by nature, are not self-documenting and in my experience rarely externally documented. By now you should be able to identify the existence of implicit values in overloaded, NULL, or implied default values within your organization. More importantly, you should understand the potential consequences of each situation and be able to intelligently challenge those who perpetuate these practices. This last section provides alternatives to implicit designs and provides methods for mitigating risks within legacy systems.

Governance Policies are where many data quality issues get resolved or avoided. Without defined Governance, the status quo becomes the standard and data quality does not improve. Adding new policies may do little for directly combatting legacy system shortcomings, but it does set the stage for preventing the proliferation of implicit values by mandating that new interfaces to legacy systems not accept them.

An involved and informed Data Governance team is the best defense to limiting proliferation of implicit values. Introducing an enterprise-wide standard to restrict the use of optional relationships between code parent and referring child tables is a start. As stated earlier, optional relationships are ones that allow NULL child table values to exist. For example, a sales table allows a discount type code of NULL if no discount applies. To satisfy such a mandate in this situation, discount code table entries of ‘Not Provided’ and ‘No Discount’ would be added. If a NULL is received, the relationship maps to the ‘Not Provided’ entry. The Point of Sale system could now default a discount to the ‘No Discount’ code value if none was given. This solution explicitly provides a discount value in all situations and provides an easy means to identify when data transformation issues occur if a ‘Not Provided’ value sneaks in. The approach leaves open possibilities for additional entries in code tables to represent a ‘Not Applicable’ business condition or an ‘Invalid’ situation which could be set if a code value is detected during loading of the sales table that does not match a value within the domain of valid discount codes.

A Data Governance body worth its salt creates standards concerning definition, collection, and dissemination of metadata. In the policy planning phase, the Governance body can define the required column-level metadata collected to include values like: Default Value, Min/Max values, Domain, and Nullability. If business users define that NULL values are allowed, a follow-up discussion is required to determine the business meaning of a NULL value. Such a discussion is the opportune time to explore options to NULL values. Forcing the business data owners to define data elements to this degree provides the rules to eliminate several forms of undesired implicit values and it makes for better business system designs.

A strong Data Architecture group plays a role in Data Governance policy enforcement. Data Architecture groups influence database and application designers to use NOT NULL and default value constraints and to define relationships as ’Required’ instead of ‘Optional.’ Mandatory peer code reviews using checklists ensure designs meet enforced policies.

Overloading, as defined in the first segment of this series, is a costly practice that is easily avoided through proper data design techniques. Overloaded code values are avoided by creating separate child fields and separate parent code tables. Overloading of transaction metrics is handled similarly by splitting transaction values into different attributes if at the same summary level or different tables if transactions are at different summary levels. Overloaded records, like copybooks, just have no place in relational database design. If somebody tries to justify them, you have my permission to mock them in public but I am not responsible for any repercussions if the mock-ee is your boss.

The goal of this article was to demonstrate that using implicit values comes with long term consequences and this practice should be scrutinized. There are compelling arguments to use implicit design methods, but doing so should be done knowing all of the options, consequences, risks, and potential long-term costs. Resist temptations for quick-fixes when it comes to data design and focus instead on best practices.

The event is sponsored by Trillium Software. Here are the invitation details:

Is your data governance program delivering?

Having a center of excellence to govern and manage your corporate data is no easy task. Yet, a solid data governance program is critical to making your data work for your organization.

In Part 4 of our Data Governance Series: Impact You Can See, you have the chance to hear from a panel of experts who explain how to make your data governance efforts help you achieve your organizations goals by aligning data to your business.

Have a question for our experts? We’d like to hear from you. Submit your question when you register and get the chance to have one of our experts answer it!

The featured speakers are:

• Dan Power, President, Hub Solution Designs
• Jon Asprey, VP of Strategic Consulting , Trillium Software
• Lovan Chetty, Senior Manager (Product Management), Kalido
• Patrick Beatty, Director of Information Management, PwC

Please click here to register.  It should be a very interesting session!

The first segment of this series defined implicit values as ones that require additional context in order to infer an unambiguous meaning from the value. The segment went on to define how coded values are a valuable design practice that does not cause concern when each component of the implicit relationship exists within a single domain. A derivation to the coded values concept called “Overloading” was discussed as an undesirable practice where one column or record spans more than one domain. In this segment, we’ll discuss the practices of implicit defaulting and the assigning of implicit business meaning to blanks or NULL values other than the business meaning of “uh, nobody put a value here.”

From a user perspective, text fields containing a NULL or any number of blank spaces look the same. Although the casual business user cannot tell the difference between NULL and blanks, the computer treats them very differently and even changes retrieval performance based on the distinction.[1]  Suppose a data source or entry screen of a Yes/No business attribute defines that a NULL has an implicit meaning of ‘No’ because the value was not explicitly set to ‘Yes’. Since ‘No’ and NULL implicitly mean the same thing, the business cannot determine if the value was intentionally meant to be ‘No’ or if the user accidently missed data entry, but intended to select ‘Yes’. Unless somebody realizes and corrects the mistake, the value will be incorrect and nobody will ever know. Now suppose this column propagates through and ends up in an MDM system and the corporate data warehouse as Null and considered the best version of the truth. To accurately perform analyses using the column with this implied business rule, one must check for a ‘Y’ or the existence of both NULL and ‘No’. Another analysis option would be to check for a ‘Yes’ or a Not Equal to ‘Yes’ condition. In either case, a consumer of this column has to implicitly know that a NULL and ‘No’ are the same and even with this knowledge, the consumer still does not have confidence that the column was intentionally meant to be ‘No’.

Expand the above discussion to consider a lookup code value. The practice of allowing NULL code values is referred to as creating an ‘optional relationship’ within a parent (code table) and a child (reference table). Code tables are, by definition, explicit domains of allowable codes with each having a mutually-exclusive meaning for a record. Allowing NULL values for codes poses a similar ambiguity towards the data meaning. Did the user or source neglect to enter a value? Does the existence of NULL implicitly mean something else, like ‘Not Applicable’?

I can think of no positive business value to introducing ambiguity in data, ever. Moreover, there should be no compelling argument to intentionally inject ambiguity into new designs, especially in MDM or BI systems which are supposed to be reliable and pristine. When doing data integration from multiple systems, such as merging data into MDM or BI systems, the existence of NULLs and implicit values introduces several unnecessary challenges. MDM and BI systems are tasked with the collect of information from multiple sources with the intention of conforming it to a single Enterprise view. Adding implicit values to that process ensures logic will be needed in the transformation layer to conform it to the “Best Version of the Truth”. Once in the MDM Hub, more logic is necessary to transform the data back to its implicit form for each of the consuming systems.

“Oh what a tangled web we weave when first we try to cut corners and be implicit.”

In the last part of this three-part series, we will review the revelations thus far and discuss ways to both manage implicit legacy conditions and implement policies to keep from creating any additional ones.

Before you report me for animal cruelty, did you paint a mental image of me or the cat in pajamas? How do you know I wasn’t referring to the phrase used in the 1920’s to describe a person at the top of their game, Cat’s Pajamas? Would your opinion of me change if you had implicit knowledge that I was a professional photographer specializing in novelty pet shoots? How about if you knew I photographed vintage sports while wearing bed clothes or made analogies as if cars were dressed? Maybe you implicitly know I am a caretaker that lives at a zoo who carries a tranquilizer gun for safety or that I am a guide for a big-game safari hunting club? Do I need to go on? A simple factual statement like this is common in tabloid press because without proper context or additional implicit information, it is easy to reach invalid conclusions.

Computers are good at many things, but unless they are programmed to be learning machines, computers are not good at inferring context.  Moreover, computers don’t need to infer context if system designers and data owners understand how ambiguity affects their MDM and Data Governance effectiveness.

Do your business users have all the information they need to absolutely reach valid conclusions in every situation? If you said ‘Yes’ and truly understand what the term ‘implicit values’ means, you don’t need to continue reading.

Looks like all of you are still here. For the record, I was not the one that was wearing pajamas.

In Data Architecture terms, an implicit value is one that is conveyed indirectly by applying information found elsewhere. The antonym of an implicit value is an explicit one, which is a value with a self-contained purpose that exists in a single domain (which means that all values are of the same type and have the same business rules applied to them). These topics will be further discussed in this article. In practice, explicit definitions are desired over implicit ones, but there are exceptions.

This article will present the various scenarios, reasons, risks, and alternatives to having certain types of implicit values in your design and why there is literally no reason to use certain types of implicit values in MDM or BI architectures. Three classes of implicit values will be presented: Coded Values, Overloading, and Null/Default values.

Using coded values is a widely accepted and valuable relational modeling practice used to classify a value within a single domain as defined by a separate code value. For example, a GL dollar amount can be assigned to one of many GL Line Types. Two columns consisting of the GL dollar amount and the GL Line Type Code are implicitly linked together because each has no context without the other. This is an acceptable design practice because each column exists in one and only one domain: GL dollar amounts or GL Line Type Codes.

Overloaded columns or records, on the other hand, exist when one of the columns can be of a different domain based on the value of another. If the GL amount from the previous example represented a dollar amount for one GL line type and an item count for a different GL line type, the GL amount column would be considered overloaded.

This practice was done quite often in the past using COBOL Copybooks. Copybooks exist under the premise that a variable record type was an efficient way to model information within a 3GL coding environment. With the advent and advances of relational databases (and normalization standards), the copybook approach does not translate well into database tables (thank goodness).

One disadvantage of using overloaded columns is that the characteristics of a column are dictated by the superset of all the domains the column represents. This removes one of the benefits of defining a domain to explicitly enforce data quality, integrity, and definition.  For example, if an overloaded column implicitly stores either the nine characters of a Social Security Number or an account holder’s alpha-numeric 14 character Member ID, the column must be at least 14 characters in length and allow for numbers and characters. This makes it more difficult to explicitly enforce SSN formatting rules and display options because it needs some outside code or test to put the value into context.

Overloading inherently restricts a record into referencing only one type of value, which on the surface may seem like a noble goal because it guarantees mutual exclusivity. In some business cases this may never change, but in others, the intent may be short-sighted when business rules change. For instance, an overloaded column of division code and line of business code created under the premise that a record would align to only one of these classifications breaks if products realign to span multiple LOBs.

Whether the reason for using this approach was to upgrade older systems cheaply, or because a designer believed at the time that it was an innovative solution to a problem, the results are the same. Overloading makes data management more difficult, data quality harder to enforce, adds risk of incorrect use of data, and longer-term data changes become more expensive. Despite the brow-beating (or possibly because of the lack of it), designers and developers still overload columns in ERP, CRM, and custom-developed systems. Their existence in legacy systems may not be undoable without costly rework, but they have no place in good MDM or BI design (more on this later).

In the next installment of this series, we’ll discuss the use of NULLs and default values.

Not because we need to jump on the social networking bandwagon to keep MDM interesting and relevant. But because companies like Google (with Google+) and data.com (formerly Jigsaw and now SalesForce.com) have already cemented the relationship between the two.

The evolving conversation regarding MDM and social networking has two perspectives. One angle considers how to use members of a social network in the management of master data. The other addresses how to use MDM for managing data about members of a social network.

The first context for discussing MDM and social networking has been a subject of interest for some time. Crowd-sourcing is the basic social networking mechanism used to accomplish this goal. The “crowd” can be a well-defined group of employees or a coalition of like-minded members of an online service.

The second recently came to the forefront when Google chairman Eric Schmidt proclaimed that the company will only accept “real names” in Google+. According to GigaOM, Schmidt admitted in an interview “Google is taking a hard line on the real-name issue because it sees Google+ as an ‘identity service’.”

That sounds a lot like an MDM policy in the making.

Marrying MDM and Social Networking

The idea of bringing social networking and MDM together has its roots in “community data stewarding,” which some in the MDM space started touting a few years back. The idea is to leverage the collective efforts of individuals within an organization to help manage master data. More recently the notion of community data stewarding expanded to include using “the masses” for MDM.

Online data services like data.com, which is the reincarnation of Jigsaw after it was acquired by SalesForce.com in 2010, is a good example of mass data stewardship.  Jigsaw is probably the first real adaptation of social networking, specifically crowdsourcing, to managing data about people and organizations.

Even though the stated intent of Jigsaw was to share contact names, I can easily argue that Jigsaw is a social MDM application. By tapping the universe of salespeople for information about people and organizations, Jigsaw, now data.com is, in effect, the first commercially successful crowdsourced Master Data as a Service (DaaS).

Master DaaS is different than MDM SaaS, which is the delivery of MDM software capabilities in the cloud. Personally, I believe all the action is (or will be) at the intersection of Master DaaS and MDM SaaS, which is where data.com is likely heading. But that’s a conversation for another time.

One can also show that even LinkedIn is readily re-purposed as a Master DaaS, where individuals manage their own identity. Consider offerings such as “LinkedIn for Salesforce.com.” It’s not much of a leap to see how contact names in SalesForce.com are indirectly managed by the contacts themselves through their LinkedIn profiles.

While understanding how to adapt crowdsourcing dynamics to data stewardship is worthy of further consideration, this article focuses on the second context: how to use MDM to manage data about members of the crowd, i.e. a social network. This is where the conversation gets contentious, as Schmidt unwittingly demonstrates.

At first blush, all the fuss about banning pseudonyms on Google+ seemingly has nothing to do with MDM. But it does. Very much so.

Master Identity Keeper

According to GigaOM, during an interview with Andy Carven, National Public Radio’s digital editor, Google’s Schmidt stated:

“G+ was build (sic) primarily as an identity service, so fundamentally, it depends on people using their real names if they’re going to build future products that leverage that information. If you think about it, the Internet would be better if we had an accurate notion that you were a real person as opposed to a dog, or a fake person, or a spammer or what have you… So if we knew that it was a real person, then we could sort of hold them accountable, we could check them, we could give them things, we could you know bill them, you know we could have credit cards and so forth.”

Fred Wilson at Union Square Ventures further notes, Google “wants to be an identity gatekeeper,” where individual crowd members basically manage their own identities.

Schmidt and Wilson suggest what sounds like the ultimate Master DaaS for person-type parties. But it raises some interesting questions. Does a business need to definitively know the individuals that they interact and transact business with? While this may seem absurd to even ask, apparently we are at that crossroad. And from the dialogue it seems not everyone agrees on the answer.

Is MDM Evil?

To date, a business naturally assumes that a customer cooperates by exposing their identity. After all, how else can you conduct business?

That said, there are many instances of businesses that don’t know the identity of their customer. P&G, for example, cannot readily identify the consumers who purchase, say, Tide®. The “customer” represented in P&G’s business systems are retailers and wholesalers and some of their employees, but not likely the person pouring Tide® into their washing machines.

On the other hand, take a retailer like Target. Recently, they had some challenges with handling online purchases as described in this story on CNN.

“…shoppers who bought the Missoni Target line are posting on social media websites Facebook and Twitter that they won’t shop at Target again because their online orders are being delayed — or worse, canceled — by the retailer.”

Before the internet, Target did not likely know the identity of many “shoppers,” i.e. customers who made their purchases at a store, unless they had a Target credit card or exposed their identities at the cash register. And even if they did have shoppers’ identities, they would not likely have the capability to resolve the relationships among them, which is a major goal for social media sites.

In the past Target’s customers for MDM purposes were suppliers and wholesalers and not “shoppers.” Just like P&G. With online purchases, however, this is no longer the case.

Of course, Target needs to know the identity of the suppliers and wholesalers and those related contact names. The question is this. Will they ever know the identity of the supplier or wholesaler contact, who, when behaving also as a shopper, is bad mouthing Target on some social media site under a pseudonym…or is part of some anti-Target group? Meanwhile, a Target representative may just have had a seemingly pleasant lunch with the person. They’re being blindsided.

Now consider this. The blog GIZMODO states “Google’s horrible new policy on using real names in Google+ effectively means that the service is now a danger to real people.” (As opposed to fake people?) And “…the policy is evil.”

GIZMODO adds:

“You can’t use initials (even if that’s what you go by). You can’t use a pseudonym (even if that’s what you go by). And you can’t use numbers or symbols (even if they are part of your name).

Æ, e.e. cummings, Malcolm X, and T.S. Eliot would all be in violation of Google’s policy. So, too (by my reading) would be Mark Twain, George Eliot and doubly so, R.U. Sirius. I’m pretty sure nobody whose name you actually know in the band U2 can use Google+ or, by extension, Gmail.”

Play Nice

Apparently Google seeks birth certificate-quality names. Is this the same strength of policy making that MDM needs to succeed?

Traditionally, the answer is not necessarily, because any decent MDM technology will have the ability to relate multiple aliases to a person party. But as MDM moves to manage master data about your customers who are also members of business and social networks, not all aliases are willingly shared.

Everyone knows Mark Twain and Samuel Clements is the same person. And even if we did not, someone besides Twain himself would ultimately have to know his real identity. Otherwise how would he collect his royalties?

And here is the blind spot for MDM moving into the social realm. A 360-degree view will be without those structured or unstructured data relationships of a party that may only be revealed under a fantasy name…and not disclosed to you or anyone else.

Selling and marketing may get a whole lot harder as individuals increasingly armor themselves with pseudonyms and fantasy names (or personas). In response, we see the rise of strict policies enforcing identification…or don’t play on my social media site.

Unfortunately, that’s a double edged-sword, and the physics of it all has yet to get worked out. On the one hand, Facebook and Google have a threshold mass where people have to play by the rules or get booted. On the other hand, the resistive power of the masses can influence those policies. Just ask Netflix.

When a Customer Identity Is the Product

Consider Don Norman’s observation about Google+. Normanis the author of The Design of Everyday Things. He says, “Real names, they (Google) say, turn out to be the names on your driver’s license and your passport and your credit cards so that they can track you.”

He goes on to say Google’s “goal is to gather all the knowledge in the world in one place, but really their goal is to gather all of the people in the world and sell them.”

This appears very much like a goal of MDM: to gather all of the people in your business circle in one place so you can ultimately track them. All, of course, with the intent to manage a relationship with them…um sell them something.

I unequivocally say, yes, this is a goal of MDM. But there is a slight twist to the Google+ story.

Norman also submits that “Most people would say (of Google) ‘we’re the users, and the product is advertising. But in fact, the advertisers are the users and you (we) are the product.”

Typically, MDM assumes we are talking about the identities of persons and organizations like customers, suppliers, employees, partners and the like. Norman fundamentally asks, when a person’s identity is a product, do we have a right to enforce them to share that identity as a condition of use…I mean, participation in a service?

Frankly, I am not quite sure what the big deal is. Credit bureaus, like Experian and other companies like Acxiom, have sold our identities as a product for years.

Data Is Not An Asset. A Person’s Identity and Relationships Are.

According to GIZMODO, “Forget social networking, the big goldmine of the future is online identity verification.” Bingo, that’s Master DaaS.

Two questions related to MDM are at play here. (Since we are solely discussing individuals and groups of individuals …and for simplicity, I exclude master data on other entities such as products.)

1. How do you have transactions and interactions and perform analytics without, well, master data? After all, a party’s name is master data. In other words, how do you create an order or an account without header information about the party?

2. Is there an obligation for people who do business with you to correctly identify themselves? I am not sure how else business can occur. Even setting up a Swiss bank account requires identifying the account holder. Although, I cannot say for certain. I’ve never had enough money to hide from anyone!

Speaking of bank accounts, Dave Winer thinks Google “wants to effectively become a bank.” According to Weiner, knowing the relationships among persons is where the value of sites like Facebook and Google+ is. Knowing and managing the relationships among persons happens to be an important function of MDM too. We can think, then, of a MDM hub as a sort of bank of person and organization identities and the relationships among them.

If nothing else, one thing has been settled. Data is not an asset. The true identity…and corresponding relationships of a party are. It’s an asset to a person. It’s an asset to a business. Even if we mean “asset” rhetorically.

In my opinion, Google+ and other social media sites purporting to be identity services should offer an incentive like Jigsaw does. You gain points by sharing the identities of people you know. If anything, Jigsaw is the evil one. It’s a platform for everyone to “rat out” everyone else they know by disclosing their name, employer, work address, phone number and email address.

Identity and Relationship Resolution and Keeping

The concern voiced by Google resonates in MDM because a single representation of customer information and a “single version of truth” are frequently stated desired outcomes. Given the largest percentage of MDM initiatives involves resolving and managing identities and relationships, we need to reconsider what and who is a party to a transaction and interaction.

For starters, we need to rethink the notion of “customer.” It’s becoming an increasingly marginalized, if not obsolete and irrelevant concept. “Customer” seems inadequate for collectively describing the myriad roles played by all the participants in business transactions and interactions.

If abandoning “customer” altogether is too radical, we should limit it to a very narrow meaning. Take a health insurance company for example. They narrowly define a customer as the employer sponsoring a health plan. That’s it.

An employer a.k.a customer is only one of many participants in their business transactions and interactions. Other participants include subscribers, providers, members, administrators and brokers. They collectively refer to these participants as constituents, and any participant may play more than one role. A provider, for example, can also be a subscriber or member, or if the provider is an organization, such as a hospital, the provider is also an employer that sponsors a health plan for its own employees.

Additionally, we need to recognize that the collection of participants forming a crowd, group, organization, circle or whatever we call it is also an entity. Crowds, circles and groups are similar to organizations, except they usually don’t have a legal standing and tend to be virtual. Nevertheless, they are living, breathing entities that matter and require identity and relationship resolution. This is particularly the case if they are participants or potential participants in a transaction, interaction or analysis.

We can no longer easily segregate the persons and organizations we engage in a transaction or interaction into separate entities like customer, supplier, employee, partner, user, shopper, guest, donor or whatnot. It’s time to recognize these concepts and terms for what they are: role designations, not discrete entities.

The entity is the person or group. That’s why some organizations now collectively recognize all of them as a single entity and dub it a “party” or “constituent” or the like. Where the entity can behave in one or more roles as a customer, supplier, employee, partner, shopper or whatever.

Personally, I prefer “participant” to describe the individuals in a group. “Member” or “party” is probably easier on the brain, but “participant” is an action word, even if it doesn’t nicely roll off the tongue. “Constituent” works too, but sounds a little formal.

I also prefer Google+’s “Circles” to simply “group” or to, say, Facebook’s “Smart Lists” when depicting a crowd. After all, people travel in circles, not on lists! A “list” is too linear, even hierarchical sounding. “Circle” is a clean break from the chronic misrepresentation of all relationships as hierarchical. Plus, we can be participants in multiple related or unrelated circles.

“Smart group” might work too, where smart means identifiable. Then, of course, there is always “network.” A network is neither linear nor hierarchical but it’s too clinical sounding. We can always just say persons and organizations. But that’s unimaginative.

Instead of categorizing persons and organizations by their role in transactions and interactions, just designate them as participants in a circle. Some seek to distinguish social networking and business networking, but in the end, we can’t always distinguish interactions as social or business.

The following table depicts the evolving conceptual models for representing participants and circles in a system. While the goal is to move toward a Stage VI model, most organizations will have systems representing participants and circles in transactions and interactions at different stages.

Where Do We Go From Here

Deriving a panoramic view of fragmented customer data for transaction processing, business interactions and analytics involves multiple parties arranged in complex network configurations. But not all these parties traditionally and simply behave as a “customer.”

Business applications must rapidly evolve to a Stage VI model that can support different sets of relationships among persons, organizations and groups and different sets of data relationships for them. A blind spot may still exist where the identity of participants in a combined social and business circle are irreconcilable.

Until business applications can support a Stage VI model, MDM will have an interim function as keeper of participant and circle identities and relationships. However, even with MDM, separate data relationships based on participant role may not be fully supportable until business systems themselves can support complex network configurations.

Once business applications can support a Stage VI model, they still may not be suitable for managing all participant relationships for two reasons. One, if enterprise applications are not optimized for a large, complex arrangement of participant and data relationships, performance could be a significant roadblock. And two, a production transaction system may not be optimal for dedicated data stewards and curators to maintain participant identities and relationships.

For these reasons, the importance of MDM will increase as it becomes the main mechanism for managing current and historical participant and circle identities and relationships with pointers to corresponding data relationships. As cloud computing penetrates enterprises, resolving and keeping participant identities and relationships will increasingly occur as a combination of MDM SaaS and Master DaaS.

The Master Data Management Track of this conference will be the best we’ve ever had.  There’s been a separate MDM track for five years, since I joined the Education Committee of the Oracle Applications Users Group in 2006.

This year’s COLLABORATE conference will be held in Las Vegas, Nevada at the Mandalay Bay Convention Center on April 22-26, 2012.

We’re looking for papers from new people and returning speakers, on topics like Customer MDM and Product MDM, as well as other domains of MDM. Multidomain MDM – the ability to manage many different domains of data within one MDM hub – would be a great topic.  Data governance is another hot topic that people definitely want to hear about.  Here are the sub-categories that you can use within the MDM Track:

• Best Practices
• Data Quality
• Data Governance
• External Content
• MDM Hubs
• Organizational Change

We’d like a good mix of presentations from end users and consultants. This year, we want more case studies, more real world presentations, and more examples from different industries.

To respond to the MDM Track Call for Papers before the October 17th final deadline, just click here.

VERY IMPORTANT: be sure to indicate the “Master Data Management (MDM)” track (and only that track) in your entry so it doesn’t get lost in the shuffle. When we “build the room”, it’s hard to find your submission if it’s got 4 other tracks added on to it. So just stick with the MDM track, please.

If your paper is selected, you’ll get a free conference registration.

Don’t forget – click here to submit your paper before the October 17th final deadline! 

If it were so, I could redefine an entire industry with this article. The gray area between DG and MDM is real and (artificially) exists due to a lack of definition, maturity, or understanding within an organization. Considering that methodologies for MDM and DG have many variances, it would be a challenge to hit on every possible permutation of DG and MDM to define the dividing lines. Doing so would make for a long and painful read and probably be published under a self-help genre for curing insomnia. Instead, let’s generalize on the goals of each and try to find that utopian business model where there is complete harmony.

To do so, I have broken the discussion into three topics: DG, MDM, and that gray matter in between.

Governance

Data Governance organizations are business entities that define and manage the most vital corporate asset, business information. Governance organizations may vary in participation and influence, but they share common goals of corporate data policy definition, policy enforcement, and communication. DG initiatives arise from self-awareness amongst the business leadership that they create and own information and that IT serves as its librarian. Throughout the business are pockets of information, some self-contained within a single business process and some shared across many.

The self-contained “departmental” information can be manipulated, manufactured, and retired with little concern over consequences to external business processes and are of lesser concern to Governance organizations. But when information created by one business process is integral to other business processes, it becomes apparent that a lack of control introduces organizational risk. DG strives to define the structure within the organization to manage the complete information lifecycle of information deemed to be of business importance in order to mitigate risk.

The “best” or “most effective” methods to accomplish successful Data Governance can be the subject of numerous heated debates. I submit that such debates detract from the true mission of identifying what information is important, how to manage it, how to communicate about it, and most importantly, how to measure the effectiveness of your Governance efforts. If the business already realizes a need for Governance, half the effort is done. As long as there is a desire to institute and improve a process, most any dialog is a step in the right direction.

One aspect of Governance that is rarely in dispute is the identification of data owners and data stewards. Data Owners are responsible for the accuracy and completeness of the information and Data Stewards are entrusted to maintain this accuracy.  Also common is the Governance Council which defines the standards and processes to be followed by the Owners, Stewards, and Librarians. These standards and processes are company assets that become the ‘rules of law’ for all things data.  Like rules that govern acceptable behavior in our society, these rules are in place for the good of the business.

MDM

MDM is an information-centric business process to consolidate and manage specific enterprise data that just happens to use technology to assemble, merge, and distribute the data in question.  MDM arose from a need to ensure consistency of strategic shared information to improve data quality, accessibility, and security.

MDM is unique in that it is limited to specific shared information that is not transactional in nature such as: common reference codes, persons, products, or locations.  Value realized from an MDM solution occurs when information is made consistent across the organization, duplicate records are identified and resolved, and the quality of the information is markedly improved.

Achieving data consistency and quality generally requires a thorough understanding of the information at hand, how and where it is created or modified, and what roles and rules are needed to manage its data life cycle.

Methods for MDM, like DG, vary by business and business need.  Long before any MDM solution can be implemented, extensive process and information re-engineering must be planned for. Organizations that do not integrate information across departments effectively have a much harder time getting consensus during this planning process.  Despite the MDM data subject, methods, or tools used, a common practice of the planning process is to identify those responsible for the data in question and those responsible for its daily management (similar to the data owners and stewards above). For the select information within its domain, MDM should consider management from data inception through retirement and all uses between.  Roles of an MDM project include Business Analysts, Data Architects, Data Owners, Data Stewards, data providers, and data consumers.

Gray Matter

Unless my hints were not blunt enough or I’ve put you to sleep already, it should be apparent that there is significant overlap in the purpose, roles, and assets between Governance and MDM. Both processes define data elements and rules around their creation, management, and retirement.  They also both identify owners and stewards of information and place a structure around the process for ensuring data quality, security, and interaction.  So it seems simple that these overlapping roles would be one and the same, right?  Right?

In a utopian business world, the DG organization would be in place before an MDM initiative begins.  Such a DG organization would be structured with the foresight to handle an MDM initiative. In fact, in Utopia, the DG organization would be the driving force for identifying the business need and performing the cost-benefit analysis to justify such an MDM project.

Without tight coupling between MDM and DG, each initiative will see voids in their processes and fill them in order to be successful in their own right.  If DG has not instituted data standards prior to the MDM envisioning stage (or they are not followed), MDM may limit itself to what is needed to satisfy the current phase or one system. For example, “MDM in a vacuum” may have no reason to validate the business definition, domain, or business name of an attribute or list of values does not conflict with an already-established business data element.  A source system may use NULLs to indicate yes or no conditions or have other non-documented defaults that flow into the Best Version of Truth.

Without a global view of how Yes/No indicators should be handled, an MDM project could proliferate ambiguous data to all its consumers.

Another area of consideration is the logging and reporting of data errors and exceptions. In Utopia, I’m told, it is a law that each data element is assigned a data owner, a data steward, a domain of allowed values, restrictions, and a distinct definition. When data is introduced that break these laws, the violation is recorded and the owner or stewards are notified to rectify the situation. A clearly defined policy to automatically identify and report such violations along with a policy to address these issues in a timely manner ideally springs from a strong DG presence rather than an MDM afterthought.

It is expected that DG will evolve to handle new technologies regardless of whether the technology is new to the organization.  In the case of MDM, a DG organization needs to be able to address MDM concepts like trust and survivorship as well as create and expand policies around data dictionaries and canonical modeling. With tight coupling, an MDM project will look heavily to a DG organization for guidance and resources, and MDM will become another data standardization model to demonstrate the value of DG.

Summary

MDM should be considered an extension of DG.  Without proper controls and standardization of data, the worst case for an MDM project is that is becomes a waste of budget. The best case under the same lack of vision is that the project becomes a waste of potential.  Strong DG methods are undeniably needed when defining and standardizing information within an MDM solution. Without an Enterprise-wide focus on DG, an MDM solution will eventually arrive at a solution that meets the myopic needs of its immediate source/target systems, but little else. When an MDM expansion opportunity arises, the original lack of global vision will result in either a re-evaluation of the entire MDM solution or a limiting of the new audience to the initial design goals.

When considering a new or expanded MDM initiative, the first step should focus on your DG program. DG Stewards and Sponsors are the driving force for MDM justification and definition and are the true customer for MDM. Defined global controls should be finalized and introduced early into the process. Only then will you be considered a citizen of Information Utopia.

Rob DuMoulin is an information architect with more than 26 years of IT experience, specializing in master data management, database administration and design, and business intelligence.

Rahul Kamath led off with the quote “Nothing is certain but death and taxes …” which is attributed to Ben Franklin, but went on to say that it’s also true that constant change is inevitable in our businesses. A lot of companies struggle with handling changes like new sales territories, new financial accounts or cost centers, new legal entities, etc. in their front office, back office, their own organization, and their performance management efforts.

As companies face these changes, they often have to work across process silos and varying perspectives, and be prepared to handle labor-intensive merger & acquisition efforts. In the case of M&A in particular, we need to be able to onboard new companies quickly and easily, as that in itself can create a significant competitive advantage.

And all companies need to increase their agility and to comply with the applicable regulations in their industry.

When changes occur, there’s a ripple effect – usually communicated via e-mail, spreadsheets or some other unstructured mechanism, that leads to reports being updated, databases being modified, and so on.

As a result of this, a lot of times the numbers don’t match across these various systems. The underlying structures don’t match – the manual integration we were relying on never happened.

Oracle’s Hyperion Data Relationship Management (DRM) solution allows customers to collapse separate structures into a single maintenance structure, and to streamline change management processes into one place, one process and one structure. This can reduce or eliminate the reconciliation headaches that we’re otherwise confronted with.

DRM as a product allows customers to consolidate data into the DRM repository, then reconcile and rationalize that information, as well as to govern it and then to share it back out to the rest of the enterprise.

DRM operates very well in a heterogeneous environment and embraces a lot of MDM best practices and principles. It supports use cases like Chart of Accounts management, and others like mastering Sales Territories, and other customer-led innovation areas.

DRM has been implemented at over 340 customers across a wide spectrum of industries.

At this point in the session, Rahul turned it over to the customer panelists. Alex Vlaisavljevic, Data Governance Manager at Navistar (formerly International Harvester) talked about their Finance Transformation initiative, which is how DRM was introduced to the company. Navistar’s goals were to introduce new tools, provide robust financial analytics, and create a new, simplified corporate Chart of Accounts, all implemented simultaneously. DRM became the backbone of Navistar’s Chart of Accounts governance process.

Navistar’s data governance framework has the typical three level structure: executive team, management team and working groups, with a mix of IT and business resources. The management team consists of an enterprise architect from IT and a data governance manager from the business. The working group level consists of a DRM architect, a Chart of Account process administrator, hierarchy stewards, element stewards, Chart of Account approvers, IT application subject matter experts, data governance analyst, etc.

Navistar created some custom Java forms to provide a front end for requests and approvals. That front end talks to DRM, which in turn is integrated with back end systems.

Business benefits include freeing up the financial analysts from managing Chart of Account values, allowing them to focus on analyzing business results (their real jobs).

Other benefits include a centralized Chart of Accounts governance process with integrity, consistency and quality.

Lessons learned include making sure you have an in depth understanding of how data elements are used by the business before you begin the DRM implementation. Aligning key MDM, Data Governance and DRM design principles is also critical.

Angie Couron, the Director of Data Governance at VMware, discussed how VMware, a $2.9 billion company, is using Data Relationship Management to master Sales Territories in their CRM application, which is Salesforce.com.

Their territory assignment rules are somewhat complex. Creating a centralized customer hierarchy master was critical.  VMware is using Trillium as part of their data standardization for customer data.

VMware did a quick deployment of DRM in support of their Sales Territory mastering efforts, but they made sure it fit into their overall strategy first. After the initial project, they had additional Enterprise Hierarchy Use cases mapped out to look at.

Geography, industry, and others that have no home, and others that are managed on individual laptops or shared drives were good candidates to be centralized into DRM and governed by VMware’s data governance team. Other candidates were hierarchies that exists today and need data governance and a robust platform. Other hierarchies where there is a publish/subscribe pattern are also good candidates.

Interestingly, Finance has not come to the table yet at VMware.

Enterprise Hierarchy Management (EHM) is what they call their project at VMware. Enterprise Territory Management (ETM) is a real time process that replaces over 200,000 lines of rules in the old territory assignment engine.

The standardization and enrichment is based on Dun & Bradstreet data and Trillium data quality tools.

Governance at VMware is business-driven, sitting under Field Operations and Marketing. Funding is provided by the Project Review Board (a joint group between business and IT). There is an MDM Steering Committee that provides oversight.

If you try to do too much too fast, you’ll struggle – instead, pick small areas where you can show value to the business and you’ll be able to stay successful and busy. Align yourself with something that has to get out the door in order to have a sense of urgency. Don’t try to do it yourself, choose an experienced development team.

It was interesting to hear the diversity of what customers are doing with Oracle Data Relationship Management, and to see how many of them feel strongly enough about the product that they’re willing to get up and speak publicly about their experiences with it.

Oracle MDM, both as a total portfolio and its individual “MDM for Customer Data” and “MDM for Product Data” solutions grew fastest in the market in 2010. The source was a recent Gartner report (“Forecast, MDM Worldwide 2010-2015”).

There are two common solution patterns for managing ongoing information change. It’s common to have a heterogeneous environment. Pattern 1 is to leverage a data quality tool. The downside is that if this is done in a siloed way, the benefits are not provided across the whole enterprise. If there are business process changes or you want to onboard new businesses, that can be a lot of work.

Pattern 2: Add a Hub. This provides a central point of reference between the operational and analytical sides of the business, and allows for changes and new business to be onboarded more smoothly.

Oracle’s MDM Suite allows you to handle the customer domain, including social information and trusted third party data. The supplier domain is a special case of the party. Financial, site and location mastering is available. Finally, the Product and Services domain. Product and Customer comprise about 2/3 of the market, according to Gartner.

All of our hubs are layered with Enterprise Data Quality. Product and Customer data quality needs are different. Data Relationship Management product provides cross-dimensional analysis.

Data Governance and Compliance works across all of the hubs to keep data as a trusted source for the enterprise.

All of the data that gets generated across the enterprise can now be mashed up with “big data” using Hadoop Clustering, so you can understand who are your customers, what they’re doing on the social web, who are your customer’s customers, and other “big data” questions so you can get true insights on your customers.

Enterprise data is projected to grow by about 40x over the next 10 years. For companies with rigid architectures, this will be an astounding blow. Oracle’s Enterprise Information Solution leverages Oracle’s best of breed components. In addition to Oracle’s hubs, Oracle provides data integration tools such as ETL/E-LT, Data Federation, Replication, SOA and Big Data.

Oracle now has over 1,500 customers using its MDM customers, including marquee names in every vertical industry. There’s a great ROI story around British Telecom (BT). VMware is using Data Relationship Management. Cummins is here to talk about their success in the manufacturing sector. Financial Services is a fast growing vertical for Oracle. Citibank is going live with Product Hub.

The Sunshine Act has had a big impact in the Health Sciences industry.

The key drivers for Oracle customers are:

1. Revenue generation and Customer Retention (CRM booster) – cross-sell / up-sell
2. Better operations (cost reduction) – 1^st time accurate deliveries
3. IT agility – lower operations and IT costs
4. Risk management and Compliance – law enforcement, regulatory compliance and privacy

MDM benefits can be calculated using Oracle’s proprietary ROI tool, available through Oracle’s sales teams.

Oracle customers want a spectrum ranging from complete integrated solutions to complete freedom of choice, with modular components.

Oracle has invested heavily in the MDM space over the past five years. 74% of Oracle’s customers are on the most recent two releases.

Strategy and product updates include:

• Fusion MDM 11g R1 is now in General Availability, including full cloud support. The first release is focused on Customer and Product domains. In the last five years or so, Oracle has been aligning its Siebel and E-Business Suite MDM customers towards Fusion MDM.
• There are purpose-built Hubs on top of a common MDM platform, with Master Data Objects, common MDM components and services, and common infrastructure components and services.

The workflow is a standards-based Java workflow. Even the UIs are extensible purely by end users.

Oracle Enterprise Data Quality, the combination of Silver Creek for Product Data Quality and Datanomic for Customer Data Quality, is a specialized data quality solution for Customer, Product, Location, and Compliance business problems.

On the broader stage, the OEM data quality offering from Informatica is still available.

Oracle’s Enterprise Data Governance solution, allows you to define your policies, monitor compliance to them, and resolve issues that crop up over time.

Oracle’s Customer Hub 8.2+ (Universal Customer Master or UCM) has new functionality to support massively scalable hubs, enhanced data quality and usability, MDM-aware and Social MDM and increased Verticalization.

A benchmark that Oracle has run with an Australian bank showed more than 4x the throughput (using a half rack of Oracle’s Exadata and Exalogic hardware), and now supports 14.4 million real-time transactions per hour, compared to commodity hardware, assuming 90% reads and 10% writes on a 250 million customer database.

The E-Business Suite based hubs (Customer, Product, Supplier and Site) on Release 12.x are being positioned to align better with Fusion MDM.

Data Relationship Management 11.1.2.1 has new features to manage and sync via out-of-the-box integration across multiple EBS R12 instances.

Oracle’s integrations using Application Integration Architecture has been upgraded to 11g Fusion Middleware, and new integrations for both Customer Hub and Product hub are available.

Oracle’s vertical industry strategy covers communications & media, retail, public sector and higher education, life sciences and financial services.

The public sector vertical now supports Health Insurance Exchanges. Another one in this area is Citizen Services.

In life sciences, there is an Aggregate Spend Compliance solution, to assist in complying with various states’ compliance regulations in the pharmaceutical industry. This solution leverages Advantage MS data for information on doctors and health care providers.

In the oil & gas industry, Oracle has verticalized the Site Hub to become a well head management solution.

Oracle has also integrated its ATG e-commerce acquisition, to include customer preferences into the e-commerce tool.

Oracle has a group of 70+ customers on its MDM Strategy Council / Customer Advisory Board.

Oracle now has an ecosystem of 75 partners global providing global and regional systems integration expertise.

Oracle’s MDM investment and differentiation areas include the core platform, industries, business process management and integration, governance and content, making solutions “enterprise grade”, and providing thought leadership.

Oracle is committed to ongoing enhancements to all of its various MDM solutions, and will continue to make sure its solutions are enterprise grade, industry specific, and massively scalable. And Oracle as a result is experiencing the fastest growth.

Balaje Govindan, an IT Director of Data and Content at Cummins Inc., presented a case study for how Cummins has used Oracle MDM. Cummins had $13.2 billion in 2010, from 190 countries and territories, with 40,000 employees.

In its Future State, Cummins expects to have a series of web-based portals, with user-based access control, with common core applications (such as Engineering, PLM, Collaboration, Purchasing, Supply Chain, Manufacturing, Service, etc.), using middleware, business intelligence and a data integration layer connecting its multiple transactional and MDM databases and its enterprise data warehouse.

The Current State diagram, on the other hand, is a spaghetti diagram of point-to-point batch interfaces, multiple versions and instances of legacy ERP and other applications, and no significant MDM or data quality capabilities today.

The Enterprise Information Management in Cummins’ future state features shared ERP applications, with middleware connecting Finance, Manufacturing and Supply Chain applications to an MDM foundation.

MDM business drivers included:

• single source of master data
  • Better business decisions
  • Increased synergy
  • Uniform communications
  • Risk management
  • Data security, privacy, and regulatory compliance
• Operational efficiency improvements
  • Global logistics expansion
  • Reduction in redundancy in manual efforts
• Continuous data quality improvement as data quality processes are embedded upstream rather than downstream

Cummins’ future state environment includes Oracle Product Hub, Supplier Hub, Finance Hub (Oracle DRM), Party Hub (Oracle UCM for Suppliers and Customers), with Fusion Middleware as a service layer and a strong data quality tool set including Oracle Product Data Quality (formerly Silver Creek) and Oracle Data Quality (OEM solution from Informatica, including Address Doctor).

Cummins used a Four Pillar approach: Data (Information), Process, Governance and Technology (very similar to the Four Dimensional approach that Hub Designs recommends).

Cummins’ journey includes Product Hub, with 460,000 items and bills of material, Customer Hub (UCM) with 70,000 customers live (2 million more to follow), Supplier Master has a pilot implementation in progress (going live in Q2 2012, with 75,000 suppliers in scope), and Finance Hub (DRM) also live.

Key take aways include:

• Have a clear end goal in mind
• Engage business stakeholders early
• Focus on tangible business deliverables and ROI
• Build your strategy and roadmap with both business and IT
• Build your technical foundation capability in advance of your business deliverable projects
• Create a business-led data governance team progressively with clear roles and decision rights
• Partner actively with Oracle
• Remember MDM is a journey, not a short term project!

It was good to hear Cummins share their case study after hearing Oracle’s 2011 / 2012 MDM strategy and product roadmap. It always helps to have a customer explain how they’ve implemented a solution in order to “keep it real”.

“MDM should be a business driven program” is the mantra we hear time and time again. And while that is the right prevailing wisdom, doesn’t it seem that there is far more articulation about the technical elements of MDM than about the business elements?

So often I have seen MDM publications and presentations start with the “should be business driven” perspective but then quickly shift to covering the technical elements of MDM, topics related to architecture, data models, data integration, metadata management, match logic, consolidation, synchronization, application services, and so on.

Yes, those all are important elements in the MDM equation, but this over-abundance of technical insight can be frustrating for those seeking practical knowledge and guidance for the business planning of an MDM program. After all, without a strong business presence, MDM can’t develop into a core competency throughout the company. So why, in comparison, is so little insight provided about the business side of the equation?

Actually, it’s easy to understand this situation. The plethora of technical insight is a byproduct of the influence that IT and vendors have on the MDM market. And that’s not suggesting that business organizations don’t care to promote their MDM practices, it’s just that there is much more uniqueness and latency involved with getting these business components established, and these dynamics are harder to articulate.

Let’s take a closer look at these influences and dynamics.

The call for MDM

The initial need and sponsorship for an MDM initiative often emerges from the IT and/or analytical groups in a company. These groups usually exist separately or on the periphery of the business. The IT and analytical areas responsible for activities such as system integration, data migration, information management, data warehousing, business intelligence and reporting will typically be the first to expose where poor data quality and lack of data governance is creating project delivery delays, data management problems, integration issues, distrust of the data, and reporting problems.

Therefore, it’s understandable that these groups become initial proponents of MDM solutions. And while the root cause for many of these data problems lies in front-end business practices and data entry points, the IT and analytical groups are most directly impacted and naturally tend to implement their own back-end corrective actions.

At some point these ongoing problems and workarounds become a tangled mess, and that situation becomes a catalyst for seeking an MDM solution. Current state issues, improvement opportunities, and cost/benefit projections cited in MDM proposals are usually centered on correcting these legacy issues, improving data governance and information management practices, and eliminating the workaround overhead. In other cases, MDM programs often emerge when legacy applications are being replaced with more integrated platforms such as business application suites, data hubs, and enterprise warehouses where IT planning and activity is already underway.

Product vendor and consulting influence

Because of the large market opportunity and competition among MDM product vendors and consulting firms, it is expected that they will have a significant influence on the MDM information being presented. Technology and consulting help should always be factored into an MDM assessment and implementation plan. Although the overall impact and practical use of both can often be overstated, good technology and consulting help applied in the right places can definitely advance MDM capability and reach.

But many other aspects of implementing MDM involve new and unique business practices, business context, and process management where technology and consulting help will only have limited application, if any.

MDM is as much about people and process as it is about technology, which is why product vendors are increasingly looking at improving their business consulting practices or consulting partnerships as a competitive advantage. Over time this trend should help bring a better balance of information and practical insight for the planning and implementation of MDM.

Discovery and Analysis

At the onset of an MDM initiative, a “Discovery and Analysis” phase is conducted to fully detail the issues, root causes, business and system impacts, and opportunities for improvement. This discovery and analytical phase is often led by IT or an analytical team, if not a consulting group. Business leads should be actively involved in this phase, but specifically how and where business groups will be needed to drive and support MDM plans will largely depend on what priorities and direction come out of this discovery and analysis work. And where the business engagement needs are identified, the business groups will often need time to plan, budget, and prepare for these activities.

Implementing process improvements and driving data governance activity can be new or disruptive territory for business organizations. Many MDM programs do not have sufficient budgets to cover these business area needs, therefore the implementation of these business elements can take some time.

MDM job roles

IT groups already have most of the necessary job roles and skill sets needed (e.g. analysts, architects, project managers, programmers, data miners, database administrators) to initiate and support the technical aspects of MDM. The business roles needed within a MDM program (e.g. data governance leads, data stewards, process analysts, quality managers, data access gatekeepers) often do not already exist as formal job roles. Consequently these roles either have to be assigned to existing resources that have other job responsibilities and/or created through new hires, job transfers, or contracted positions which can require budget allocation and also take time to fulfill. Therefore it’s common to see IT and analytical related activity initiated early on while the business roles and needs are being addressed.

Maturity models

In MDM and data governance maturity models, we typically see the advanced levels of maturity defined using terms such as Managed, Optimized, Proactive, Advanced, or Transformational. But in many of these models, those levels are characterized by more system and analytical orientated achievements, related to systems integration, establishing a system of record, business process automation, using SOA/SaaS, quality monitoring, delivering trusted analytics and KPIs, and so on.

The business area maturity needs and milestones — associated with data governance, data life cycle management, and quality control — are much less apparent in these maturity models.

Granted, much of the program development and maturity on the business side of a MDM program is going to be tied to unique business practices that are harder to generalize in a high-level maturity model. Therefore it often takes some digging in MDM and data governance forums or finding business case studies to gain good insight on how and where the business events and the maturity of business practices have occurred in a MDM implementation.

A developing dynamic

So even though it is critical for business to be involved and provide leadership in an MDM program, defining, articulating, and achieving that is likely to be a developing dynamic that spawns from IT and analytical underpinnings where product vendors and consulting firms are also well entrenched. As more companies launch MDM initiatives and develop their MDM core competencies across their business model, we can expect to see increasing amounts of business insight and best practice information made available that will express how and where business roles and business practices evolved in MDM programs. I look forward to that.

Mark Allen is co-author of the book “Master Data Management in Practice: Achieving True Customer MDM”. Mark has over 20 years of data management and project management experience including extensive planning and deployment experience with customer master initiatives, data governance programs, and with leading data quality management practices. Mark is a senior consultant and enterprise data governance lead at WellPoint, Inc. Prior to WellPoint, Mark was a senior program manager in customer operations groups at both Sun Microsystems and Oracle Corporation. At Sun Microsystems, Mark served as the lead data steward for the customer data domain throughout the planning and implementation of Sun’s enterprise customer data hub.

Process-centric initiatives are often optimal approaches for the things that companies want to accomplish. Business processes make it possible to directly connect work to business outcomes that lend themselves to various metrics that provide clarity for business performance. On the other hand, data and data-oriented solutions, by themselves, don’t usually map directly to business outcomes, and as such, usually can’t be measured by value contributed to desired outcomes and achievement of key corporate goals.

But — business processes and data are inextricably bound together. Processes create, modify and consume data – one might even say processes live and breathe data. It is through business processes that data can be mapped to business value.

Companies generally have not taken a holistic view of business information, technologies to improve it, or its impacts across end-to-end processes or businesses, instead treating data as information islands within specific system silos.

Kalido, The State of Data Governance Maturity 2011

Notions like “Big Data” and “Information as an Asset” have stimulated thinking in many enterprises about extracting more value from the data already in enterprise repositories to benefit the business and its goals. But to extract that value, it’s not about the data nor is it about data governance. The key focus should be the business processes that impact revenue, business agility, competitiveness, and overall positive performance. However, both reliable data and effective data governance are key enablers of gaining value from business processes.

Data governance is an important strategy both to maintain data in its most usable condition and to ensure consistency in how processes work effectively across the enterprise, by sustaining the quality, accessibility and relevance of data used by processes. Instead of many siloed processes working on siloed data, data governance contributes greatly to connecting essential pieces to synchronize significant business processes to the bigger enterprise strategic picture.

Here is Martin Atherton on data governance as an enabler of business value:

For many companies, it is common to focus on the minimum standards required to achieve compliance, instead of seeking to incrementally expand the effort and reap the benefits of being compliant more broadly across the business. In a previous research note, we used the phrase ‘value driven compliance’ to highlight the difference between approaching regulatory compliance as a chore versus seeing it as a business enabler.

Some of the difficulty of transitioning how enterprises derive value from business processes (and the data running through them) is the fact that most data-related initiatives have been driven by IT instead of by more business-oriented roles.

To enable better business outcomes, data governance needs to bring business roles and IT teams together, as well as incorporating the support of various levels of management and strategic decision-makers. Too many times, data governance has not been implemented strategically. Instead it has been marooned by ad hoc decisions and “policies” tied more to daily operational situations. Without involving data governance in key corporate strategies, enterprises are not in good position to handle internal data issues, let alone address the ever-growing proliferation of data from many new sources inside and outside the enterprise.

Ross Dawson expands on the notion of “governance as an enabler”:

Governance should focus as much on enabling innovation and taking useful risks as about managing and mitigating risk… What needs to happen is that governance, risk, and compliance become part of the process of creating business value… More generally, I think it is important to note that IT governance and corporate governance have long been treated as separate. As I pointed out in my keynote this morning, technology – including in data management – is increasingly central to corporate strategy.

With good commitment from management, business units and IT teams alike, data governance initiatives are a very good way to manage data more effectively for greatly improved use in business processes. Data governance should provide the means to coordinate efficient and non-redundant business processes, along with the necessary data assets, across the enterprise, between lines of business.

From data governance coordination, standards for reliable data and optimized business processes can be implemented, maintained and changed as needed. Data governance is also an enabler of acquiring important insights into both data and business processes.

In my interactions with Forrester clients, I get the sense that data governance is receiving the most senior-management-level attention today than I’ve seen throughout my 18+ year data management career.

Rob Karel, Forrester

Tangible Value

Data alone carries little value. Unusable and poorly managed data carries no value. Real cost savings obviously result when data sources are streamlined and cleaned up, when processes are more responsive to business needs.

Great value can result when business intelligence and other processes are based on the right data that can be trusted to lead to decisions that are more accurate and timely. The tangible benefits and value of data governance and data are further revealed through their connections to business processes and the desired outcomes from those processes that connect to corporate strategies and goals.

About the author: Julie Hunt is a software industry strategist and analyst, providing market and competitive insights. Her 25+ years as a software professional range from the very technical side to customer-centric work in solutions consulting, sales and marketing. Julie shares her take on the software industry via her blog Highly Competitive and on Twitter: @juliebhunt. For more information: Julie Hunt Consulting – Strategic Product & Market Intelligence Services.

The Master Data Management Track of this conference will be the best we’ve ever had.  There’s been a separate MDM track for five years, since I joined the Education Committee of the Oracle Applications Users Group in 2006.

This year’s COLLABORATE conference will be held in Las Vegas, Nevada at the Mandalay Bay Convention Center on April 22-26, 2012.

We’re looking for papers from new people and returning speakers, on topics like Customer MDM and Product MDM, as well as other domains of MDM. Multidomain MDM – the ability to manage many different domains of data within one MDM hub – would be a great topic.  Data governance is another hot topic that people definitely want to hear about.  Here are the sub-categories that you can use within the MDM Track:

• Best Practices
• Data Quality
• Data Governance
• External Content
• MDM Hubs
• Organizational Change

We’d like a good mix of presentations from end users and consultants. This year, we want more case studies, more real world presentations, and more examples from different industries.

To respond to the MDM Track Call for Papers before the October 14th deadline, just click here.

VERY IMPORTANT: be sure to indicate the “Master Data Management (MDM)” track (and only that track) in your entry so it doesn’t get lost in the shuffle. When we “build the room”, it’s hard to find your submission if it’s got 4 other tracks added on to it. So just stick with the MDM track, please.

If your paper is selected, you’ll get a free conference registration.

Don’t forget – click here to submit your paper before the October 14th deadline! 

Recapping our series so far:

• Why Govern Master Data?
• Getting Started with Data Governance, Part 1
• Getting Started with Data Governance, Part 2
• Organizing Data Governance for Success