Modeling the MDM Blueprint – Part 2
digg this | 
del.icio.us | 
Reddit | 
Stumble It!
In Part 1 of this series, we discussed what essential elements should be included in an MDM blueprint. The important thing to remember is that MDM is a business project that requires establishing a common set of models that can be referenced independently of the technical infrastructure or patterns you plan on using. The blueprint should remain computation and platform independent until the models are completed (and accepted by the business) to support and ensure the business intent. The essential elements should include:
- Common Information Model
- Canonical Model
- Operating Model, and
- Reference Architecture (e.g. 4+1 views, viewpoints and perspectives).
We will now turn our attention to the first element, the Common Information Model.
A Common Information Model (CIM) is defined using relational, object, hierarchical, and semantic modeling methods. What we are really developing here is rich semantic data architecture in selected business domains using:
- Object Oriented modeling: reusable data types, inheritance, operations for validating data
- Relational: manage referential integrity constraints (primary keys, foreign keys)
- Hierarchical: nested data types and facets for declaring behaviors on data (e.g. think XML schemas)
- Semantic models: ontologies defined through RDF, RDFS and OWL
I believe (others may not) that MDM truly represents the intersection of Relational, Object, Hierarchical, and Semantic modeling methods to achieve a rich expression of the reality
in which the organization operates. Expressed in business terms, this model represents a “foundation principal” or theme we can pivot around to understand each facet in the proper context. This is not easy to pull off, but will provide a fighting chance to resolve semantic differences in a way that helps focus the business on the real matters at hand. This is especially important when developing the Canonical model introduced in the next step.
If you want to see what one of these looks like visit the MDM Alliance Group (MAG). MAG is a community that Pierre Bonnet founded to share MDM Modeling procedures and pre-built data models. The MDM Alliance Group publishes a set of pre-built data models that include the usual suspects (Location, Asset, Party, Party Relationship, Party Role, Event, Period [Date, Time, Condition]) downloadable from the website. And some more interesting models like Classification (Taxonomy) and Thesaurus organized across three domains. Although we may disagree about the “semantics”, I do agree with him that adopting this approach can help us avoid setting up siloed reference databases “…unfortunately often noted when using specific functional approaches such as PIM (Product Information Management) and CDI (Customer Data Integration) modeling”. How true. And an issue I encounter often.
Another good example is the CIM developed over the years at the Distributed Management Task Force (DMTF). You can get the CIM V2.20 Schema MOF, PDF and UML at their web site and take a look for yourself. While this is not what most of us think of as MDM, they are solving for some of the same problems and challenges we face.
Even more interesting is what is happening in semantic technology. Building semantic models (ontologies) includes many of the same concepts found in the other modeling methods we’ve already discussed but further extend the expressive quality we often need to fully communicate intent. For example:
- Ontologies can be used at run time (queried and reasoned over).
- Relationships are first-class constructs.
- Classes and attributes (properties) are set-based and dynamic.
- Business rules are encoded and organized using axioms.
- XML schemas are graphs not trees, and used for reasoning.
If you haven’t been exposed to ontology development, I encourage you to grab the open source Protege Ontology Editor and discover for yourself what this all about. And while you are there see the Protégé Wiki and grab the Federal Enterprise Architecture Reference Model Ontology (FEA-RMO) for an example of its use in the EA world. Or see the set of tools found at the Essential project. The project uses this tool to enter model content, based on a model pre-built for Protégé. While you are at the Protégé Wiki, grab some of the ontologies developed for use with this tool for other examples, such as the SWEET Ontologies (A Semantic Web for Earth and Environmental Terminology. Source: Jet Propulsion Laboratory). For more on this, see my post on this tool at Essential Analytics. This is an interesting and especially useful modeling method to be aware of and an important tool to have at your disposal.
This is hard challenging work. Doing anything worthwhile usually is. A key differentiator and the difference between success and failure on your MDM journey will be taking the time to model the blueprint and sharing this work early and often with the business. We will be discussing the second element of the MDM blueprint, the Canonical model in Part 3. I encourage you to participate and share your professional experience via the comments here.
Modeling the Blueprint for MDM
digg this | del.icio.us | Reddit | Stumble It!
Several practitioners have contributed to this complex subject (see Dan Power’s Five Essential Elements of MDM and CDI, for example) and have done a good job at describing the critical elements. There is one more element that’s often overlooked however, and it remains a key differentiator and all too often, it’s the difference between success and failure among the major initiatives I’ve had the opportunity to witness – modeling the blueprint for MDM.
This is an important first step to take, assuming the business case is completed and approved. It forces us to address the very real challenges up front, before embarking on a journey that our stakeholders must understand and support. Obtaining buy-in and executive support means we all share a common vision.
MDM is more than maintaining a central repository of master data. The shared reference model should provide a resilient, adaptive blueprint to sustain high performance and value over time.
An MDM solution should include the tools for modeling and managing business knowledge of data in a sustainable way. This may seem like a tall order, but consider the implications if we focus on the tactical and exclude the reality of how the business will actually adopt and embrace all of your hard work.
Or worse, asking the business to start from a blank sheet of paper and expect them to tell you how to rationalize and manage the integrity rules connecting data across several systems, eliminate duplication and waste, and ensure an authoritative source of clean, reliable information can be audited for completeness and accuracy. Still waiting?
So What’s in This Blueprint?
The critical thing to remember is the MDM project is a business project that requires establishing a common information model that applies whatever the technical infrastructure or patterns you plan on using may be. The blueprint should remain computation and platform independent until the Operating Model is defined (and accepted by the business), and a suitable Common Information Model (CIM) and Canonical Model are completed to support and ensure the business intent.
Then, and only then, are you ready to tackle the Reference Architecture.
The essential elements should include:
- Common Information Model
- Canonical Model
- Operating Model, and
- Reference Architecture (e.g. 4+1 views).
I’ll be discussing each of these important and necessary components within the MDM blueprint in future articles in this series, and I encourage you to participate and share your professional experience. Adopting and succeeding at Master Data Management is not easy, and jumping into the “deep end” without truly understanding what you are solving for is never a good idea.
Whether you are a hands-on practitioner, program manager, or an executive planner, I can’t emphasize enough how critical modeling the MDM blueprint and sharing this with the stakeholders is to success. You simply have to get this right before proceeding further.
Continue with Part 2.
Governing Unstructured Data Gets Easier
digg this | del.icio.us | Reddit | Stumble It!
I first discussed Varonis on this blog last August in Governing Unstructured Data.
Varonis is a very interesting software company whose flagship product, the Varonis® Data Governance Suite, focuses on governing unstructured information.
Unstructured data (i.e. data that’s not stored in a structured form such as a database and which either doesn’t have a data model or has one that is not easily readable by a computer) accounts for as much as 80% of all business information. So governing and securing it properly is a huge challenge that is only made harder by the predicted annual growth rate of more than 60%, roughly three times faster than the growth rate for structured data.
And the security threat environment is getting more challenging, with examples like Heartland Payment Systems, a large credit card processing company, which was breached in an attack in late 2008 that may have compromised more than 100 million accounts.
And serious data breach incidents are increasing, according to a study by Enterprise Strategy Group, up from 30% of large organizations (1,000 or more employees) in 2005-2007, to 56% of large organizations in 2008.
So I was interested when Varonis let me know recently they’ve released version 4.0, a major new release of Varonis DatAdvantage and Varonis DataPrivilege. The increased automation and integration means that a business can get up and running with a framework for managing, protecting and monitoring their unstructured data within hours. The product can recommend and enforce permission revocations, taking the guesswork out of assigning and revoking permissions so companies can start controlling access with consistency and regularity.
What do you think? Are you including unstructured data in the scope of your efforts to govern, manage and secure your enterprise’s information? Please let us know by commenting here or on the MDM Community.
Upcoming Webinar with Siperian
digg this | del.icio.us | Reddit | Stumble It!
Loraine Lawson from ITBusinessEdge has a good article on our upcoming webinar sponsored by Siperian.
She pointed out that Andrew White of Gartner thought the webinar raises some good questions in his recent blog article.
Andrew White said “ERP might be a good place to master data. The real question for the user is this: where is the right source of master data for you?”
Andrew is on the right track. In the webinar, we’ll outline the differences between SAP ERP and Siperian MDM, and when it makes sense to have a separate MDM hub.
I think companies should evaluate whether their ERP system is helping them solve business problems involving master data – or causing them.
To register for the webinar, please click here.
Webinar: Top 5 Reasons Not To Master Your Data in SAP ERP
digg this | del.icio.us | Reddit | Stumble It!
Siperian, an innovative provider of Master Data Management (MDM) solutions, is teaming up with Dan Power from Hub Solution Designs on a webinar titled “Top Five Reasons Not To Master Your Data in SAP ERP”.
A lot of organizations use SAP Enterprise Resource Planning (ERP) for their transaction processing, but struggle to manage their non-transactional (or master) data, including customer, product, and supplier information. These types of data require a separate Master Data Management (MDM) system – to streamline business processes, reduce costs, and increase revenue by creating a single view of the customer, product, or supplier.
Dan Power will discuss the following topics during this 45-minute webinar:
- Why SAP ERP is not the right place to master data
- Why a separate MDM system is required for streamlining business operations
- How MDM and SAP ERP coexist
- The technical attributes, strengths and weaknesses of SAP and Siperian MDM products
- The requirements of an effective MDM system and best practices for implementation
This free webinar will be held on Thursday, Feb. 5, 2009 at 11:00 AM Pacific (y:00 PM Eastern), and will include a live question & answer session.
To register, please visit http://forms.siperian.com/content/5Reasons-SAP.
Top Posts for 2008
digg this | 
del.icio.us | 
Reddit | 
Stumble It!
I thought I’d recap our “Top 10 blog posts for 2008″ for anyone who might have missed some of them.
- Ten Best Practices for Master Data Management – by far our most popular post (with thanks to MDMSource.com for featuring it)
- Our MDM Partnership Strategy — discusses our vendor-neutral strategy for partnering with the MDM hub vendors
- Different Styles of MDM Hub — outlines differences between Registry, Transaction and Hybrid style hubs
- MDM Business Case Creation & ROI Analysis — links to a “one-pager” on our Business Case Creation & ROI Analysis service
- How Master Data Management is Similar to ERP – talks about the similarities in organizational disciplines and processes between ERP and MDM
- Metadata and Master Data Management — discusses what metadata is and different approaches to managing it in the context of MDM
- Critical Data Quality Questions – outlines four “hard questions” relating to data quality and MDM and suggests ways to answer them
- Building a Data Governance Organization – five points to keep in mind as you build your data governance organization
- Five Essential Elements of MDM — technology (hub, integration, data quality, external content) and organization/process (data governance) required to succeed with MDM
- Importance of Integration to MDM — piece urging MDM project champions to think about the role and importance of data integration
These have all been read 100-800 times in 2008. If you’re interested in my articles in DM Review and speaking engagements, they’re outlined on the Publications page of our web site. If you use Twitter, you can follow me here.
Thank you all for reading this blog!
MDM: Buzz-Worthy But Not A Back-Breaker
digg this | del.icio.us | Reddit | Stumble It!
I saw an interesting post by Thomas Wailgum the other day called “MDM: Buzz-Worthy Since 2000, But Still a Back-Breaker”.
While I don’t agree that “there’s ongoing uncertainty as to when to take [MDM] seriously”, he does make some good points. The software vendors who’ve flocked to MDM and put the MDM label on everything under the sun have certainly confused the market.
Even so, the MDM software market grew 24% from 2007 to 2008. In spite of the tough economic times we’re currently in, that rapid growth rate should continue for the next several years.
One area I don’t agree with is the statement “it’s just plain hard to do … and even harder to do well”.
Don’t get me wrong, I’m not saying Master Data Management is easy. But I think it’s eminently “doable” if you:
- get yourself and your team educated on what MDM is all about and what it can do for your company
- develop a compelling MDM strategy that aligns well with your organization’s long term strategy
- get folks from the business and management on board through education, communication and evangelization
- create a strong business case and use it to manage expectations throughout the lifecycle of the project
- thoughtfully select the essential components (hub, integration, data quality, external content) and plan for data governance
- after starting your data governance program and selecting the technology components, follow some best practices for MDM implementation
Of course, there are going to be some failures along the way. But I come from the Enterprise Resource Planning (ERP) world, where a typical project was 1-2 years in length and cost in the tens of millions of dollars. To me, MDM doesn’t seem like a back-breaker. It seems like a great way of breaking down the walls of the typical corporate silos, complying more easily with ever-growing government regulations, increasing revenue by becoming more customer-centric (which in a recession, can make a big difference), and saving money through more efficient processes and consolidating out-dated systems.
What do you think? Please let us know via a comment here.
MDM Community
digg this | del.icio.us | Reddit | Stumble It!
Recently, I created an online community for everyone in the MDM community.
After attending the Fall 2008 MDM Summit in New York, and the Gartner MDM Summit in Chicago, I was looking for a way to keep that feeling of community alive.
Conferences like these are a great way to see old friends and to meet new people, to learn from our colleagues, to exchange best practices and lessons learned, and to investigate vendors of Master Data Management and related technologies.
The MDM Community is an effort to keep that going after everyone heads home.
To join, just click here. And please let me know what you’d like to see there. And I’ll need your help to make it a place that adds value for everyone.
Keynote at Oracle BI SIG Conference
digg this | del.icio.us | Reddit | Stumble It!
The Oracle Business Intelligence Special Interest Group, which is part of the Oracle Applications User Group, is hosting Desktop Conference 2008, its annual online conference, in mid-November.
Here’s a brief description:
“Join the Oracle Business Intelligence community in the only global, online business intelligence conference that addresses business intelligence and data warehousing topics related to the Oracle technology stack.”
The SIG president, Faun deHenry of FMT Systems, asked me to do one of the keynote sessions.
It’s titled “Master Data Management 101″ and will be covering:
- what is Master Data Management (MDM)?
- some useful MDM and Data Governance best practices
- what works and what doesn’t
- importance of a holistic approach to MDM
- how to get the political aspects right
- the relationship between MDM and Business Intelligence
The session will be held online on Wed. November 12th at 2:45 pm Eastern, 11:45 am Pacific. Click here to see the agenda and here to register.
Importance of Metrics in Data Governance
digg this | del.icio.us | Reddit | Stumble It!
A critical component of any Data Governance program is the tracking of data quality metrics over the life cycle of the data. When a new record enters a Master Data Management system, it does not stay static; it undergoes updates until the last transaction (and beyond).
After the last transaction, at some point, it should be purged to maintain the freshness of the data. At all these stages, the information’s quality, security and compliance can be prone to compromise. A good data governance program should address measurement at these various stages of the data life cycle. Efforts must be made to build suitable metrics, as the organization progresses through the maturity levels of its data governance program.
Here’s an example. As part of the Data Governance program, a company identified one key metric as the “number of validated Ship To addresses”. Why? Because for a significant number of deliveries, FedEx would return the package and charge the company for giving an undeliverable address. And FedEx, as part of its business process, would not let the company know what was wrong with the address or where the correction was needed.
If a company does a large volume of shipments, even a small percentage of returns amounts to a substantial cost. When a data governance program was instituted, the company ensured that for all new customers’ Ship To addresses, the Customer Hub validated the new addresses via FedEx’s web services. FedEx has an elaborate address validation and other shipment-related web services available on its web site.
The company also ensured that any other projects that touched the customer master were aware of this integration. This was published as an official data governance policy. If any other program or user attempted to update the validated address, an approval workflow was initiated. Periodic system refreshes were also developed that would end-date the validated address and create a new validated Ship To address, using U.S. Postal Service’s National Change of Address service.
For historical customer addresses, the company started doing validations of the “defective” FedEx addresses first and after that set was processed, the remaining addresses were cleansed and validated.
The most important thing to remember is that unless visibility is provided thru a data governance metric, it’s easy for management to lose sight of your accomplishments. Therefore, it’s critical to build the data governance metrics first, even before embarking on an MDM project.
October Column in DM Review
digg this | del.icio.us | Reddit | Stumble It!
Here’s a brief excerpt from my latest “MDM Insights” column in DM Review.
A recent article in the Boston Globe titled “Tougher Consumer Data Rule Adopted: Businesses Must Improve Safeguards,” described how “state regulators released new rules … ordering businesses to better safeguard consumers’ personal information.” This got me thinking about the often-overlooked relationship between master data management (MDM), data governance and data security.
Companies that don’t have MDM capabilities yet usually don’t have a data governance organization either. But it’s a critical best practice to implement MDM technology in concert with developing a data governance organization (if not already in place).
Click on “Data Security in Master Data Management” to continue reading.
And please let us know your thoughts by commenting here …
Evan Levy’s Workshop at MDM Summit
digg this | del.icio.us | Reddit | Stumble It!
I really enjoyed Evan Levy’s session at the MDM Summit on “Best Practices for MDM Delivery: Lessons from the Trenches”.
I heard some great quotes today:
- “Measure data quality levels and continually publish them”
- “Knowing that data is bad is very different from knowing how to correct it”
- “MDM is all about comparing a source system record to the hub’s golden record and asking ‘is it better?’”
- “Do a few things and do them fast.”
- “We’re probably 2-3 years aways from the hubs being really mature and supporting applications smoothly. There’s usually a big impact on how the company has deployed service-oriented architecture.”
- “Do batch first, but don’t forget to design for real-time transactional use too. Get the data problems out of the way first.”
- “A lot of shops profile data during the design phase and never look at it again. Continue to profile during development, and profile the data after it’s loaded. Even profile your data as a production activity.”
Evan’s session was interesting, with a wealth of examples from his years of real-world MDM experience.
Highly recommended!
MDM Summit
digg this | del.icio.us | Reddit | Stumble It!
In a few days, I’ll be heading to the Fall 2008 MDM Summit in New York (Sunday, 10/19/08 through Tuesday, 10/21/08). It will be the 6th MDM Summit I’ve attended and the 5th one where I’ve spoken.
I’ll be on an “Experts and Analysts Panel” with Jill Dyché, Partner & Co-Founder of Baseline Consulting and Aaron Zornes, Chief Research Officer of The MDM Institute. The panel is on the first day of the conference (Sun. 10/19) from 5:15 – 6:00 pm. For more information, go to www.mdm-summit.com/MDM/agenda.html.
If you’re interested in meeting, just drop me a note at www.hubdesigns.com/contact_us.html.
It’s always fun to meet the great people who read this blog!
What’s in a Name?
digg this | del.icio.us | Reddit | Stumble It!
As part of implementing Master Data Management (MDM) for customer information, one needs to define the “customer data model” that will be deployed in the hub.
To do this, quite often, a company will conduct workshops to get agreement on the common definition of “the customer”. The participants are all the groups or departments that touch and use customer data. These may include Marketing, Sales, Finance, Customer Service (and sometimes Legal).
The objective of these workshops is to list out the entities that are in scope for the MDM project, identify the attributes which define an entity, the possible sources of data for that entity, the business purpose of the entity and the consumers of the entity. As a secondary objective, the next step is to define the relationships among the entities and if there is any need for hierarchical representation of these relationships in the hub. But all this is definitely not an easy task to accomplish.
As an example, take the “company name” attribute for a corporation. The Sales function defines the “company name” as the name on the customer’s business card. Legal, however, needs the legal entity’s name and any alternative names, DBAs or tradestyles. Finance may want to identify the corporation with its D&B-provided name (since credit reports may use that). Tax folks may need the previous names under which this customer has transacted. Customer Service gets the “customer name” from the installed base and Marketing gets it from an external list vendor.
So there you go. These are several different potential views just for “company name”. And you thought, agreeing on the “name” definition would be easy!
Similar issues surface when defining the address-related attributes.
By now, you may be asking yourself, “So, does this end up like spaghetti, with no easy way out?”
A better approach is to gather the customer data from various systems and profile that data before the workshops. Observe the variances in “company name” from various systems and build rules based on those variances. Typos can be weeded out. Standards can be designed and proposed to eliminate the “name duplicates”. Use examples proactively. Then based on these findings and the proposed standards, conducting these workshops will be a much smoother task.
Even after this, if there is no agreement, your data model may need multiple “company name” fields to represent the “name” attribute. The objective is to minimize the number of such occurrences.
Experts and Analysts Panel Discussion at MDM Summit
digg this | del.icio.us | Reddit | Stumble It!
I’m going to be attending the upcoming MDM Summit in New York on Sunday, 10/19/08 through Tuesday, 10/21/08.
I’ll be on an “Experts and Analysts Panel”, moderated by Jim Ericson, Editorial Director of DM Review, along with Jill Dyché, Partner & Co-Founder of Baseline Consulting and Aaron Zornes, Chief Research Officer of The MDM Institute.
The session will be on the first day of the conference (Sun. 10/19) from 5:15 – 6:00 pm, followed by the opening night reception in the exhibit hall.
I’m looking forward to it – Jim is really sharp, and I always enjoy hearing Jill’s and Aaron’s perspectives on the MDM space.
For more information, go to www.mdm-summit.com/MDM/agenda.html.
Customer Data Quality
digg this | del.icio.us | Reddit | Stumble It!
Quite often, an enterprise faces an event where it needs to load massive customer files into its enterprise information systems.
Examples include integrating a new subsidiary’s customer master with the parent’s CRM or ERP system, migrating to a brand new ERP, consolidating customers from various silos within the enterprise, importing partner files and their customers, etc.
Sometimes, attempts are made to programmatically improve data quality within a customer record, but because of tight deadlines, data quality across the file is usually not given serious attention.
IT’s thinking is usually that “We received 50,000 customer records; we uploaded 50,000 records – job well done!” But wait a minute, is that really true?
It is highly likely that duplicates exist within the file and the same customer is being loaded more than once. There’s also a possibility that the same customer already exists in your target system.
Multiple instances of a single customer can lead to end-user confusion, serious reporting errors and even to reduced efficiency and impacts to customer service.
A good approach is to be proactive about data quality and to plan for spending extra cycles correcting these types of problems in the customer files before doing the migration.
A simple tactic is to extract the existing customer records from the target system and run this file along with the legacy / source system data through an address validation and matching process. A number of vendors can do this task for you at a reasonable cost, ranging from 15 cents to 55 cents per record.
The next step is to separate the non-duplicates and load only these records in the target system. The duplicates are either managed outside the target system (by building cross-references in your data warehouse, for example) or, if your target system has a way to maintain cross-references, by uploading the cross-references only into the target system (typically an MDM hub or ERP application).
A major benefit of this approach is that the new records are genuinely new and have validated addresses for deliverability. This significantly enhances corporate data quality. Then, IT can say “We received 50,000 customer records; we uploaded only 40,000 records, the other 10,000 were duplicates – job well done!”
Announcing an Intensive, Two-Day On-Site Seminar
digg this | del.icio.us | Reddit | Stumble It!
Attaining High Quality, Integrated Information
High quality, integrated master data is all that matters in your business/IT landscape. Sure, people, processes and technology are important. After all, they get the brunt of everyone’s attention. But in the end, superior business intelligence, smooth transactions and harmonious customer interactions all depend on the quality and usability of your master data.
Why This Seminar Program?
Hub Designs and Perera have teamed up to create this important, two-day on-site seminar program for organizations struggling with low quality, fragmented enterprise data. With over 40 years of combined experience, you can expect a new level of practical insight that is unavailable in any other forum.
Our goal is to bring unparalleled Master Data Management expertise to your front door. A blend of education and hands-on guidance, your organization will gain the knowledge to confidently undertake and succeed with your MDM initiative… and transform your enterprise master data into an appreciating corporate asset.
Agenda For Mastering Your Data
Together, we will explore the issues, challenges and opportunities associated with creating and maintaining high quality, integrated enterprise master data:
- Creating a business case for managing customer, product, supplier, financial and employee master data
- Analyzing the types, nature and severity of enterprise data quality problems
- Determining quality and integration requirements for enterprise master data
- Creating enterprise master data architecture and models
- Formulating a plan to correct and transform your existing enterprise master data
- Developing and embracing master data content and format standards
- Integrating and synchronizing master reference data within and across enterprise systems
- Identifying, evaluating and selecting MDM software and third-party data sources
- Designing data quality processes for continuous master data management
- Determining metrics for assessing, monitoring and certifying the quality of master data
- Organizing and managing a data governance and stewardship program
This program is geared to business, project management and IT personnel who are actively involved in Master Data Management (MDM), Customer Data Integration (CDI) and data quality initiatives. The ideal session brings together up to 15 participants from your organization to discuss the production, distribution, consumption and maintenance of enterprise data.
By conducting this program at your site, stakeholders have the flexibility to join program segments that are appropriate to their functional areas. We charge a fixed program fee so you can tailor attendance to your needs.
Schedule TODAY!
For more information or to schedule this two-day program at your location, please call us at 781-749-8910 or visit our web site.
Getting to the Single View
digg this | del.icio.us | Reddit | Stumble It!
If not Master Data Management, what?
Enterprise Resource Planning (ERP) – the “back office” – has been around forever, and the “customer master” function in most ERPs is adequate, but due to acquisitions, many companies have more than one ERP system, and some companies let major business units build their own separate technology architecture.
Customer Relationship Management (CRM) – the “front office” – was supposed to be a “silver bullet”, bringing businesses closer to their customers, delivering 1-to-1 marketing, and increasing sales.
And data warehousing and business intelligence were supposed to deliver performance management and analytics, enabling better decision-making and deep analyses, but have sometimes proven to be difficult to deliver and extend.
But to varying extents, all of the technologies failed to deliver on all of their promises.
So circa 2004, along came Customer Data Integration (CDI) and Master Data Management (MDM). I call it the “hole in the donut”. MDM takes information from source systems like CRM and ERP, and eventually passes it on to downstream applications like data warehousing and business intelligence. But a lot of magic happens in that “hole in the donut”.
Information is consolidated into an MDM hub, usually using service-oriented architecture based integration technology. It’s cleansed using data quality software and completed or enriched with third party information. And it’s managed by a data governance organization. For more details on the end-to-end MDM process, see our earlier post on the “Five Essential Elements of MDM“.
So that would give you the Single View of the Customer (or Product, or Supplier, or whatever data domain you were mastering).
And from there, most companies would, in fact, flow the consolidated / cleansed / completed information into a data warehouse or business intelligence application.
But if your MDM hub is missing, and you don’t have the data governance organization or processes, all of the above is going to be much more difficult, if not impossible.
Organizations are waking up to this, realizing that they’ve got “the donut” i.e. key pieces of the puzzle (plenty of source systems, decent integration technology, tons of third party data) but no data quality tools and no central MDM hub.
If you want the Single View (the “whole donut”), you need to invest in those missing pieces.
Structured vs. Ad Hoc Data Governance
digg this | del.icio.us | Reddit | Stumble It!
I was struck recently by the difference between companies that have a formal, structured approach to data governance, versus an informal, unstructured or “ad hoc” approach.
In many cases, companies with an ad hoc approach already have the right people, in the right places, doing the right things.
But it’s not formally part of their job description. They just do it because they know it’s the right thing to do, or that the company really needs it.
So they act as unsung heroes of data stewardship, cleaning up data manually, writing scripts to make data corrections in bulk, even working together in teams to do data governance tasks, without ever formalizing it into a data governance program.
I wrote yesterday about whether data governance should be located in the business (with support from IT) or in IT (with support from the business). It’s a natural tendency of business people to think that data management, since it involves computers, should be part of IT. And it’s a natural tendency of the IT people to think that only the business knows the subject matter well enough to manage it.
But wherever you stand on this question, I think it’s better to have a structured approach to data governance. Set up a data governance committee or team, define its mission and processes, and give them the technology tools they’ll need to achieve the mission.
Relying on an ad hoc or informal approach is risky. People take new jobs, go on vacation, or get burned out. So you can’t rely forever on the unsung heroes of data stewardship.
I’ve said many times that if companies treated their physical assets (like inventory or cash) the same way they treated their information assets (particularly customer data, for some reason), then people would be going to jail.
Start thinking about how your organization can improve its data governance maturity, or start a data governance function, if you don’t already have one. You’ll find that “when the student is ready, the teacher will appear”. In other words, once you start, if you remain diligent and patient, the rest of the organization will ultimately see the value of adding data governance to “how we do things here”.
Here are some good resources for further reading:
- “So You Want to be a Data Champion?” by Tom Carlock
- Wikipedia article on Data Governance
- The Data Governance Institute’s Data Governance Framework
- The Master Data Management Institute
- Data Governance Blog
Please let us know via a comment if you have any other resources on data governance you’d like to suggest.
Where Data Governance Belongs
digg this | del.icio.us | Reddit | Stumble It!
Both IT people and business people usually realize when data management issues are having an impact on the company. And senior executives are usually at least aware of the issues with important master data domains like customer, supplier and product, because they live with the end results of data quality issues every day.
But sometimes the business is reluctant to hire anyone to work on data quality or data governance. So here’s my question: is it better for the IT team to take that on, if the business doesn’t step up to the plate?
I usually recommend that Master Data Management (MDM) and data governance programs be driven by the business, and in a perfect world, that probably is the best route.
But even if the business is driving, they usually need a lot of IT support. And if the business doesn’t want to take on the issue at all, perhaps it’s better to have IT doing it than have no one doing it.
Please share your thoughts via a quick comment here.
Governing Unstructured Data
digg this | del.icio.us | Reddit | Stumble It!
I had a very interesting briefing the other day by Johnnie Konstantas, the VP of Marketing at Varonis.
Varonis is a software company that focuses on governing unstructured data. Johnnie’s perspective was pretty illuminating:
- A 2007 IDC study found unstructured data accounts for over 80% of all business data
- In 2008, a study by the Ponemon Institute found that 84% of organizations believe their unstructured data is accessible by people with no clear business need and 32% have experienced an unstructured data breach
- The IDC study also found that data grows at a rate of 57% per year
There have been high profile stories lately about unauthorized people snooping in presidential candidates’ passport files, the theft of 94 million credit cards from TJX, and the exposure by an investment firm of data on 2,000 clients, including Supreme Court Justice Stephen Breyer.
And in June, The Identity Theft Resource Center reported that nearly 16 percent of breaches this year came from insiders, up from 6 percent in 2007.
Given that many of the people I talk to or work with are building Master Data Management solutions for their companies, or putting together a Data Governance program, I had to stop and ask myself “maybe we’re all missing the forest for the trees here”.
Granted, the picture on the structured data side of things needs improvement too. Companies still struggle to pull together the “Single View of the Customer”. Islands of data still exist, and artificial silos still cost companies money and hurt productivity.
But I think we ignore the unstructured data problem at our peril. I believe savvy business owners will eventually expect an integrated approach to governing both structured and unstructured data. Even though the technology tools might be quite different, a common organization and policies addressing both types of data will be necessary.
It’s not enough to lock up the structured data, when over 80% of the information in the company is unstructured and is not adequately protected or managed.
I haven’t had a chance to thoroughly research Varonis and its products yet, but it looks like a unique way to govern the unstructured data on file systems, and I’m impressed by the company’s approach.
One Year Anniversary of This Blog
digg this | del.icio.us | Reddit | Stumble It!
We’ve been writing this blog for a year now, with a total of 83 posts so far.
It’s been a very positive experience. We’ve had clients tell us “the reason we hired your firm was because of your blog”. And we’ve gotten lots of great feedback from our partners (Oracle, IBM, SAP, Initiate Systems and Siperian).
What we’ve tried to do is to write for people who are new to Master Data Management (MDM) and looking for basic information (like “Useful Definitions for MDM”,“Five Essential Elements of MDM” and “Ten Best Practices for Master Data Management”).
But we’ve also tried to cover more advanced topics too (such as “Master Data Management and the Art of Politics”, “The Key Requirement in Choosing a Product MDM Hub”, and “Data Governance Critical to MDM Success”).
We thought that by presenting a mix of basic and advanced topics, and highlighting key milestones in the development of the firm, we could keep your interest, and hopefully keep you coming back.
The numbers tell a good story. We’ve had a total of 8,100 hits in the past year, with an average of 32 hits per day (over the last 30 days), 200 hits per week and 835 per month (over the last 6 months).
Our “Top 10″ posts have been:
- Ten Best Practices for Master Data Management
- Our MDM Partnership Strategy
- How Master Data Management is Similar to ERP
- Different Styles of MDM Hub
- Metadata and Master Data Management
- Five Essential Elements of MDM
- Critical Data Quality Questions
- The Key Requirement in Choosing a Product MDM Hub
- Master Data Management and the Art of Politics
- MDM Business Case Creation & ROI Analysis
We get most of our traffic from our web site at www.hubdesigns.com (there’s a prominent “Blog” link there), and from the “Master Data Management” and “Customer Data Integration” tags at WordPress.com. We also get a fair amount from Google Reader, My Yahoo, and my LinkedIn profile.
Our Top 10 search terms that people are using to get to the blog are: “Hub Solution Designs”, “Dan Power”, “Gaurav Arora”, “data quality questions”, “MDM vendors”, “Master Data Management best practices”, “critical to quality”, “Oracle MDM”, “ERP and MDM” and “Master Data best practices”.
We’ve tried to keep the blog vendor-neutral, and have resisted the temptation (so far at least) to accept any form of advertising.
In the coming year, we’re looking forward to more in-depth coverage of the leading MDM and data quality platforms, more insights gleaned from working with our clients, more pointers to other places where our writing appears (like my monthly column in the online edition of DM Review), and continuing to try to break new ground and be thought leaders on MDM.
If there’s anything in particular you’d like to see us cover here, please let us know via a comment. It’s been an honor to write for you over the past year, and we’ll work hard to make this a useful resource for you in the coming year.
July Column in DM Review
digg this | del.icio.us | Reddit | Stumble It!
Here’s a quick excerpt from my latest “MDM Insights” column in DM Review.
It’s a long journey from the first efforts of “customer cleanup” to a full-fledged data governance program. But that’s where many companies start. They gradually accept that there are issues with their customer data such as:
- A lack of consistently applied standards and controls,
- Problems arising from conversion of customer data from acquired companies,
- Lack of ownership of customer data,
- Invalid addresses leading to undelivered and returned mail or
- Customer service problems caused by large numbers of duplicate and inaccurate records.
So they form a committee, hire a consulting firm, and involve their internal IT folks. That’s a great start, but it’s important to realize that this is not a once-and-done project.
Click on “From Customer Cleanup to Data Governance” to continue reading.
And please let us know your thoughts by commenting here …
DM Review’s E-Book on MDM Best Practices
digg this | del.icio.us | Reddit | Stumble It!
DM Review magazine has an excellent e-book on “Best Practices in Master Data Management”, with a cover story on “Project Management Challenges with a Changing Landscape” by Hub Solution Designs.
You can access it at http://sm.ebookhost.net/dmr/mdm/1/ or http://sm.ebookhost.net/dmr/mdm/1/ebook/1/allcontent.pdf.
Our article begins on page 4.
Please let us know what you think by commenting here.
Building Integration using SOA
digg this | del.icio.us | Reddit | Stumble It!
Many companies are still deploying Enterprise Application Integration (EAI), with proprietary adapters and integration servers. However, for a Master Data Management (MDM) solution, we recommend a Service-Oriented Architecture (SOA) approach for integration between the hub and source systems using web services.
A typical web server provides Hypertext Transfer Protocol (HTTP), so Web browsers can receive pages from a web site.
Application servers provide the Simple Object Access Protocol (SOAP) interface and host the web services. The web services also provide object components, which provide the business service layer above the applications.
The development time for SOA-based MDM integration will depend on the number of business entities to be exchanged, the availability of a vendor-supplied Software Development Kit for the Web Services Definition Language (WSDL), the complexity of the applications to be integrated and the number of Web services to ultimately be deployed.
Some guidelines for developing an SOA integration for an MDM hub are:
- Use XML (eXtensible Markup Language) for all data exchange (XML is a language that provides a standard way of representing data and information).
- Use UDDI (Universal Description, Discovery, and Integration) for listing and locating applications. UDDI is a directory standard that is provided by some application tools as a built-in service to use during integration.
- The WSDL (Web Services Description Language) file should be obtained from the source system to which data needs to be sent or retrieved. WSDL is a “descriptor standard” that an application uses to describe its interface and interaction rules to other applications. WSDL is a document written in XML which describes a Web service. It specifies the location of the service and the operations (or methods) the service provides.
- WSDL should be leveraged with the help of proprietary tools provided for each application to generate the XML message required to meet that data structure.
Currently, some of the Master Data Management platforms (such as Siperian, Initiate Systems, Oracle and IBM) provide excellent SOA libraries of web services.
With some work by the end customer, these products can provide a standard set of data services at the application level. We believe this approach ultimately will give you more flexibility and adaptability than EAI-based integration.
Building a Data Governance Organization
digg this | del.icio.us | Reddit | Stumble It!
I’m often asked how many data stewards a company needs.
A rule of thumb I’ve used for quite a while is “one data steward per $1 billion in annual revenue”, but of course that can’t be a hard and fast rule.
Depending on the company, the degree of automation, what data quality tools are available, and the major processes around the enterprise, that number is going to vary.
Here are some thoughts to keep in mind as you build your data governance organization:
- Build the business case and secure your funding. Don’t forget this is a multi-year effort. Several times, I’ve seen data governance teams and MDM initiatives that never thought beyond their first year’s funding, and who were surprised when funding for Year 2 and beyond wasn’t forthcoming.
- You’ll probably need a multi-layer, cross-functional approach. Start with a high level group that sets data policy, usually called a Data Governance Steering Committee or Data Governance Council (DGC). Your executive sponsor(s) should be involved with the DGC. Then you’ll probably have a small group of data stewards which handle day-to-day data management, which reports to the Data Governance Council.
- Focus on your data governance processes, including relevant metrics that show the impact to the business, and on communicating those metrics and their impact to the rest of the enterprise. Create a communications and education strategy to get the word out.
- Look for IT people who “speak business” and business people who “speak IT”. These liaison-type people are going to be crucial to your data governance organization. Although I believe that data governance should be driven by the business, there’s no question that you’re going to need a lot of support from IT. So look for the “bi-lingual” folks in your company – they’re out there and they can be incredibly valuable in understanding the business impact of data issues, and in providing the technology support that the data stewards will need.
- Don’t forget about metadata. In your travels as a data governance organization, you’re going to run into a lot of metadata (information about information). Develop a disciplined approach to cataloging and maintaining the metadata you come across. In many ways, that will probably be one of the most useful outputs of your data governance initiative.
We’ll have more to say on data governance in this blog in the future, but let me leave you with one idea. Instead of describing your MDM initiative as (hopefully) including data governance, think of it as a data governance initiative that has an MDM component.
You’ll be doing data governance in your enterprise long after your current MDM hub implementation is over.
If you don’t agree, or you have a perspective on data governance you’d like to share, please comment here on this blog.
June Column in DM Review
digg this | del.icio.us | Reddit | Stumble It!
Here’s our latest “MDM Insights” column in the online edition of DM Review.
The technology industry is the mother lode of acronyms, and in the field of master data management (MDM), there are more than a few. We have the subsets of MDM known as customer data integration (CDI) and product information management (PIM). And in the closely related fields of middleware and integration, we’ve got enterprise information integration (EII); enterprise application integration (EAI); extract, transform and load (ETL); service-oriented architecture (SOA), among others. You get the point. Today, the acronym I’d like to focus on is business process management (BPM).
Click on “Master Data Management and Business Process Management” to continue reading.
And please let us know your thoughts by commenting here …
Importance of Integration to MDM
digg this | del.icio.us | Reddit | Stumble It!
We’ve discussed many other topics on this blog, such as data quality, best practices for Master Data Management, the five essential elements of MDM, and Master Data Management and the art of politics.
But one topic I don’t think we’ve given enough “airtime” to is integration. There are many different types of integration technology available today, and a veritable alphabet soup of acronyms to go with them.
There’s Extract-Transform-Load (ETL), Enterprise Application Integration (EAI), Enteprise Information Integration (EII) and Business Process Management (BPM).
In an upcoming piece in the online edition of DM Review magazine which I wrote last week, I go into more detail on the different types of integration and why I think Business Process Management offers some real advantages, due to the close fit with Service Oriented Architecture (SOA) and its flexibility and ability to model complex business processes that span multiple application silos.
But as I said, that’s a topic for another piece, and we’ll post a link to that here when it becomes available later this month.
In today’s piece, I want to urge MDM project team leaders, program managers, and “data champions” to think about the importance of integration itself, and the existence of certain typical requirements.
While integration doesn’t always have to be real-time, if you find yourself thinking solely in a batch-oriented mode, take a step back and ask yourself, “what will we be giving up by not providing for any real-time capabilities?”
And while straightforward, point-to-point XML data exchange may be all you need, ask yourself if you’ll be giving up anything important by not being able to model, deploy and manage business processes.
And try not to limit yourself by planning only for one-way integration into the hub. As hard as it is to convince the business owners to subscribe to the data quality improvements and external content that you typically do in an MDM hub project, you’ll make it much harder to achieve your expected ROI for the MDM initiative if the source system business owners don’t receive any of those improvements.
What are some of the integration “sticking points” you’ve run into on your MDM projects?
OAUG Collaborate 08 Presentation
digg this | del.icio.us | Reddit | Stumble It!
TechTarget recently had an article that quoted heavily from my presentation at the recent Oracle Applications Users Group (OAUG) conference, COLLABORATE 08.
The article discusses the Master Data Management program at Tektronix, Inc. and segues into an overview of MDM based on my presentation at the conference.
Here’s the full link:
http://searchoracle.techtarget.com/news/article/0,289142,sid41_gci1312492,00.html
Please let us know what you think of the article by commenting.
Tracking and Managing Corporate Hierarchies
digg this | del.icio.us | Reddit | Stumble It!
Here’s a great video that illustrates the difficulties in tracking and managing corporate hierarchies:
As the video shows, in today’s acquisition-heavy environment, it’s pretty tough to keep track of all the M&A activity in your customer base. As they say at the ballpark, “you can’t tell the players without a scorecard”.
Having worked for D&B in my previous life, I know they do a good job in maintaining the most important corporate hierarchies, with over 8.7 million company records linked to the appropriate family tree. D&B’s not perfect, but they do try pretty hard. If corporate hierarchies are important to your MDM initiative, think about including D&B early in your planning process.
And thanks to my friends Jack Dally from Transitions [2], and Mani Kumar Manda from Rhapsody Technologies for pointing me to this video.
Next Week’s DIG Conference
digg this | del.icio.us | Reddit | Stumble It!
I’m really looking forward to speaking at next week’s Decisions, Information and Governance conference in Las Vegas, sponsored by The Palladium Group.
And I spoke earlier this week at the New England Oracle SOA Users Group, talking about Master Data Management as a foundation for Service-Oriented Architecture.
MDM initiatives seem to be getting linked to Service-Oriented Architecture or to advanced analytics and business intelligence programs.
I think there can be a problem (but also an opportunity) for MDM in inserting itself between two things that used to talk directly from one to the other (an ERP system to a data warehouse) or (b) asserting itself as a predecessor task to ensure a better outcome (for example, when MDM is used to consolidate and improve the quality of enterprise data before people try to use it in analytics or business intelligence).
While I think it’s true that MDM is in fact needed at most large organizations, having to coordinate with an already-underway SOA initiative, or step back from a planned BI initiative and first tackle MDM, does complicate things a bit. So that’s the “problem” part.
The “opportunity” part is that, for organizations that have the foresight or the luck to tackle MDM first, it makes implementing SOA or achieving business intelligence success that much easier. There’s already a centralized repository of information on customers or products (or whatever domains have been mastered), and that information is proactively managed so that it’s trusted to be accurate, complete, timely and consistent.
Whichever situation your organization is in (tackling MDM first or building it into something else like SOA or advanced analytics), spend the time to develop a workable MDM strategy, using a holistic approach that addresses people, process, technology and information. By all means include an MDM hub in your planning, but make sure you also plan for business process management or sophisticated integration, as well as built-in or bolted-on data quality and enrichment capabilities. And be sure to build a data governance framework around your MDM initiative.
I’ll write a trip report after next week’s DIG conference, to let you know what I thought of the conference itself and whether I got lucky at the tables!
Keys to a Successful MDM Program
Master Data Management (MDM) initiatives often seem to begin with the CIO and consequently, the implementation takes on a strong technology focus.
But in today’s article, we want to suggest an approach that’s more likely to succeed in the long run – tying the MDM project to solving an important business problem, and then getting the business to not only sponsor the initiative, but to “own” it.
Depending on your industry, there are key business drivers frequently seen in that industry. For example, in manufacturing, the key drivers are usually margin analysis, supply chain analysis, product profitability and customer satisfaction. In the software industry, license revenue analysis, maintenance contract revenue (new and renewals), support margins and customer satisfaction are the key drivers.
When you talk about how MDM may improve results in these areas, the business owners perk up and listen. So invest some time in understanding the corporation’s strategic priorities for the next few fiscal years, and then choose a small number of these strategic priorities as the key drivers to be tied to MDM.
At a leading software company, Marketing had recently undergone a radical overhaul. The new head of Marketing was swamped by the number of “mini-databases” that had sprung up, both within the department itself and within IT. For their launch of their new software product, he needed to know who his customers were – on a particular version, at a particular support level, and in a particular geography.
It took the Marketing department weeks to get that final list. As a result, the CIO stepped up and linked the MDM initiative’s success to specific metrics used by Marketing.
Marketing was then totally engaged in the MDM project, and that momentum carried right through the product launch. And Marketing even hired a data steward for ongoing data management.
Had it only been the Technology group carrying the burden of doing the MDM project, I’d bet the project would have fallen by the wayside and there would not be any surviving MDM program there.
The key takeaway is to link your Data Quality and MDM initiatives to your enterprise’s key business drivers and your executives’ priorities. Only then you will get the business to put their money where their mouth is. And only then will you be assured of a successful ongoing MDM program.
MDM and Data Governance – the Value of Planning
digg this | del.icio.us | Reddit | Stumble It!
Building a “Single Version of the Truth” can be more expensive than you expect, and documenting and measuring its ROI requires careful thought.
Good planning is more necessary than ever in an uncertain economic climate. The result of inadequate planning can be misdirected spending, chewing up valuable time and resources and then, six or twelve months later, having to go back and “right the ship”. And the second time around, the efforts are often overstaffed (to “make up for lost time”), while the organization as a whole might still be marching down the wrong path.
Master data management and data governance initiatives can have a disruptive effect on the organization, and the budget is often millions of dollars. Now the stakes are even higher, because in times of economic uncertainty, the pressure is on to “do more with less” and to take shortcut approaches for achieving corporate data objectives.
One such shortcut is to turn the MDM initiative into a “technology-only” project, perpetuating a “silo” approach to data and selectively purchasing the latest data quality or hub tools. This approach should be used with caution, because at the end of the day, data is still data, and without process and stewardship, even the latest technologies will probably fail to meet the intended objectives.
Because of the organizational effects (new processes, roles and responsibilities) in MDM and the budget requirements, our advice is to take the time for a readiness assessment, understand where on the maturity curve you are, see if your business drivers make a sufficient case, think through cultural issues, etc.
The results of an assessment may surprise you. Even with a strong business case and senior management buy-in, don’t underestimate the amount of preparation and time that a well conceived planning process for MDM and data governance will take.
