Andy on Enterprise Software

MDM and Spaghetti

September 14, 2009

When looking at the business case for MDM it is normal to look at the kind of business initiatives that can be enabled by better master data. For example with higher quality, consistent customer data it is possible to run more efficient marketing campaigns, or by having a complete picture of a customer it is possible to cross-sell products effectively or better manage an account. However such things tend to rely on having MDM as a piece of infrastructure, so it is hard to claim all the benefits directly for an MDM project. Perhaps it is time to take a look at some of the more murky and less sexy areas that can benefit from MDM, specifically by lowering the cost of maintaining application interfaces.

Large companies have hundreds of applications, even after they have finished implementing an ERP system (and then re-implementing it to reduce the number of ERP instances). One company I work with owns up to 600 applications, another to 3,500. In many cases data needs to be shared across applications, and of course the very fact of having so many systems can cause master data issues to occur, since each application frequently generates and maintains at least some master data that it needs rather than being fed such data by a consistent enterprise-wide master data repository.

One key difference between MDM hubs and a data warehouse is that a warehouse needs to have clean, pure data; this is achieved by an extensive process of data cleansing and validation that is conducted outside the warehouse prior to data being loaded, perhaps through a mix of data quality tools and ETL processing. Indeed one major issue is that in order to come up with high quality data for the warehouse, business rules end up being embedded in sometimes complex ETL scripts, which are opaque and hard to get business people to engage with. A good master data hub should be able to take on much of this burden of strong and managing business rules, and may be a more productive place to carry this out. For example it may be more effective to use probabilistic techniques to help determine matches angst potential data sources (say, multiple product masters) rather than needing to hard-code business rules, as usually happens with ETL scripts. If this is the case then you may be able to get away with a much smaller set of business rules in an MDM hub than were typically necessary in ETL scripts. In turn this reduction of complexity may be able to cause a lot of the maintenance effort needed in maintaining such scripts to be go away.

I have not seen any quantitative analysis out there of the relative productivity of MDM hubs v ETL processing for storing business rules, or the potential effect that this could have on the support effort needed to maintain interfaces. However it was always the case that a high proportion of overall support effort in an enterprise was associated with interfaces, so even a small effect here could have a significant saving in terms of IT costs. I think there is an opportunity here for someone to do some serious research into this area, getting hard data rather than making hand-waving benefits statements. If followed through, it would not surprise me to see this as an area where properly implemented MDM could have a significant effect on IT support costs. Given that so many people claim that making a business case for MDM is one of the biggest obstacles, this would seem to me a fruitful area of further research.