A Burning Platform

I was amused by a piece regarding data quality in which a data quality initiative at a chemical manufacturer was kicked off only after a warehouse burnt down and the company discovered that they had no way of tracing which customers would be affected. I recall a similar example at Shell, where a data quality initiative received a serious management boost when an oil well was drilled into an existing well (fortunately it was not in use at the time) due to faulty positional data in a computer system.

These kind of incidents demonstrate that management care about data quality when there is a crisis, but when things are running smoothly it is usually a long way down the priority lists of businesses. This is a puzzle in many ways, and frustrating for the data quality software industry, where few vendors see widespread deployment in enterprise initiatives. Usually data quality software is implemented (if ti is at all) in a piecemeal project by project fashion, and many companies have no data quality software deployed at all.

Perhaps it is just one of those subjects that few get excited over. Author Kurt Vonnegut once said: “Another flaw in the human character is that everybody wants to build and nobody wants to do maintenance.” and perhaps this is an inherent problem with data quality – it is hard to make it appealing to executives. Some more creative marketing by the industry could perhaps change this perception.

Tilting at Windmills

I wrote recently about likely further consolidation in the MDM market. A further example, albeit on a small scale, happened today as FullTilt, a PIM provider, was bought by QAD. QAD is a public company selling ERP software, with around 1,500 employees which has been listed since 1997 though its history goes back to 1987. FullTilt was known by industry insiders to be “in play” for many months, and has been openly for sale for some time. It is a relatively small company that has struggled to get scale, and so a deeper pocketed parent makes some sense for it.

This is another example of companies in more mature markets seeking to get exposure to the fast growing MDM market. It will not be the last such move.

Buying an Identity

Informatica did indeed make a purchase this week, but not the one some people were expecting. Instead of buying an MDM platform vendor they purchased Identity Systems for USD 85 million. Identity systems has a strong reputation in the data quality world, and amongst others has plenty of US federal agency clients amongst its 500 customers. It has 55 employees. This will shake things up a bit in the data quality market, where Similarity Systems was already one of the stronger players. The combination of the two under Informatica’s large sales channels will be a potent combination.

It still leaves open the question of whether, and in what way, Informatica will choose to enter the fast growing MDM market. I doubt this will be the last acquisition that they make.

Snack Time?

The MDM market, from a term barely used before 2003, has grown up into a market where literally dozens of vendors compete. As with any fast growing market, the big players have tended to acquire rather than build their own technology. IBM bought Trigo and DWL, Oracle bought Siebel and Hyperion, SAP bought A2i, TIBCO bought Velosel, Teradata licensed the code to I2’s MDM offering, D&B bought Purisma, Microsoft has bought Stratature and the list goes on.

In my view this consolidation as vendors seek to fill out their product lines is far from complete. For any data integration vendor it makers sense to have an MDM offering. TIBCO has already made its move, but this still leaves Informatica (so far just buying a data quality tool in Similarity) and even Ab Initio, though it is hard to figure out anything much about that secretive vendor. This also leaves Sun (with SeeBeyond) and other EAI vendors, and perhaps even a dark horse like EMC from a storage perspective.

While a number of technologies have already been purchased, as noted earlier, there are still several MDM independents out there that could potentially be bought up: Initiate, Siperian, Kalido, Visionware and, in France,, Orchestra Networks and Amalto. Some of these are big mouthfuls, but there are plenty of vendors with deep pockets who may like the look of a fast growing market (over 50% annual growth in the coming years predicted by Forrester).

MDM platform vendors themselves should consider whether to plough their own furrow or be vendor-neutral with regards to data quality. Some have their own algorithms, some partner with one or more data quality vendors, some offer a choice. There are a scary number of data quality vendors out there, many of them quite bite sized (few have more than USD 10M in revenue) and so I would be surprised if nothing happened here n the next twelve months. Moreover MDM platform vendors from a CDI heritage often get a rabbit in the headlights look when asked about data governance and complex work-flow (a screen or two to handle customer address mis-matches does not count as business process workflow support in my book). By contrast vendors from the PIM or analytic MDM world often have quite sophisticated offerings here, but may lack raw high volume performance. Hence there is room for some of the smaller vendors with tasty technology offerings to be snapped by those wishing to fill out their technology stack.

If there was a big move e.g. Informatica buying Initiate, then this may trigger a response from other MDM platform vendors who at the least could then decide to gobble up a data quality vendor like SilverCreek or Exeros in order to expand their own technology stack.

All in all 2008 I expect the next 18 months will an interesting time in the MDM market from a mergers and acquisitions perspective. After all, those silver tongued investment bankers will need something to do with their time now they can’t jam dodgy credit derivatives down gullible corporate treasurer’s throats any more.

Squaring the MDM circle

Jill Dyche raises an important point about the how companies are tackling MDM. She mentions the “random acts of MDM” that are done in isolation in a particular business area, or involving a particular data domain, which are unlikely to evolve into an enterprise-wide MDM solution.

The tricky issue that companies face is that MDM is a genuinely large-scale endeavour, and because we all know how well giant enterprise projects usually go, they are understandably reluctant to take on an enterprise-wide project. Instead they pick off an easier piece, such as one particular data type, or perhaps a broader set of master data types but only in a subset of the enterprise, say across one division. As Jill says, such isolated initiatives won’t in themselves magically grow into enterprise MDM. There is a further danger in disconnected initiatives. At this point the vendor technology out there is at very different stages of maturity depending on what kind of data you want to tackle, and on what scale. Some vendors have a well proven customer hub technology, but with limited experience in tackling product data (and may lack key functionality to do this e.g. attribute inheritance) and usually have very limited ideas about business process workflow and data governance support. Other vendors from the PIM or the analytic MDM world usually have much better business workflow support, yet may have limited scalability e.g. you it would be a brave person who tried doing a 100 million record customer hub using a PIM product. The vendors with a CDI heritage are adding more workflow capability, and the PIM and analytic MDM vendors are working on scalability, but these are works in progress rather than completed and tested features and functions. Hence separate initiatives may end up using different technologies due to the demands of a particular area, and it would be easy to end up with one technology to handle product data, another for customer data, and maybe another where analytics were the driver.

In my view you need to combine an enterprise-wide vision with a practical, bite-sized approach i.e. thing big but start small. You can build a broad enterprise strategy that encompasses data governance processes for example, even if you decide to build out actual master data hubs in a stepwise fashion, beginning with certain high value data domains or company divisions that can best benefit form improve master data. However you need to keep the big picture in mind in order to avoid (or minimise) duplicate technology investments that may prove hard to fit together. There are no magic bullets here, but enterprise architects need to put in place the processes and broad strategy that will lead to a better master data in the long term ,even if the technology to deliver across the enterprise is only partly here today. Setting up proper data governance, and getting business people committed to it, should have real benefits and will be valid efforts whatever technologies are deployed.