I have spent some time recently in building up an MDM course. Normally such things are done at conferences, so unless you happen to be at some distant venue, they pass you by. However this one is different. It is an on-line course, marketed by a new company called eLearning (who have some well-known founders). The company reckons that it is harder and harder for people to justify trips to conferences for education, and this is certainly true at the moment from what I have observed and heard about technology conferences. Hence its model is to sell courses on-line.

The course is “MDM Fundamentals and Best Practice”, and you can see more about it here. It is actually quite a lot of work to put together such a course, but I am pleased with the result, and you can now get over three hours of my views on MDM for a very fair price indeed, all from the comfort of your desk or living room, and at a pace that you can control. Of course you do miss out on the trip to Las Vegas or similar, but you can’t have everything.

We have recently completed a major survey into the deployment styles used in MDM implementations. My colleague Dave Waddington has recently posted a summary of the results here. As can be seen, MDM projects are turning out to be quite meaty in size, but encouragingly the sucess rates were higher than I was expecting.

There were several quite interesting results that came out, and we will be doing further research into this area. The full report can be purchased from our website.

There are a number of approaches of “styles” to tackling MDM within a company, at least in terms of what to do first. If your most pressing issues are improving the quality of business reporting you may opt for “analytical” MDM, or if your issues are mostly with the data in transaction systems you would go for “operational MDM”. Within such categories there are different degrees of invasiveness, from a “registry” style where you leave the master data intact within operational systems, to an extreme root and branch approach “transaction style” where you rip out the ability of operational systems to maintain master data and put in new MDM system(s) to do this and feed the rest of the enterprise, while “co existence” recognises the reality that it may be necessary to live with a mix of approaches. But just how many companies go with which style, and how successful are they?

To answer this The Information Difference is launching a major piece of market research. This survey, sponsored by Microsoft and with media sponsors DM Review and CIO Magazine, is an in-depth look at this topic. To take the survey please click here.

When the survey is complete and the analysis is done I will let you know. Participants will receive a summary of the finding.

We recently completed the Q4 “Market Landscape” for MDM. As part of this we looked at all the vendors in the market and obtained figures for software revenues and growth of each. One interesting aspect of this is that the MDM software market so far appears to be holding up well. Indeed it is currently growing at an annualised rate of 30% according to our research,a healthy clip. It should be noted that the market size figures that we use exclude systems integrator revenues associated with MDM – these are estimated at around three times the size of the software market. As an aside, it is these kind of assumptions that can lead to seemingly wide discrepancies between market size estimates from different firms; typically you see a figure quoted in the press, but what does it include and exclude? Our figures for MDM software exclude data quality vendors, which are handled in a separate twice yearly update.

These figures, which are admittedly retrospective, confirm our November market research study looking at the effect of the financial crisis on MDM spending. This found that about as many companies were planning to accelerate their MDM spend as were planning to slow it down or defer projects (admittedly nearly a third of respondents were undecided).

So far at least, then, both actually Q4 spend as seen by vendors, and spending intentions in our survey are telling the same story. MDM software revenues are holding up well. We will continue to track the market closely, with a Q2 2009 Market Update to be published in July.

In doing the research for our six-monthly update of the MDM market I came across something which surprised me. Generally master data can be complex (such as bill of materials structures) but is not generally very large in volume, at least compared to transaction volumes. The exception is the “customer” dimension in a B2C company, where it is easy to see that 50 million or so records may be needed. Luckily “customer” data is usually quite simple compared to, say, product data, which may have hundreds of attributes.

However I have come across three cases now where the volume of master data records being managed is claimed to be around 500 million records. One vendor I spoke to said they had a customer planning a billion record MDM system. Dealing with hundreds of millions of records rather than tens of millions is a lot more challenging, especially where the data need to be dealt with in real time e.g. if you are adding a new customer account then you need to check whether that apparently new customer account is really a duplicate of an existing account; this should ideally be done straight away.

If anyone reading this has come across one of these really large MDM implementations then I’d be interested to hear your experiences.

According to the Celts, today is the day of the year when the boundaries between the living and the dead dissolve. While the ghosts of the departed such as Lehman Brothers stalk the earth, the living put on masks in order to mimic or placate the evil spirits. I’m a little unclear as to what would be the most suitable mask to don to mimic a deceased investment bank (do Armani make masks?), but software vendors across the globe will be nervously hoping that the spirit of Lehman has been thoroughly placated by the kindly sprites of Hank Paulsen, Gordon Brown et al. Fear is stalking the enterprise software market on a scale not seen since the aftermath of the millenium party in 2001.

Ghouls and goblins in the form of financial controllers in large companies are, as we speak, preparing a witches brew of budget cuts sufficient to make the bravest software salesman quail. Companies look at each other nervously and sing around the campfires to keep the spirits up, saying that their particular type of software does such an important job that it won’t be affected, and indeed that in times of adversity, perhaps companies will actually spend more money on critical IT projects? After all, data quality/MDM/(insert sector) is more important than ever now right? Right?

If you believe that you probably also believe in fairies and that derivatives make our financial systems more stable. As sure as night follows day, in times of economic downturn the finance department bring out their trusty red pen and seek out advertising budgets,travel allowances, training and information technology projects to carve up. At present there is a delayed reaction, as IT projects lumber on like zombies, unaware of the carnage waiting ahead. Perhaps our particular sector or project will be unaffected? Perhaps, but few will escape unscathed.

There will shortly be a lot more tricks than treats out there on offer for the enterprise software salesman as he makes his calls this winter.

Happy halloween!

I noticed an interesting blog post by Andrew Brooks about a possible side effect of the credit crunch, that of an increasing interest in MDM and data quality by recruiters. This may sound paradoxical, but it makes sense. Companies struggle to get good management information (for example about levels of counterparty risk for trading organisations such as investment banks) due to inconsistent master data across multiple systems. When times are booming this may be glossed over, but with prestigious companies going to the wall on a daily basis, being certain of the information that you rely on gets a higher priority. The blog resonated with me since I have just had a couple of recruitment agencies call me in the last few days asking what this “MDM thing is all about” in response to recent client inquiries.

Every major company I talk to struggles with getting reliable enterprise-wide data, and in every project I have been involved with, the data quality in corporate systems is worse than people think it is. If the current tough financial conditions prompt a new focus on fixing these issues, then indeed perhaps there will be a modest silver lining at the end of the credit crunch cloud.

There is an encouraging increase in MDM education, reflecting the growing interest in the subject. As well as Aaron Zornes’ pioneering work at the MDM Institute, we had a dedicated TDWI MDM conference in Savannah earlier in 2008. Now TDWI is running a dedicated MDM track on its next week-long conference, in New Orleans in early November. Just in case you didn’t have enough reason to attend, I will running a half day workshop as part of the conference.

For more information or to download a complete copy of the brochure, visit: www.tdwi.org/neworleans2008

Register before October 4, and receive an early registration discount, if you use this code when booking.

Priority Code: IN34

By early November it is nearing the end of hurricane season, and you can take advantage of New Orlean’s famous easy-going atmosphere and fine music and cuisine (at least after you have finished the workshop sessions). You may even learn something.

Hope to see you there!

Increasingly when I talk to customers about their plans for master data management or data quality, the area of data governance comes up. Data governance is the set of business processes and controls that surround the lifecycle of data, rather than necessarily involving technology. Some MDM vendors provide sophisticated support for data governance, others less so, but I believe that it is key to the success of a data initiative. Without thinking about the processes, it is of limited use to just put in a piece of technology, or carry out a once-off data cleansing exercise without considering what happens next.

Indeed in talking recently with data quality vendors, a number have commented on how the increasing profile that data governance is getting has enabled them to, in turn, get a higher profile. Previously data quality was seen by many companies as strictly something for IT or those direct mail geeks in their marketing department, but now data quality is getting an airing as part of broader initiatives to improve data across the board.

In order to put more flesh on the data governance bone, The Information Difference is conducting a major piece of primary market research into data governance. You can be part of this by taking the survey Completed surveys will receive a full copy of the research, and there is even a prize draw with exciting and valuable prizes (this view may depend a little on your taste) to tempt you.

In a survey of telecoms companies by Yankee Group, it would appear than master data management has made it to the second highest investment prioity amongst the wireless, wireline cable, and satellite companies. This is interesting since it appears to confirm the generally very bullish predictions for MDM as a market, which I wondered whether may be affected by the general economic climate. After all, MDM is not a trivial matter on an enterprise scale, so it could be tempting to defer an MDM project and put it into into the “too difficult” pile for a year or two to see how the economy recovered.

Later in the year the Information Difference will conduct some research amongst MDM vendors to see how the predictions from analyst firms such as Forrester and Yankee are stacking up to the reality of a tricky economy.