All of those puzzled as to why HP would enter the data warehouse appliance market with some time back Neoview now have an answer – they indeed should have stayed well clear. The old saying goes that if HP was to market sushi they would call it “cold, dead fish”, and the Neoview marketing team seemed to take this philosophy to heart. The product had only attracted a few customers, and I struggled to even find anyone in HP willing to talk about it.

HP finally put Neoview out of its misery this week. It is tough for companies to market technologies outside of their core business, and this is a good example of how even such a powerful company as HP can appear (to continue to tomrnet the tortured metaphor) like a fish out of water when it is not selling servers or printers (or printer cartridges, which I believe is where the money really is in that business).

There will be much chortling amongst competitors, but this is also a sign that the appliance market is pretty crowded, and that it is far from an easy one to succeed in, even fior a big fish like HP.

Some entertaining tales of data quality issues in this article today. The full link is here:

http://searchdatamanagement.techtarget.com/news/2240030455/The-top-five-information-management-meltdowns-of-2010

It is remarkable how few companies make any effort at all to address data quality, which if left to fester can not only be embarassing to a company but also cost it real money. Yet in our surveys we see that barely a quarter of companies attempt to even measure their own data quality.

Happy New Year to all.

I attended some interesting customer sessions at the Netezza user group in London yesterday, following some other good customer case studies at the Teradata conference in the rather sunnier climes of San Diego. Once common thread that came out from some sessions was the way that the use of appliances changes the way in which companies treat ETL processing. Traditionally a lot of work has gone into taking the various source systems for the warehouse. defining rules as to how this data into be converted into a common format, then using an ETL tool (like Informatica or Ab Initio etc) to carry out this pre-processing before presenting a neatly formatted file in consistent form to be loaded into a warehouse.

When you have many terabytes of data then this pre-processing in itself can become a bottleneck. Several of the customers I listened to at these conferences had found it more efficient to move from ETL to ELT. In other words they load essentially raw source data (possibly with some data quality checking only) into a staging area in the warehouse appliance, and then write SQL to carry out the transformations within the appliance before loading up into production warehouse tables. This allows them to take advantage of the power of the MPP boxes they have purchase for the warehouse, which are typically more efficient and powerful than using regular servers that their ETL tools run on. This does not usually eliminate the need for the ETL tool (though one customer did explain how they had switched off some ETL licences) but means that much more processing is carried out in the data warehouse itself.

Back in my Kalido days we found it useful to take this ELT approach too, but for different reasons. It was cleaner to do the transformations based on business rules stored in the Kalido business model, rather than having the transformations buried away in ETL scripts, meaning more transparent rules and so lower support effort. However I had not appreciated that the sheer horsepower available in data warehouse appliances suits ELT for pure performance reasons. Have others found the same experience on their projects? If so then post a comment here.

It is clear to anyone that has worked in a global organization that there are distinct differences in the approach to technology in different countries. Of course generalizations are dangerous, but usually US companies are early adopters and happy to take risks on relatively unproven technology if it delivers real benefit. The UK and Scandinavia usually follow the US (except in mobile technology, where the US tends to be a laggard). After that, other European companies adopt at varying pace: the Dutch are usually fairly early adopters, the French less so, while the Germans and the Swiss like to see everything proven before taking a chance on something new. Asia is a complex set of individual markets, with some areas that are leading e.g. South Korea in broadband, while in other cases they may lag Europe in the adoption curve. On a recent visit to Japan I saw both ends of the spectrum, with very advanced GPS and mapping systems yet some fairly archaic back-office technology.

I am curious as to whether MDM will merely follow the contours of this conventional technology adoption pattern, or whether it will be different, which it may be since a key difference is that MDM requires more significant business engagement than many technologies. For example I was speaking at a conference in Sweden last week and was a little surprised at how new MDM appeared to be in a country that is generally an early adopter of technology. I am curious as to whether MDM practitioners have noticed any cultural differences in the way that MDM is being tackled? If so please post a comment of your views on this blog.

MDM appears to be getting trendy at the moment judging by the number of calls I have had in the last few weeks from headhunters (sorry: executive search consultants) wanting to recruit people with serious MDM experience, both for systems integrators and end-users companies. Of course, knowing what MDM actually stands for would be an advantage if this is your task, so I’d encourage such worthy people to get some education on the subject first before contacting me to plunder my contact network. This on-line course:

http://ecm.elearningcurve.com/Andy_Hayler_s/72.htm

will do the job, and of course there are alternatives. I was most amused by the conversation with one recruitment person, who has been asked by a big systems integrator to urgently recruit experienced MDM consultants in order to populate a project that they have apparently already sold to an unsuspecting client. I am guessing their pitch to the client was not “We have no idea what this MDM thing is, let alone any experience in it, but if you give us a load of money we’ll definitely try and hire someone who does”. Or in this case hire someone else who doesn’t know what it means to find someone who might do and may know someone who does.

Don’t you just love the world of consultancy?

Andy

In talking recently to several companies that have data governance initiatives in place, it is clear that such initiatives costs a lot of money (an average of eleven full time people according to or latest survey) and that there is pressure on to identify whether such initiatives are delivering value. One difficulty here is in comparing your data governance to other – after all there is no “benchmark” available. We aim to change that situation, and have today launched a data governance benchmarking initiative, in conjunction with the Data Governance Institute.

If you have a data governance program, up and running, and want to see what you are doing compared to others, then please join this:

http://informationdiff.data-governance-benchmarking.sgizmo.com/s3/

In return for a completed survey (it is lengthier than our usual surveys, but you “get what you pay for” with surveys i.e. the more information you provide, the greater range of analysis we can do) you will receive a free summary report of the findings. There will also be an opportunity to purchase an in-depth, customised study of your organization’s performance relative to others.

I came across an intriguing poll today. A small company called Software Advice , who provide advice on evaluation of ERP and MRP systems, has been running an on-line poll about which software company Oracle will acquire next. The results of their poll so far (which is still open) is here.

What interested me most was not the actual poll results as such, but the way in which this poll has managed to attract a very high response rate (well over 1,000 responses). It could be viewed as an example of crowdsourcing, which is an idea explored by James Surowiecki in his book “The Wisdom of Crowds”. This is a fascinating idea, reasoning that in some cases a large sample of answers will actually generate a more accurate response than a supposed expert. Of course this method does not always work (Gary Kasparov beat thousands of players voting in a poll in a chess match against “the world” in 1999) but it seems like an idea that is worth further testing.

This is the first time I have seen this notion being applied to the world of enterprise software. There are plenty of other questions that could be tackled by such an approach, and I hope that this company or others extend this poll to other areas.

EMC has entered the data warehouse appliance market via the purchase of Greenplum, who made good progress in the space with their massively parallel database (based on PostgresSQL). Greenplum had impressive reference customers and some of the highest end references out there in terms of sheer scale of data. They had also been one of the few vendors (Aster is another) that went early into embracing MapReduce, a framework designed for paralleism and suitable for certain types of complex processing.

Data warehouse appliances can be a tough sell because conservative buyers are nevous of making major purchases from saller comapnies, but the EMC brand will remove that concern. Also, EMC have a vast existing customer base and the sales channel that can exploit this. Seems like a sensible move to me.

We are just kicking off a major piece of research into data governance, at the heart of which is a survey. This survey has an impressive array of media sponsors and sponsors, and aims to get to the heart of what is going on in practice with data governance today, what level of effort is being put into it, and how successful is it. The survey will also address the linkage between governance and the key areas of MDM and data quality. If you have some background in this area then please participate in the survey.

As an added incentive, if you complete the survey then you be sent a summary of its results, and there are several of our vendor profiles (which costs $495) to be won via a prize draw.

Our research shows that building a business case is one of the major barriers when putting together an MDM initiative. Some of this is due to people not knowing the correct information to present in order to make a convincing business case, and in other cases it is due to the difficulty in estimating costs and benefits. This is actually not rocket science.

Join me for a webinar this Wednesday (at 11 EST = 16:00 UK time) on how to build the business case for an MDM project. It contains lots of practical information to make a proper business case, including data from our latest research e.g. on how much MDM projects cost to maintain once live.

You can register for it here:

http://register.mdmdqu.com/forms/MDMDQURegistrationForm

It is even free!