Andy on Enterprise Software

Oracle continues its shopping spree

April 12, 2011

A further element of consolidation in the data management occurred when Oracle purchased Datanomic, a data quality company based in Cambridge (for a change, the original one in England rather than the one near Boston). Datanomic has been an interesting story, set up in 2001 and bringing to the market a well-rounded data quality product. This is a crowded market, and in the dreadful conditions for enterprise software that occurred after the market crash in 2001 the company initially struggled. There were, after all, an awful lot of data quality products out there that people had already heard of. Then Datanomic did a very smart thing and re-positioned itself to focus on a business rather than a technical issue: compliance, especially in financial services.

This turned out to be an inspired change of marketing strategy, and the company went from layoffs to hiring again, growing rapidly over the last three years, far in excess of the 9% annual rise in the general data quality market that has been seen recently. Datanomic has had positive customer references in our regular annual surveys, and it seems to me a well-architected solution. From Oracle’s point of view, this complements their purchase of Silver Creek, which was a specialist product data quality tool. These two acquisitions suggest that Oracle is changing its view of data quality – previously they relied on partner arrangements with companies such as Trillium for their data quality solution. Now it would appear that they see data quality as a more integral issue. The price of the deal was not disclosed, but given Datanomic’s rapid recent growth, it will have doubtless been at a healthy premium.

Teradata Goes Shopping

March 4, 2011

The recent flurry of acquisition activity in the data warehouse appliance space continued today as Teradata purchased Aster Data. HP’s purchase of Vertica, IBM’s of Netezza, EMC of Greenplum and (less recently) Microsoft of Data Allegro underscore the fact that demand for high performance analytic databases is perceived to be strong by the industry giants. At first glance this may seem an odd buy for Teradata, itself the original appliance vendor, but Aster in fact occupied a very particular niche in the market.

Aster’s strengths (and its intellectual propertty, patent pending) were around its support for intergated MapReduce analytics. MapReduce is the distributed computing framework pioneeed by Google, which inspired the open-source framework Hadoop. This framework is suited to highly compute-intensive analytics, particularly of high volumes of unstructured data. This includes use cases like fraud analysis, but has found a particular niche in social networking websites, who have to deal with vast and rapdily increasing volumes of data. Certain analytic queries such as social network graph analysis, signal analysis, network analysis and some time series analysis are awkward for conventional SQL, involving self-joins and potentially multiple passes through a database, which is a big deal if the database is hundreds of terabytes in size. The MapReduce appoach can offer significant perfomance advantages for such use cases, though it typically requires specialist programming knowledge.

Aster’s customers included companies like LinkedIn and FullTilt Poker,and its SQL-MR technology had a good reputation in such situations. Aster was a relatively small company, so this purchase is loose change for Teradata but buys it a jump-start into this fashionable area of analytic processing. Aster of course gains access to the channels and deep pockets of Teradata. Conservative buyers may have been unwilling to jump into these waters with a start-up but will be reassured by the legitimisation of the technology by a big software brand. Hence it seems like a win-win for both companies.

This leaves very few stand-alone independent data warehouse vendors: ParAccel, Kognitio and more obscure players like Exasol and Calpont can continue to plough an independent path, but I suspect that this will not be the last acquisition we will see in this market.

HP has a Vertica strategy

February 14, 2011

Having recently abandoned its Neoview offering, HP today revealed its plans in the data warehouse market by purchasing Vertica. Vertica is one of a clutch of data warehouse vendors that has apeared in recent years, employing MPP architecture and usually a specialist database structure in order to achieve fast analytic performance on large volumes of data. In Vertica’s case it uses a columnar database (of the style pioneered by Sybase), but in this case combined with MPP. This combination works well for many analytic use cases, and a well executed sales strategy based around this has meant that Vertica has achieved consderable market momentum compared to many of its competitors, building up a solid roster of customers such as Comcast, Verizon and Twitter.

In principle HP’s vast sales channel should be very beneficial to spreading the Vertica technology further. Nervous buyers need no longer be anxious about buying from a start-up, and HP clearly has a vast marketing channel. Yet success is far from guaranteed, as HP’s previous debacle with its Neoview data warehouse offering showed. Now at least HP has a proven modern data warehouse offering with traction in the market. It remains to be seen whether it can exploit this advantage.

Diversity in MDM

February 8, 2011

It has long been a theme of my writing that MDM is something that goes beyond customer and product data, and after making that point publicly at a conference in 2006 (to much ridicule from a VP of a well known MDM vendor in the audience) it seems as if the tide of opinion has definitely switched in the multi-domian direction in the last couple of years. A good example of this was shown today. Orchestra Networks is a French MDM vendor with a multi-domain MDM product. In their latest release of their EBX product they announced specific solutions for MDM in Finance and Accounting, MDM in HR and MDM in sales and marketing.

This business line orientation seems a sensible mmove to me. Most vendors talk about almost entirely about customer and product data as that has been their heritage and comfort zone, but there are plenty of opportunities fro MDM solutions to be applied outside these two domains. Indeed I recall that the very first MDM deployment of Kalido MDM, in 2002, was actually for finance and accounting data (general ledger etc) at a Dutch bank. Given the vast scale of an enterprise-wide MDM solution it seems to me wise to pick off specific domain areas where there is “low hanging fruit”, and at present most of the industry has been ignoring these other domains, at least in their marketing.

HP reels in Neoview

January 28, 2011

All of those puzzled as to why HP would enter the data warehouse appliance market with some time back Neoview now have an answer – they indeed should have stayed well clear. The old saying goes that if HP was to market sushi they would call it “cold, dead fish”, and the Neoview marketing team seemed to take this philosophy to heart. The product had only attracted a few customers, and I struggled to even find anyone in HP willing to talk about it.

HP finally put Neoview out of its misery this week. It is tough for companies to market technologies outside of their core business, and this is a good example of how even such a powerful company as HP can appear (to continue to tomrnet the tortured metaphor) like a fish out of water when it is not selling servers or printers (or printer cartridges, which I believe is where the money really is in that business).

There will be much chortling amongst competitors, but this is also a sign that the appliance market is pretty crowded, and that it is far from an easy one to succeed in, even fior a big fish like HP.

Data Quality Stories

January 6, 2011

Some entertaining tales of data quality issues in this article today. The full link is here:

http://searchdatamanagement.techtarget.com/news/2240030455/The-top-five-information-management-meltdowns-of-2010

It is remarkable how few companies make any effort at all to address data quality, which if left to fester can not only be embarassing to a company but also cost it real money. Yet in our surveys we see that barely a quarter of companies attempt to even measure their own data quality.

Happy New Year to all.

Appliances and ETL

November 19, 2010

I attended some interesting customer sessions at the Netezza user group in London yesterday, following some other good customer case studies at the Teradata conference in the rather sunnier climes of San Diego. Once common thread that came out from some sessions was the way that the use of appliances changes the way in which companies treat ETL processing. Traditionally a lot of work has gone into taking the various source systems for the warehouse. defining rules as to how this data into be converted into a common format, then using an ETL tool (like Informatica or Ab Initio etc) to carry out this pre-processing before presenting a neatly formatted file in consistent form to be loaded into a warehouse.

When you have many terabytes of data then this pre-processing in itself can become a bottleneck. Several of the customers I listened to at these conferences had found it more efficient to move from ETL to ELT. In other words they load essentially raw source data (possibly with some data quality checking only) into a staging area in the warehouse appliance, and then write SQL to carry out the transformations within the appliance before loading up into production warehouse tables. This allows them to take advantage of the power of the MPP boxes they have purchase for the warehouse, which are typically more efficient and powerful than using regular servers that their ETL tools run on. This does not usually eliminate the need for the ETL tool (though one customer did explain how they had switched off some ETL licences) but means that much more processing is carried out in the data warehouse itself.

Back in my Kalido days we found it useful to take this ELT approach too, but for different reasons. It was cleaner to do the transformations based on business rules stored in the Kalido business model, rather than having the transformations buried away in ETL scripts, meaning more transparent rules and so lower support effort. However I had not appreciated that the sheer horsepower available in data warehouse appliances suits ELT for pure performance reasons. Have others found the same experience on their projects? If so then post a comment here.

MDM Cultural Differences

October 18, 2010

It is clear to anyone that has worked in a global organization that there are distinct differences in the approach to technology in different countries. Of course generalizations are dangerous, but usually US companies are early adopters and happy to take risks on relatively unproven technology if it delivers real benefit. The UK and Scandinavia usually follow the US (except in mobile technology, where the US tends to be a laggard). After that, other European companies adopt at varying pace: the Dutch are usually fairly early adopters, the French less so, while the Germans and the Swiss like to see everything proven before taking a chance on something new. Asia is a complex set of individual markets, with some areas that are leading e.g. South Korea in broadband, while in other cases they may lag Europe in the adoption curve. On a recent visit to Japan I saw both ends of the spectrum, with very advanced GPS and mapping systems yet some fairly archaic back-office technology.

I am curious as to whether MDM will merely follow the contours of this conventional technology adoption pattern, or whether it will be different, which it may be since a key difference is that MDM requires more significant business engagement than many technologies. For example I was speaking at a conference in Sweden last week and was a little surprised at how new MDM appeared to be in a country that is generally an early adopter of technology. I am curious as to whether MDM practitioners have noticed any cultural differences in the way that MDM is being tackled? If so please post a comment of your views on this blog.

Psst, got an MDM consultant?

September 17, 2010

MDM appears to be getting trendy at the moment judging by the number of calls I have had in the last few weeks from headhunters (sorry: executive search consultants) wanting to recruit people with serious MDM experience, both for systems integrators and end-users companies. Of course, knowing what MDM actually stands for would be an advantage if this is your task, so I’d encourage such worthy people to get some education on the subject first before contacting me to plunder my contact network. This on-line course:

http://ecm.elearningcurve.com/Andy_Hayler_s/72.htm

will do the job, and of course there are alternatives. I was most amused by the conversation with one recruitment person, who has been asked by a big systems integrator to urgently recruit experienced MDM consultants in order to populate a project that they have apparently already sold to an unsuspecting client. I am guessing their pitch to the client was not “We have no idea what this MDM thing is, let alone any experience in it, but if you give us a load of money we’ll definitely try and hire someone who does”. Or in this case hire someone else who doesn’t know what it means to find someone who might do and may know someone who does.

Don’t you just love the world of consultancy?

Andy

Benchmarking Data Governance

August 31, 2010

In talking recently to several companies that have data governance initiatives in place, it is clear that such initiatives costs a lot of money (an average of eleven full time people according to or latest survey) and that there is pressure on to identify whether such initiatives are delivering value. One difficulty here is in comparing your data governance to other – after all there is no “benchmark” available. We aim to change that situation, and have today launched a data governance benchmarking initiative, in conjunction with the Data Governance Institute.

If you have a data governance program, up and running, and want to see what you are doing compared to others, then please join this:

http://informationdiff.data-governance-benchmarking.sgizmo.com/s3/

In return for a completed survey (it is lengthier than our usual surveys, but you “get what you pay for” with surveys i.e. the more information you provide, the greater range of analysis we can do) you will receive a free summary report of the findings. There will also be an opportunity to purchase an in-depth, customised study of your organization’s performance relative to others.