A different kind of conference

I spent the last two days at the eWorld Purchasing and Supply conference on London, where I had a couple of speaking slots (you can’t escape your past, and a long time ago I worked in Shell’s technology planning area, which involved software.procurement). I was pleasantly surprised by the scale of the conference. These days most IT conferences are struggling to get decent attendance, as more and more people seek out information on-line. Yet this specialist conference managed to get over 300 attendees on both days, and from the conversations I had these were mostly “real” delegates i.e. people with actual projects and problems to solve, rather than the tyre-kickers who sales people dread and often seem to constitute much of the attendance of some IT conferences.

For another perspective on the conference see the following blog:

http://www.esourcingforum.com/?p=388

Credit where credit is due to the organisers, Revolution Events, who did an excellent job with administration and organisation (only let down a bit by the caterers on day 1 who were of the “oh, we didn’t think this many people would be coming” variety). The exhibits area had decent flow of traffic and the speaking slots stayed well on time, a particular bugbear of mine. Perhaps these more specialist conferences, concentrating on a particular vertical or in this case functional niche, are the way to go.

When is an appliance not an appliance?

What we call things is important. The recent rise of data warehouse “appliances”, pioneered by Netezza (and arguably Teradata before that) is an interesting case in point. For years the relational database vendors spent their energy in making sure that transaction systems ran quickly and reliably. Business intelligence applications were not a major focus, and this led to a number of approaches to dealing with very large data warehouse applications. Certain types of index scheme would work very well for read-only BI queries, for example, and Red Brick was an early example of a database optimised as such. Later Teradata did a superb job of carving out a high end niche by using parallel processing hardware and specialist database software to take advantage of this properly. They did such a good job that after a while Teradata almost became synonymous with large data warehouses, of the types typically encountered in retail banks, supermarket chains, telcos etc. Oracle and othe others made some half-hearted attempts to fight back with features like star joins, but by then it was too late: the specialist data warehouse device, in the form of Teradata, had become established. Of course such projects were still large and complex. Most data warehouse project costs are associated with people, not hardware or software, and this does not change whether you are using SQL Server or Teradata as your database.

However, marketing can at times (not often, but sometimes) be a clever and subtle thing. When Netezza brought out essentially a device like Teradata, but quicker and cheaper, the label “appliance” was used, and a very clever one it is. In normal English usage an appliance is something that we just plug in, like a toaster or a coffee maker. Without making any such overt claims, the “appliance” label has a comforting implication that your data warehouse project will have that toaster installation-like quality previously lacking with pesky traditional databases. Given that a DW appliance is just some clever hardware and an optimised database, your project issues are in fact identical to those of any other DW project. Analysis, user requirements, data quality, sourcing, design and reporting all have to be done, although the appliance may certainly be able to handle large volumes of data at a much better price point than a traditional hardware/database combination. Since the hardware and software on a project may typically account for less than 20% of the project costs, this is an undeniably useful thing, but hardly takes us into toaster territory.

Yet the label matters. In a rather breathless blog yesterday:

http://www.itbusinessedge.com/blogs/mia/index.php/2006/09/05/flaming-web-20/

Mike Stevens, who I don’t know personally but appears to have a background in PR rather than hands-on data warehouse project implementation, claims that appliances spell “trouble for traditional data warehouse vendors” since an appliance may cost just USD 150k whereas “conventional solutions cost millions”. He falls into the language trap of the appliance. Your data warehouse still has to to deal with all those people-intensive things (data sourcing reporting, testing) whether you use a conventional SQL database and a regular server, or a specialist DW appliance. The issues are all identical, except with an appliance you have some additional cost since less familiar skills will need to be brought to bear (there are more Oracle skills out there than Netezza ones). The savings on hardware by using an appliance may be very significant and comfortably justified on a large data warehouse, but such a project is not going to cost USD 150k and a quick plug in the wall socket.

If this kind of misconception is so easily repeated by journalists (or at least bloggers) then I wonder how widespread this view is amongst IT managers, and how much this has helped data warehouse “appliances” catch on? Would Netezza have done quite so well if they had been labelled something less reassuring, like a “data warehouse turbo toolkit”? It was said that HP was so bad at marketing it would, if it sold sushi, describe it as “cold dead fish”. The “appliance” vendors shows that smart marketing can still be done within hi-tech.

Microstrategy Joins the Party

To go with Business Objects’ excellent Q4 results, Microstrategy also reported good figures, suggesting that the BI industry is in generally good shape. Revenue was USD 92.6M, up 20% over last year. Just USD 36.6M of this was license revenue, but this was 17% up on last year.

There was very healthy USD 32M operating margin, which means an operating margin of 34%. Other measures were also healthy e.g. days sales outstanding of just 54, and cash flow from operations of USD 26M. Admittedly this is down a little as expenses have risen by 22% year over year, but all the same this is a healthy business.

These results are all the more significant because Microstrategy has been in rather a flat phase for some time, with licence growth almost flat since early 2005. This perky set of results will be all the more welcome for its staff and shareholders given this background.

No objections to Business Objects results

Business Objects delivered a very solid Q4, with revenue of USD 371M up 22% from a year previously. The good news is that the increase was mainly due to license revenue, at USD 180M up 16%. The operating margin of 23.6% was the best the company has ever achieved.

The only small cloud on the horizon was that the core BI business was actually in decline in license terms. The EIM revenue was USD 23M (well done to the First Logic boys and girls), USD 30M in enterprise perfromance management, and USD 127M (down 4% year over year) in the core BI products. This apparently paradoxical result is in line with my long-standing thesis about the saturation of the BI tools market in enterprises.

There were 13 deals in excess of USD 1M (up from nine last quarter), and overall in 2006 there were 35 deals of this size, which is actually down form 46 last year. Overall growth in Q4 was geographically well spread, with Europe up 25%, Asia Pacific 26% and the Americas 19%. The results reflect some wise acqusitions, since without the EIM/First Logic revenue, and the EPM revenue (based around the acquired SRC software and to a lesser exent the ALG acqusition) things would not look so rosy. Still, this vindicates the strategy of trying to move up the food chain in BI beyond core reporting, so credit where credit is due.

How fast can BI get?

FAST is the latest enterprise search company to dip its toe in the water of business intelligence, following Autonomy’s recent announcements. In the case of FAST, which is arguably the leader in enterprise search (and must surely be the leading Norwegian software company), they have done so through acquisition. They bought Corporate Radar, a small BI vendor who had some quite clever reporting technology (based around the Microsoft platform) that was quite flexible, and browser-based. On one project that I encountered at Kalido in the US, it was particularly good at building specialist financial reports e.g. gross margin “waterfall” analysis.

What is less clear to me is how “FAST Radar” as it is now known, really integrates with the FAST search engine. Superficially it is appealing to have “search” applied to BI, after all, if Google can scan the whole internet in seconds, why can I never find my monthly sales figures? However the problem in dealing with structured data is the ambiguity of metadata within corporate organisations (“which sales figures do you mean, exactly”), a problem that search technologies, clever though they are, barely scratch the surface of. Putting a very efficient index on a keyword is great for text searching, but it is less obvious to me how useful this would be in resolving ambiguous or inconsistent metadata. Hence I wonder whether the “integration” of these technologies goes much more than skin deep.

If anyone out there has practical experience of a project that uses one of these search engines combined with a BI tool or data warehouse, then please make a comment on this blog, as I am sure that your experiences will be of considerable interest to others.

Indian Summer

Therr will be a short intermission in the blog as I am off to India for a couple of weeks. There are many things to love and admire about India, but the quality of their internet connections outside the main urban centres is not one of them, so doing the blog from there will be impractical. I’ll just have to concentrate on the beaches of Goa instead; I expect I’ll cope.

Normal service will be resumed in February.

Impartial Advice?

HP continues with its plans for the business intelligence space with an announcement of in-house data warehouse technology:

http://www.computerworld.com:80/action/article.do?command=viewArticleBasic&articleId=9008218&intsrc=news_ts_head

with a new business unit. The offering with be based around HP’s attempt at a “data warehouse appliance”, called Neoview. This is a competitor to Teradata and Netezza, but at this stage it is hard to tell how functional this is, since it is unclear that there are any deployed customers other than HP itself.

The timing of this announcement is curious given HP’s acquisition of data warehouse consultancy Knightsbridge. Certainly data warehousing is a big market and Teradata is a tempting target – after all, most of the really big data warehouse deployments in retail, telco and retail banking use Teradata. There are lots and lots of juicy services to be provided in implementing an “appliance”, which in fact is no such thing. An appliance implies something that you just plug in, whereas data warehouse appliances are just a fast piece of hardware and a proprietary database, still requiring all the usual integration efforts, but with the added twist of non-standard database technology. Certainly plenty of business for consultants there.

However HP’s home-grown offering will not sit well with its newly acquired Knightsbridge consulting services, who made their reputation through a quite fiercely vendor-independent culture which always prided itself in choosing the best solution for the customer. People trust independent consultants to give them objective advice, since they are not (or at least they hope they are not) tied to particular vendor offerings. Presumably HP’s consultants will be pushing HP’s data warehouse solution in preference to alternatives, and so can hardly be trusted as impartial observers of the market. An analogy would be with IBM consultants, who while they may work with non-IBM software are clearly going to push IBM’s offerings given half a chance.

If you were a truly independent consultant how would you react to a brand new data warehouse appliance with a track record only of one deployment, and that in the vendor itself? Would you immediately be pushing that as your preferred solution, or would you be counseling caution, urging customers to wait and see how the new tool settles down in the market and how early customers get on with it? If you are a Knightsbridge consultant now working for HP, what would your advice be? Would it be any different to the advice you’d have offered in December 2006 before you became part of HP?

This kind of conflict of interest is what makes thing difficult for customers when choosing consultants. It is hard to find ones who are truly independent. Of course consultants always have their own agenda, but usually this is about maximising billable hours. If they are tied to a particular solution then that is fine if you are already committed to that solution, but you will need to look elsewhere for objective advice about it.

Teradata steps into the light

In a logical move that I would say was overdue, Teradata finally became its own boss. It has long been nestling under the wing of NCR, but there was little obvious synergy between ATM machines and data warehouse database software, and so it seems to me eminently sensible for Teradata to stand on its own two feet. Running two quite different businesses with the same company is always a problem, as different business models lead to natural tensions as the company tries to accommodate different needs within the same corporate structure.

Teradata accounts for about USD 1.5 billion of revenue, around one third of NCR. The challenge for Teradata is growth. It has succeeded when others failed in the specialist database market, dominating the high end data warehouse market despite competing with Oracle, IBM and (to a lesser extent) Microsoft. Yet revenues have been pretty flat in the last couple of years, and there is new competition in the form of start-up Netezza, which although tiny compared to Teradata is nonetheless making steady inroads, and causing pricing pressure. Teradata has generally loyal customers though notoriously opaque pricing, which has enabled it to achieve good margins (especially on support), though its finances were never entirely clear as they were wrapped up with NCR. Splitting the company out will allow the market to value Teradata on its own merits.

A long journey

An Accenture study:

http://www.informationweek.com/research/showArticle.jhtml?articleID=196800921

quantifies how much time middle managers in enterprises waste seeking out information, and comes out at two hours a day i.e. a quarter of an average working day. When they find it, half of the information turns out to be of no use. This sounds about right to me, and ilustrates just how far BI really has yet to go in being genuinely useful, and also shows just how bad the true state of information is in large companies.

The issue is not only that technologies are insufficiently intuitive. In my experience there are a number of factors that come into play:

– no culture of sharing information
– inconsistent data definitions
– poor data quality
– inability to locate appropriate data sources
– insufficient understanding of how to use BI tools effectively.

If you set out to produce a useful new report in some area and succeed in doing so, what incentive is there for you to make this easily shared around the company, and to help others find it? In most companies this would be pure altruism, and so people just keep the information on their hard disk, and indeed may gain kudos from the “information is power” syndrome. Overcoming such cultural barriers is hard, and few companies succeed. I should say that Accenture themselves do as good a job as anyone I have seen, where their consultants are actively tasked with documenting project lessons and storing these, with appropriate keywords, in an internal knowledge management system. However I have not seen this in other consultancies to anything like the same extent.

The other problems are all too familiar to people working in BI. Inconsistent data definitions and poor data quality are the heart of what MDM is all about, and we know how immature that is. Yet without fixing this then accurate and easy to obtain information is still elusive. A further problem which some technologies are starting to address is the sheer job of finding an existing report. Ironically there is an excellent chance that if yoiu want some partioular report, then someone else did too and has already built it. The troiuble is that may be in an Excel spreadsheet on a hard drive, or sitting on a shared server but you simply have no easy way of finding that it is there. It is ironic that Google allows us to search the whole internet in moments, yet finding a report within our own company is a much tougher proposition. Enterprise search vendors like Fast and Apptus, as well as Google itself, are beginning to apply smart technology to the problem, but here it is still early days.

Finally, most end users either don’t have access to create a new report easily, or are not trained in making best use of BI tools, or simply don’t have time to learn. This is why Excel is so popular; it is familiar and ubiquitous, and so people would rather get data into Excel and play with it there than learn a new BI tool.

I believe that these are mostly quite intractable problems, only some of which lend themselves to new and better technology. So anyone with a magic bullet e.g. “the answer is SOA” is talking nonsense. It is only by addressing the organisational, cultural and data ownership issues in combination iwth enterprise search and better tool training that a company can improve that two hours a day per person. It will be long, hard slog, and buying the latest trendy tool is not enough, whatever the salesman tells you.

New Year

Happy New year to everyone. This period is one which journalists and bloggers dread since there is virtually no news of note (unless someone did something really naughty at the work Christmas party). Fortunately Gartner’s Andreas Bitterer has come to the rescue with the idea of a virtual cocktail party for a few bloggers, who are invited to reveal five things about themselves:

http://bitblueblog.com/2006/12/you-got-tagged.html

This is certainly a fine distraction from coming up with original observations about the software industry, so in the spirit of New year here are five things about me:

1. In 2004 I completed visiting every 3 star Michelin restaurant in the world at that time (see www.andyhayler.com for more on this).

2. I have cuddled a full-grown cheetah (in South Africa at a sanctuary where they bring up baby cheetahs and then release them back into the wild; they purr just like a regular cat)

3. I have eaten fugu (the poisonous puffer fish) when in Japan and lived to tell the tale.

4. I was a junior county chess champion (Somerset is a small county).

5. I once sat next to Madonna and Guy Ritchie at a restaurant (Zafferano in London) and didn’t recognise Madonna as she was wearing glasses, but I did recognise Harry Shearer (who does most of the voices on the Simpsons) a few weeks later. This may reveal something about my grasp of popular culture.

Well done Andreas – an inspired rescue idea for bloggers everywhere in search of material this week. Someone go and take someone else over soon please so we can all get back to worthy commentary.