Andy on Enterprise Software

The Price of Failure

July 31, 2007

I enjoyed an article by Madan Sheina on the failure of BI projects. 87% of BI projects fail to meet expectations, according to a survey by the UK National Computing Centre. I wish I could say this was a surprise, but it is not. Any IT project involves people and processes as well as technology, yet many project focus almost entirely on the technology: tool choices, database performance etc. Yet in practice the issues which confound a BI project are rarely the technology itself. Projects fail to address actual customer needs, and frequently don’t acknowledge that there are several user constituencies out there with quite different requirements. Frequently a new technology is stuffed down the customer’s throat, and projects often neglect data quality to their peril.

From my experience, here are a few things that cause projects to go wrong.

1. Not addressing the true customer need. How much time does the project team spend with the people who are actually going to use the results of the BI project? Usually there are a subset of users who want flexible analytical tools, and others who just need a basic set of numbers once a month. A failure to realise this can alienate both main classes of user. Taking an iterative project to project development rather than a waterfall appraoch is vital to a BI project.

2. Data is only useful if it is trusted, making data quality a key issue. Most data is in a shocking state in large companies, and the problems often come to light only when data is brought together and summarised. The BI project cannot just gloss over this, as the customers will quickly avoiding using the new shiny system if they find they cannot trust the data within it. For this reason the project teams needs to encourage the setting up of data governance processes to ensure that data quality improves Such initiatives are often outside the project scope, are unfunded and require business buy-in, which is hard. The business people themselves often regard poor data quality as an IT problem when in fact it is an ownership and business process problem.

3. “Just one more new user interface” is not what the customer wants to hear. “Most are familiar with Excel and are not willing to change their business experience” was one quote from a customer in the article. Spot on! Why should a customer whose main job is, after all, not IT but something in the business, have to learn a different tool just to get access to data that he or she needs? Some tool vendors have done a good job of integrating with Excel, and yet are often in denial about this since they view their proprietary interface as a key competitive weapon against other vendors. Customers don’t care about this; they just want to get at the data they need to do their job on an easy and timely way. Hence a BI project should, if at all possible, look at ways of allowing users to getting data into their familiar Excel rather than foisting new interfaces on them. A few analyst types will be prepared to learn a new tool, but this is only a small subset of the audience for a BI project, likely 10% or less.

4. Data being out of date, and the underlying warehouse being unable to respond to business change, is a regular problem. Traditional data warehouse designs are poor at dealing with change caused by reorganisations, acquisitions etc, and delays in responding to business change cause user distrust. Unchecked, this causes users to hire a contractor to get some “quick and dirty” answers into a spreadsheet and bypass the new system, causing the proliferation of data sources to continue. Using packaged data warehouse that are good at dealing with business change is a good way of minimising this issue, yet even today most data warehouse are hand-built.

5. Training on a new application is frequently neglected in IT projects. Spend the time to sit down with busy users and explain to them how they are to access the data in the new system, and make sure that they fully understood how to use the system. It is worth going to some trouble to sit down with users and train them one to one if you have to, since it only takes a few grumbling voices to sow the seeds of discontent about a new system. Training the end users is never seen as a priority for a project budget, yet this can make the world of difference to the likelihood of a project succeeding.

6. Running smaller projects sounds crass but can really help. Project management theory shows that the size of a project is the single biggest predictor of success: basically, if projects fail, small ones do better, and yet you still see USD 100 million “big bang” BI projects. Split the thing into phases, roll out by department and country, do just about anything to bring the project down to a manageable size. If your BI project has 50 people or more on it then you are already in trouble.

7. Developing a proper business case for a project and then going back later and doing a post implementation review happens surprisingly rarely, yet can help shield the project from political ill winds.

You will notice that not one of the above issues involves a choice of technology, technical performance or the mention of the word “appliance”. Yes, it is certainly important to pick the right tool for the job, to choose a sufficiently powerful database and server and to ensure adequate systems performance (which these days appliances can help with in the case of very large data volumes). The problem is that BI projects tend to gloss over the “soft” issues above and concentrate on the “hard” technical issues that we people who work in IT feel comfortable with. Unfortunately there is no point in having a shiny new server and data warehouse if no one is using it.