Don’t prototype: iterate

Stephen Swoyer makes a case for using EII as a prototype for data warehouses in his recent Enterprise Systems article. As the article reflects, there are some dangers as well as benefits here e.g. the prototype may just be extended and a proper data warehouse system never built. This is a problem because, as I have argued elsewhere. EII is suitable only for a small subset of business intelligence needs. However the valid point is that business users do want to see prototypes, especially in an area like business intelligence where the requirements tend to be fluid and ill-defined. However there is an alternative to buying an EII tool, knocking up a prototype and then building the proper warehouse.

These days you do not have to build a data warehouse, since you can buy one. Packaged solutions can be deployed much more rapidly than data warehouses that have to be designed and built by hand, and if they are flexible enough then an iterative approach to the warehouse can be taken. A great example of this was at Owens Corning, who deployed a data warehouse package over a 90 day period, using weekly builds of the business model. Each week a new piece of the business problem was tackled, the business model was updated in the package and the results presented back to the users. Issues would arise, feedback taken, and the next week the model would be refined, and a new business area started. This highly iterative approach ensured that the business users were fully engaged with the project, and could see a visible improvement in what they were going to get week by week.

After a few weeks the problems became less technical and functional, and more business related e.g. issues of data quality and how certain business model issues were to be resolved. After 90 days the application was delivered, and this was no prototype: it was a fully working, deployed production warehouse. The insights this application gave saved Owens Corning a great deal of money, so much so that the project had a three week payback period. Indeed the project became an Infoworld 100 winner.

Data warehouse project leaders need to rid themselves of the notion that data warehouses have to be long, custom build projects. At present TDWI reckons they take 16 months to deliver on average. This is far too long if using a traditional waterfall methodology, and indeed needs a more iterative approach. But why build a throwaway prototype when you can implement the real thing via a data warehouse package?