An interesting, common sense debate was featured in Silicon.com last week. A panel of CIOs was asked whether they felt comfortable buying from small suppliers or whether they preferred dealing with the big players. There was a surprising degree of consensus that, in general, CIOs felt OK about small suppliers: indeed in some cases they actively preferred them. Perhaps this is a sign of a steady recovery in economic confidence, with CIOs preparing to come out of their shells into which they retreated in 2001 and 2002. As I have written about before, buying software from small suppliers carries risks, but this is true of big suppliers also. Just because a giant company may not go bust does not stop them dropping products for any number of reasons, as I can testify from personal experience.

The one element in the article that did make me smile was the assumption that code escrow was a form of insurance against a small vendor folding. Indeed code escrow arrangements have become quite standard in contracts, and generate modest fees for those organisations that provide the service. I hate to disillusion those CIOs, but code escrow is not the panacea it may seem. Sure, so you get the source code, but then what? Firstly, you have to hope that the vendor has been diligent about keeping their escrow up to date with the version of software that you are actually using. But more to the point, the raw code itself is of limited use without the design specifications that go along with it (at least assuming you actually want to continue developing it). Even if you are looking at basic support only, how well documented is the code? I had the misfortune to try and execute an escrow contract once when I was working at Esso. The tape of source code duly turned up and it was 3 million lines of undocumented assembler code. While my colleague (an expert at assembler code) got a misty gleam in his eye as he could see a job for life coming up, we concluded that we simply couldn’t justify taking this on, and opted to go for a complete replacement instead. So, if you are insisting on source code escrow from your vendor, be aware of the pitfalls and ask some searching questions about documentation.

The ERP landscape became simpler this week, when SSA was swallowed up by a private equity company called Golden Gate Capital. This group (and its subsidiary Extensity) has now absorbed Baan, Comshare, Dun & Bradstreet Software, Epiphany, Infinium, Marcam and Systems Union. As Ventana points out this means that the choice boils down to SAP, Oracle, Microsoft and this new amalgam under GoldenGate/Extensity. It is interesting that this private equity group seems to be performing the role CA used to play: hoover up under-performing companies, slim them down and milk the support revenue stream. What the article implies is that this is pretty much the endgame for the ERP space, but I am not so sure. The one dimension missing here is the hosted model.

Salesforce.com showed what could be done with a hosted software model. In the ERP world we are seeing new entrants like Intacct and Ataoi, which while they are small so far are making solid inroads into their chosen markets. At present this approach may appeal more to SMEs, but remember that salesforce.com started this way as well, only later taking on Siebel more directly. I know the CEO of one of these emerging hosted ERP vendors, who was amused to be in a competitive bid with SAP at one prospect. His company’ bid was less than one-tenth that of the behemoth. I’m not suggesting that these hosted ERP systems compare in functionality with SAP and Oracle, but perhaps that after all is the point. Traditional ERP systems have become so bloated that large parts of them remain unused and having systems hosted avoids all the environmental installation problems that ensure with traditionally installed software, where there are so many combinations of operating system, DBMS, transaction monitor etc that the vendors have to spend as much time testing combinations of software than actually writing new functionality.

It may take some time but I think the next change in the ERP market will be via this hosted model. When you start to see defensive market offerings by the giant vendors, just as Siebel started an on-demand offering (but too late), then we will know that this prediction has been fulfilled.

Stephen Swoyer makes a case for using EII as a prototype for data warehouses in his recent Enterprise Systems article. As the article reflects, there are some dangers as well as benefits here e.g. the prototype may just be extended and a proper data warehouse system never built. This is a problem because, as I have argued elsewhere. EII is suitable only for a small subset of business intelligence needs. However the valid point is that business users do want to see prototypes, especially in an area like business intelligence where the requirements tend to be fluid and ill-defined. However there is an alternative to buying an EII tool, knocking up a prototype and then building the proper warehouse.

These days you do not have to build a data warehouse, since you can buy one. Packaged solutions can be deployed much more rapidly than data warehouses that have to be designed and built by hand, and if they are flexible enough then an iterative approach to the warehouse can be taken. A great example of this was at Owens Corning, who deployed a data warehouse package over a 90 day period, using weekly builds of the business model. Each week a new piece of the business problem was tackled, the business model was updated in the package and the results presented back to the users. Issues would arise, feedback taken, and the next week the model would be refined, and a new business area started. This highly iterative approach ensured that the business users were fully engaged with the project, and could see a visible improvement in what they were going to get week by week.

After a few weeks the problems became less technical and functional, and more business related e.g. issues of data quality and how certain business model issues were to be resolved. After 90 days the application was delivered, and this was no prototype: it was a fully working, deployed production warehouse. The insights this application gave saved Owens Corning a great deal of money, so much so that the project had a three week payback period. Indeed the project became an Infoworld 100 winner.

Data warehouse project leaders need to rid themselves of the notion that data warehouses have to be long, custom build projects. At present TDWI reckons they take 16 months to deliver on average. This is far too long if using a traditional waterfall methodology, and indeed needs a more iterative approach. But why build a throwaway prototype when you can implement the real thing via a data warehouse package?

As Rick Sherman quite rightly points out in his commonsense article in DM Review, data quality is not something that you can realistically fix before you build a data warehouse. Data quality in operational systems is often scarily low, but often the only way that this will be highlighted is when it is brought together at a data warehouse. People often assume that the data inside their ERP systems is somehow sacrosanct and immune to data quality issues, and it often comes as a big disappointment when they discover that this is not so.

In one example it turned out that a product was mis-priced in the SAP system in a region, resulting in the product being sold at cost price. This anomaly went undetected for over a year before a data warehouse project brought this to the surface (by comparing gross margin by product by country). Initially everyone assumed it was a bug in the warehouse software, but it was not. Indeed this insight alone pretty much paid for the project.

If a data warehouse can be implemented rapidly and in an iterative fashion then it can quickly highlight business issues such as this one, which may be as a result of data quality or could be as a result of new insight that was previously unavailable: the “wood for the trees” problem. Eventually data quality needs to come out of the closet and be treated as a serious business issue, dealt with by a corporate business function that have the political clout to fix the problems at source. Some progressive companies have set up such organisations, which may report into finance or another corporate function, but never into the CIO.

However, to show just how long a journey data quality can be, I can recall working with such a function in Esso UK in the mid 1980s. The issue is only now dawning on many companies, and still has to surface in most.