Andy on Enterprise Software

Vendor due diligence

February 21, 2006

In a previous blog I gave some general thoughts about vendor evaluation, and expanded on this to give an outline framework for such evaluations. One thing that should be considered in any evaluation is “his stable is the vendor” i.e. will they still be around in a few years? This question can be surprisingly hard to answer, and in fact the question itself has a flaw. The question should be: “will this product still be further developed in a few years?”. The reason for the distinction is that even the largest vendors sometimes discard technologies due to changes in their product roadmap, internal political issues or because the thing isn’t selling very well.

So, if buying from a very large vendor, it will be easy enough to look at their general health because their finances are public (OK, there are sometimes accounting frauds but you can’t do much about these). The usual ways of analyzing company’s health can be used here. I highly recommend the “Guide to analyzing Companies” by the Economist, which is clearly written and gives excellent examples of the indicators of company health and also of early warning signs. You should not assume that just because a company is publicly listed then it must be OK – ask the customers of Commerce One about that approach, for example.

In such cases you will probably find that the company is fine, so the bulk of the diligence efforts should instead be directed to how important this particular product is to the company, and so assess how likely is to to get ongoing development. Of course the vendor is hardly a reliable source here, but you can seek advice from analysts, and also it is fair to ask the vendor
just how many customers of this particular product there are (you should be able to talk to some of them). For example, SAP’s MDM product had seemingly shifted just twenty copies into customer use throughout its 18,000 strong customer base in two years. Given this dismal penetration rate it was perhaps not a shock that they dropped it (their replacement MDME offering is based on an acquisition of PIM vendor A2i). Anything which is not contributing to a vendor’s sales figures on any scale should be considered suspect.

In the case of small vendors you have different problems. You can be pretty sure that the product is important to the vendor, since it is probably the only one that they have. The question is whether the vendor will survive. This is trickier, since the company is probably privately held, and so is not obliged to publish its accounts, at least in the US. In the UK you can get around this by paying a few pounds to Companies House and look up their acocunts.
If you are making a large purchase then it is fair game for you to ask for information on the company financials, and you should get nervous if they refuse. One thing that will achieve little is to ask for a letter from the VCs backing the company. They will inevitably sing its praises; they are hardly going to say “ah, this one’s on a bit of a knife-edge, I’d watch out if I were you”. Indeed I knew of one case where a major deal was in progress at a BI vendor, and through a contact I became aware that the entire future financing of the (cash strapped) company was dependent on this deal going ahead; in such cases you cannot expect objectivity from investors.

So, what can you do? Well, profits are an opinion but cash is not. Hence, assuming you can see some figures, you can get a sense of how much cash the company has, and attempt to work out the “burn rate” i.e. how fast are they burning through this cash (most VC backed start-ups are unprofitable; if they were profitable then they probably wouldn’t need expensive VC money). However this on its own may give false signals. Due to their IRR-driven instincts, VCs don’t dole out more cash than they need to start-ups; they like to always have the option of pulling out if they need to, so it is rare for a start-up to actually have more than about one year’s cash needs in hand. The question is: will they be able to raise more cash if they need it? This is a complex subject, but essentially you should be able to get a sense of this by talking to analyst familiar with the VC community. For example, companies growing at 50% or more are very likely to be able to raise cash, even if they very unprofitable. The gross margins in software are commonly 90%, so profits will come eventually if the company can just grow large enough; this is why VCs invest in software companies more than, say, restaurants. So if you cannot find someone knowledgeable to look the figures over for you and make an assessment, then a decent proxy for security is the revenue growth rate. If the company’s growth is stalling (say 10% a year growth for a small-medium software company) then things could be sticky in a future financing round. This is a generalization (and companies with a subscription model, for example, have a much more predictable life than ones selling traditional licenses) but it may be the only real set of figures that you can dig out.

Another source of due diligence is other customers, who may well have done exactly the same due diligence exercise as you fairly recently. Of course you have to be careful it was not out of date, and you should check how thorough they really were, but you may be able to save yourself a lot of work. If three Fortune 100 companies recently did detailed due diligence on a vendor and bought its software anyway, this may help you at least feel better.

Remember: the company or product does not have to be around in ten years if your payback case is 13 months. The faster the payback period for the product, the less concerned that you need to be about agonizing over the long term future of the company, or of the product within the vendor. You did do a proper business case for the purchase, right? Of course you did.

Evaluating Software Vendors – a framework


I have written previously in general about vendor evaluation processes. Over time I will flesh this topic out further, as I feel it is a much-neglected area. As I have said, it is important to define up front your main functional and technical requirements from the software package that you want to buy. It is then important to have a process to take the “long list” of candidates down to a manageable number of two to four vendors to evaluate properly.

So, you have done this, and are down to three vendors. What are the mechanics of the evaluation criteria? I show in the diagram a simplified example to illustrate. It is critical that you decide on what is most important to you by selecting weightings for each criteria before you see any products. Ensure that you group the broad criteria into at least two and perhaps three categories: functional should list all the things you actually want the product to do, and you may choose to separate “technical”, which may include things like support for you particular company’s recommended platforms e.g. “Must run on DB2″, or whatever. What is sometimes forgotten is the commercial criteria, which are also important. Here you want things like the market share and financial stability of the company, how comprehensive its support is, how good is its training etc. I would recommend that you exclude price from these criteria. Price can be such a major factor that it can swamp all others, so you may want to consider it as a separate major criteria once you have done the others. I would recommend that the “functional” weightings total not less than 50%, It is no good buying something from a stable vendor if the thing doesn’t do what you need it to.

An important thing about using a weighting system like this one is that the weights must add up to 100. The point here is that it forces you to make trade-offs: you can have an extra functional criteria, but you must reduce the existing weights to make sure that the weights still add to 100. This gives the discipline to stop everything being” essential”. You assign all the weights before the evaluation begins. You can share this with the vendors if you like. Coveniently, however you assign the weights, the scores will come in out of 1000, so can be easily expressed as a percentage e.g. vendor B is a 74% match to the criteria in the example, while vendor C is 67%.

The final stage is that you need to score the various criteria that you have laid out. You want this to be as objective as possible, which is why you do not want too many – you want to see evidence for the functional criteria. Just because the salesman says that it does something is not sufficient to credit a score – you need to see the feature yourself, preferably working against some of your own data rather than faked up demo. I recall doing an evaluation of BI tools in 1992 at Shell and having one vendor with quite a new product that due to a stellar analyst recommendation made it to the short-list. When the pre-sales guy turned up and was presented with a file of our data for him to do the trial on he went white; their whole product was virtually hard-coded around their demo dataset, and it quickly became clear that even the slightest deviation from the data they were expecting caused the product to break.

Score each criteria out of 10. Commercial criteria can be done off-line and in advance; analyst firms can help you with this, as they tend to be up on things like market shard (IDC have the most reliable quantified figures, but rough estimates are probably good enough). Financial stability is a subject all in itself, and I will cover this in another blog.

The evaluation then becomes quite mechanical, as you crank out the scores. You see that in this simplified example vendor B has won, though not by a huge margin. If it turns out that vendor B’s price is twice that of the others then you may decide this difference is not big enough to justify the slightly better scores (we will retunr to this shortly). Again, you could weight price as a factor if you prefer.

However, don’t get too hung up on price; as someone who used to do technology procurement it may seem like the be all and end all, but it is not. The total cost of a new software package to your company is far greater than the initial license cost. There is maintenance and training over several years, and also the people time and cost in actually implementing the technology, which will usually be several times the cost of the software package. Hence getting a package that is 20% more productive than the next best is worth a lot more than 20% extra in the license price, as the associated costs of people will be multiples of the software cost (people costs being five times package software costs in a project is common, ten times is not unusual). It is sensible for you to try and consider the likely full lifetime costs of the software in this way (assume, say five years) since you will then have an idea as to how important the license cost really is. For example if you are about to do a 30 country roll-out budgeted at USD 50 million, making sure that the product you select is the most productive one is a lot more important than if you are doing a single project for USD 500k. Here a product that is 10% more productive than the next one to implement may save you USD 5 million, so haggling to the death over that last USD 10k of license discount may not be so critical. This will give you a true bottom line case for the level of spend you can afford to make.

Taking a structured evaluation approach like this has a number of benefits. Firstly, it reduces the amount of gut feel and “did I like the salesman” mentality that too often creeps in. You’ll probably never see the salesman again unless you want to buy more software, but you will be stuck with the product that you select for years. Secondly, it gives you a documented case for selection that can, if necessary, be used to back up things internally e.g. in the case of an audit, or just to give comfort to senior management that a sound process has been used.

Moreover, given that salesmen get paid on how much they sell you, you’d be surprised at the tactics they can adopt; they will try and go over your head if they think they are going to lose, and make all sorts of accusations about how the process wasn’t fair and how you are about to make a terrible mistake, so having a solid, documented case will make it much easier for your manager to do the right thing and tell the salesmen to take a running jump. I am amazed at how often this tactic was tried when I was procuring software, but I never once had a decision overturned. If you ever find yourself in this situation, remember that revenge is a dish best served cold. After a particularly acrimonious session with one vendor salesman when I was working at Exxon, I was amused to find, a few years later, the same salesman turning up a few years later when I transferred to Shell. He walked in the room, his face fell when he saw me and he walked back out again. Good professional sales staff know that the world is a small place and that it does not pay to aggravate the customer, but all too few remember this.

In another blog I will return to the subject of assessing the financial stability of vendors.