Andy on Enterprise Software

Data mis-governance

October 22, 2007

I spent this morning at a data governance seminar sponsored by Dataflux, at which Jill Dyche or Baseline Consulting spoke about her experiences of data governance best practice in client organisations, and Philip Howard of Bloor gave his perspective. Data governance seems to be something very much in its infancy despite the long-established issues it addresses, with only a tiny proportion of organisations having made a lot of progress (according to an IBM Global Services 5 point data governance maturity scale, no company is further along than stage three, and only a handful of companies even manage that). There seems little in the way of a silver bullet here, just missionary work to convince the business that data ownership needs to be taken seriously. Sometimes a “burning platform” can stimulate interest. Recently Nationwide Building Society was fined GBP 1 million due to the theft of a laptop on which customer data was stored (albeit in encrypted form). Interestingly, the fine was not directly due to the loss of the data but the fact that they had no processes in place to determine that there was actually customer data on the laptop. Such cases illustrate the risks, at least in regulated industries, of having poor data governance polices.

Another aspect of data governance often overlooked is the proliferation of data in corporate spreadsheets. Apparently Allied Irish Bank have a stunning 185 TB of storage devoted to spreadsheets alone, and who knows how much of this is duplicated. With studies showing that, in a spreadsheet with over 200 rows there is a 90% chance of an error, the potential for problems is self evident. When I was at Shell there was a whole group on the corridor opposite me who built spreadsheet models and audited existing ones, some of which are highly important (e.g. financial models used for capital intensive projects). This group paid its way many times over by uncovering flaws in existing operational models. Yet I suspect they only scratched the surface, and how common are such initiatives? This should be a promising area for companies such as Compassoft, which do spreadsheet “discovery and control”. Indeed there are no shortage of scandals related to manipulation of spreadsheets, including the USD 700M one at Allied Irish. And you thought you had enough data quality problems in your corporate systems….