I have recently been spending some time looking at the data quality market, and a few things seem to pop up time and again. The first thing is, in talking with customers, just how awful the quality of data really is within corporate systems. One major UK bank found 8,000 customers whose age was over 150 according to their systems. All seemingly academic (if you are taking money out of your account, who cares what your age is?) until some bright spark in marketing decided that selling life insurance to these customers would be a fine idea.
Story after story confirms some really shocking data errors that lurk beneath most operational systems. These are the same operational systems that are used to generate data for the end-year accounts which senior executives happily sign off on pain of jail-time these days. I hope no one shows these sames execs the data inside some of these systems, or they might start to get very nervous indeed.
Yet in a survey we did last year, only about a third of companies in the survey have invested in data quality tools at all! Does anyone else find this in any way scary? Do you have any entertaining data quality stories you can share?