Andy on Enterprise Software

The surprisingly fertile world of database innovation

July 24, 2007

I came across a thought-provoking article, an interview with Michael Stonebraker. As the inventor of Ingres this is someone who knows a thing or two about databases, and I thought that some interesting points were raised. He essentially argues that advances in hardware have meant that specialist databases can out-perform the traditional ones in a series of particular situations, and that these situations are in themselves substantial markets that start-up database companies could attack. He singles out text, where relational databases have never prospered, fast streaming data feeds of the type seen on Wall Street, data warehouses and specialist OLTP. With Streambase he clearly has some first-hand experience of streaming data, and OLTP is what he is working on right now.

I must admit that with my background in enterprise architecture at Shell I underestimated how much of a market there has been for specialist databases, assuming that the innate conservatism of corporate buyers would make it very hard for specialsit database vendors. Initially I was proved right, with attempts like Red Brick flickering but quickly becoming subsumed, while object databases were clearly not going to take off. With such false starts it was easy to extrapolate and assume that the relational vendors would simply win out and leave no room for innovation. However to take the area of data warehousing, this has clearly not been the case. Teradata blazed the trail of a proprietary database superior in data warehouse performance to Oracle etc, and now Netezza and a host of smaller start-ups are themselves snapping at Teradata’s heels. The in-memory crowd are also doing well, with for example Qliktech now being the fastest growing BI vendors by a long way, thanks to its in-memory database approach. Certainly Stonebraker is right about text – companies like Fast and their competitors would not dream of using relational databases to build their text search applications, an area where Oracle et al never really got it right at all.

Overall there seems to be a surprising amount of innovation in what at first glance looks like an area which is essentially mature, dominated by three big vendors: Oracle, IBM, Microsoft. Teradata has shown that you can build a billion dollar revenue company in the teeth of such entrenched competition, and the recent developments mentioned above suggest that this area is far from being done and dusted from an innovation viewpoint.

The tortoise and the hare

December 15, 2006

Business intelligence applications typically deal with data that is already stored, often in a depressing number of places i.e. a number greater than 1. Much BI data is of its nature not real-time e.g. looking at monthly averages or trends. However at the other end of the spectrum there are some applications that are truly real-time, and not just in the sense that a marketer puts the term in a brochure.

An interesting start-up in this area is StreamBase, which specialise in genuinely real-time applications, such as trading systems but also inventory monitoring and anti-fraud applications. StreamBase provides StreamSQL, which essentially extends SQL to a real-time environment, a run-time engine as well as a graphical developer environment that allows transformation logic to be written. For example you might have a need to compare the current stock price of an equity to a competitor, and take some action e.g.”buy” if the price hits some threshold related to the competitor. Such applications would occur in memory, but StreamSQL also provides in-memory hash tables and also an embedded database in case you want to persist data (temporarily or permanently, respectively). To continue the trading example, you might want to take an action based on the stock price relative to its average over the last month, for which you would need to store some data temporarily in order to carry out the calculation.

Set up in 2003 by database luminary Mike Stonebraker (who founded Ingres and Illustra) the company has now done two venture rounds, including a series B round led by premier league VC Accel. They have over 60 employees and 50 customers, though this includes pilot customers i.e. not all these are fully paying yet. Public customers include Goldman Sachs and Bridgewater, a leading hedge fund. The company is cagey about revenues but assures me that they are growing.

StreamBase plans to provide in its product roadmap easier integration of real-time and historical data i.e. more StreamSQL enhancements, continued performance enhancements and improved ease of programmability. Two OEM agreements are already in place.

Competition is mostly in-house coding, though there are some vertical point solutions (Progress software in trading) and there is potentially some overlap with EAI tools at some point. For example, StreamBase partner with Tibco Rendezvous but there is an offering of Tibco for event processing that could potentially compete. From a marketing viewpoint the relationship to EAI and middleware tools will need to be carefully stressed as these tools themselves develop. However the company has certainly picked an attractive niche to operate in, and its high quality VC backers and experienced management will make it a credible player.