New Models in Warehousing and PaaS

February 29, 2008

Is a column-oriented database the optimal format for a warehouse? Database pioneer Michael Stonebraker thinks so. InformationWeek reports Stonebraker’s assessment that a column-oriented database improves warehouse performance 50x, and the larger the warehouse, the greater the gain. Why? Warehouses typically store transactional data. Where a row of data stores many pieces of one transaction, the typical row-based DBMS would retrieve all rows then aggregate the selected column, a column-based DBMS would not require the same overhead of row processing. Because column information is generally of similar format, columns could also gain compression and storage efficiencies. Interesting thoughts. How viable is a column-based platform? I’m not sure, but Vertica has secured $23.5MM in venture funding to find out.

At the same time Sybase has funded column-based research since the mid-1990’s. An also-ran in the database world, Sybase saw revenues up 70% last year “because the column approach yields better query performance,” says Sybase Engineering VP, Richard Pledereder.

Column-oriented DBMSs require rethinking the data and indexes because the transaction is not the central idea. Instead, the data architect must think in terms of collections of similar records, and subject based indexes rather than transactional element indexes.

One company, Sonian Networks, archives e-mail for other businesses housing data in Vertica’s data warehouse on Amazon.com’s Simple Storage Service. Which segues into the platform as a service model. Sonian expects its warehouse to grow from a few terabytes to a petabyte sometime in 2009. And to deliver, Sonian relies on infrastructure hosted elsewhere. Sonian develops the warehouse platform and releases to a hosted environment. Their clients never miss a beat and always have the most up-to-date platform. Which reminds me of the Salesforce.com platform-as-a-service model.

If you haven’t heard, Salesforce.com, Oracle, and Google have partnered to bring PaaS to an application near you. In this model you can develop business solutions on the Salesforce.com APEX platform that targets every end-user device without having to develop custom code for each device and, drumroll here, without having to manage the infrastructure behind the applications. Right now companies must manage the intricacies of their infrastructure along with devoted staff to ensure users can perform their business functions. With PaaS, your company can continue to narrow it’s strategic and tactical focus to the services and business solutions that matter, and can offload the infrastructure responsibilities to partners who effectively do this. Okay, we’ve heard that before. Yes, except that Salesforce.com signed up 100,000 customers soon after its announcement. With a goal of disrupting the Microsoft model of software delivery, we’ll see where this goes.

- Andy

Share and Enjoy: These icons link to social bookmarking sites where readers can share and discover new web pages.