By kemer on Jul 12, 2012
One might think that things like disk space and even computer speed had become irrelevant. After all, our cell phones have more computing power and storage than million dollar computers of fifteen years ago. (Stop me if I’ve told this story too many times: 25 years ago we sold a terabyte of disk using the big 200 MB disk platters. So, that was a total of over 4,000 disks. Since this is ancient history, the details are hazy, but I do remember that it involved a sizable facility for the storage alone and an even more sizeable commission for the sales rep. Those were the good ol’ days!)
The truth is, there is always an opportunity to take advantage of more resources. Indeed, we are in the era of big data and it would seem that our big limitation now is the speed of light. Rather than brute force, clever engineers continually come up with better ways of doing things. The RDBMS world has tended to think in terms of rows, but there is a new trend to organize it in columns instead. Wikipedia has a great summary of the pros and cons worth taking a look at, if this is new to you. In a nutshell, columnar databases can provide real performance advantages for data warehouses.
Oracle’s Hybrid Columnar Compression technology is nicely described in this paper. Long time storage specialist Art Licht has written a paper about a study he did, explaining How We Improved SAN and NAS Performance with Hybrid Columnar Compression with some remarkable results: 19x reduction in required storage space and an average 6x increase in database query performance.
Art provides specifics on how to do this using the Pillar Axiom Storage System and Sun ZFS Storage Appliance, with detailed test results. This is an article you don’t want to miss: a real hands-on description that quickly brings you up to speed with the technology and its application in the real world.