Wednesday Nov 16, 2005

Thank You

I was originally planning to blog about RAID-Z today, but I can't.

There's been such a tremendous outpouring of energy and passion for the ZFS launch today that the only thing I can think about is this: thank you.

Eric Schrock and Dan Price have been tireless in their efforts to get the OpenSolaris ZFS community up and running. Bryan Cantrill has rounded up all the ZFS blog entries from today, which run the gamut from informative to surprising to downright funny. And for me, as the papa bear, every one of them is also personally touching. Thank you for using ZFS during the early days, for providing insights on how to make it better, and for taking the time to write up your personal experiences and spread the word.

I'm incredibly proud of the ZFS team, and deeply grateful: no one person can do something on this scale. Thank you for making the leap of faith to join me in this effort, to create something out of nothing, to take an idea from vision to reality. It has been an amazing and wonderful experience.

To Cathy, Andrew, David, and Galen: thank you for support, encouragement, patience, and love. Six years is a long time to wait for Daddy to finish his project. But somehow, you managed to understand how important this was to me, and for you, that was good enough: you've been a constant source of energy the whole way. I can't imagine doing it without you.

Awww, now I'm getting all weepy.

Please come back tomorrow and I promise we'll have that chat about RAID-Z.

Technorati Tag:
Technorati Tag:
Technorati Tag:

Monday Oct 31, 2005

ZFS: The Last Word in Filesystems


Halloween has been a special day for ZFS since its inception.

On 10/31/2001, we got the user-level prototype working.

On 10/31/2002, we got the first in-kernel mount.

And today, 10/31/2005, we integrated into Solaris. ZFS will hit the street in a couple of weeks via Solaris Express.

We (the ZFS team) will have much more to say about ZFS in the coming weeks. But tonight, I just want to tell you what it was like to drive this thing home.

The ZFS team is distributed: we have people working in Menlo Park, California; Broomfield, Colorado; and several other places. This week, we flew everyone in to Menlo Park and took over a giant conference room.

The first thing you notice is the heat. These rooms are made for 100 watt humans, not multi-gigahertz CPUs and associated paraphernalia. And, like any big company, Sun is all about saving money in the dumbest ways -- like turning off the A/C at night, and outsourcing the people who could possibly turn it back on.

At first, things went pretty well. We comandeered the Nob Hill conference room, which has a long table and lots of power and network taps. We brought in a bunch of machines and created 'Camp ZFS'. Each new box amped up the heat level, to the point that it became difficult to think. So we grabbed a 36-inch fan from one of the labs to get the air flowing. That was a huge help, although it sounded like you were wearing a pair of lawn mowers as headphones.

On Sunday, we plugged in one more laptop. That was it -- we blew the circuit breaker! So here we are, less than 24 hours from our scheduled integration date, and all of our computers are without power -- and the room containing the circuit breakers was locked. (Thanks, OSHA!)

So we took a three-pronged approach: (1) went through the Approved Process to get power restored (ETA: April); (2) hunted down someone from campus security to get us key access to the electrical room (ETA: Sunday night); and (3) sent our manager to Home Depot to buy a bunch of 100-foot extension cords so we could, if necessary, run Nob Hill off of the adjacent lab's power grid (ETA: 30 minutes).

All three came through. We ran half of the load over extension cords to the lab, the other half on the Nob Hill circuit. It took a bit of experimentation to find a load balance that would stay up without tripping the breaker again. Apparently, we had angered it. (Even now, I'm typing this blog entry courtesy of a shiny new yellow extension cord.)

Meanwhile, the clock was ticking.

At the end of a large project like this, it's never the technology that kills you -- it's the process, the cleanup of home-grown customizations, the packaging, the late-breaking code review comments, the collapsing of SCCS deltas, stuff like that. With power back up, we slogged on until about 4AM. Everything looked good, so we went home to sleep. Actually, some people just crashed on the couches in my office, and Bill's office next door.

By 10AM Monday we were back, making sure that all the final tests had run successfully, and working through the last bits of paperwork with the Solaris release team. After five years of effort, it was time to type 'putback'.

Denied! The permissions on a directory were wrong. Fix them up, try again.

Denied! One more TeamWare directory with wrong permissions. Fine, fix that too.

Last try... and there it goes! 584 files, 92,000 lines of change, 56 patents, 5 years... and there it is. Just like that.

Fortunately we were prepared for either success or failure. We had brought in a massive array of vodka, tequila, wine... you name it, we had it. And not in small quantities.

As I said at the beginning, we'll have lots more to say in the coming days. But right now, it's time to sleep!


Friday Sep 24, 2004

128-bit storage: are you high?

One gentle reader offered this feedback on our recent ZFS article:

64 bits would have been plenty ... but then you can't talk out of your ass about boiling oceans then, can you?

Well, it's a fair question. Why did we make ZFS a 128-bit storage system? What on earth made us think it's necessary? And how do we know it's sufficient?

Let's start with the easy one: how do we know it's necessary?

Some customers already have datasets on the order of a petabyte, or 250 bytes. Thus the 64-bit capacity limit of 264 bytes is only 14 doublings away. Moore's Law for storage predicts that capacity will continue to double every 9-12 months, which means we'll start to hit the 64-bit limit in about a decade. Storage systems tend to live for several decades, so it would be foolish to create a new one without anticipating the needs that will surely arise within its projected lifetime.

If 64 bits isn't enough, the next logical step is 128 bits. That's enough to survive Moore's Law until I'm dead, and after that, it's not my problem. But it does raise the question: what are the theoretical limits to storage capacity?

Although we'd all like Moore's Law to continue forever, quantum mechanics imposes some fundamental limits on the computation rate and information capacity of any physical device. In particular, it has been shown that 1 kilogram of matter confined to 1 liter of space can perform at most 1051 operations per second on at most 1031 bits of information [see Seth Lloyd, "Ultimate physical limits to computation." Nature 406, 1047-1054 (2000)]. A fully-populated 128-bit storage pool would contain 2128 blocks = 2137 bytes = 2140 bits; therefore the minimum mass required to hold the bits would be (2140 bits) / (1031 bits/kg) = 136 billion kg.

That's a lot of gear.

To operate at the 1031 bits/kg limit, however, the entire mass of the computer must be in the form of pure energy. By E=mc2, the rest energy of 136 billion kg is 1.2x1028 J. The mass of the oceans is about 1.4x1021 kg. It takes about 4,000 J to raise the temperature of 1 kg of water by 1 degree Celcius, and thus about 400,000 J to heat 1 kg of water from freezing to boiling. The latent heat of vaporization adds another 2 million J/kg. Thus the energy required to boil the oceans is about 2.4x106 J/kg \* 1.4x1021 kg = 3.4x1027 J. Thus, fully populating a 128-bit storage pool would, literally, require more energy than boiling the oceans.




« November 2015