Saturday Apr 11, 2009

iSER integrated!

iSCSI Extensions for RDMA (iser) was integrated into build 112 of Open Solaris. The team worked their butts off, and, at the end of a very long and very hard six months (probably too hard) put back the project. This action culminates some 18 months of work for Peter and his team. iSER is a key aspect in the Open Storage platform for IB connectivity solutions.

COMSTAR now supports Fibre Channel, iSCSI, IB (via iSER) and FCoE. We have achieved a true multiprotocol SCSI target. Now it is time to productize!

Well done team!

Thursday Mar 05, 2009

COMSTAR update

Just a little update on some of our latest developments:

1. We are getting ready to release a new SBD lun provider that will support PGRs, some cleaned up management interfaces and a couple of bug fixes. Should be out in the next 5 builds or so. Anyone that wants to help us with testing, send me a private note.

2. Tim Szeto, a lead engineer of mine is spending time thinking (deeply) about things like dtrace probes for COMSTAR, MDB walkers for COMSTAR, and kstats for COMSTAR. Look for open solaris discussions, ARC docs (join PSARC for that) and a storied discussion. Your observations and requirements are invited. Hit me.

3. iSER is just about ready to integrate. Would have put it back in 110, but, quiet build and all that. Don't want to destabalize the mother ship.

4. SRP target is cooking right along. Check it out, sign up for the developer's mailing list and poke at things. ( Peter C is rocking and rolling on the community update stuff. Shout at him if you need something!

5. FCOE is so very close. Roger's team is cooking this one pretty hard. Check it out at

6. iSCSI for COMSTAR screams! We knew it would, and it continues to get great reviews. It isn't perfect yet, but, it is turning heads. Check it out!

Summary? COMSTAR coupled with the deep traceability of DTrace and the power of The World Class filesystem is changing the storage game. My team continues to kick butt. Together we are crafting a superlative open storage foundation. Hop on board the Open Storage train.

Wednesday Mar 04, 2009


Wrote this in December and time got away....

My kids are growing up a bit, and it is both great and a little painful. For instance, both of my older kids came home with their report cards a day or two ago. The did well, and, I was impressed. (The 11 month old is still at the drool stage, but she can pick up a cheerio and put it in her mouth....Yea! Best baby!). Anyway, we discussed the report card which I turned into a riff on how to use the Fibonacci sequence for a simple cipher for passing notes in class and (probably) not get busted. That was a great conversation, and I felt comfortable sharing because I can still crack any code that they might come up with. At least today. And then we had snack. I am a bit of a freak about cooking, and I like to try new things etc. Snack for me is a fun thing to do. For this snack I took apples and sliced them to resemble teeth. I used Raspberries for eyes and nose and ears. I used a little bit of chocolate syrup for hair and eyebrows and of course the face was the plate.  It was fun, and it probably represents some of the more important things in life.

How do you respond to that?

I learned something new today. I was having a conversation with one of my daughters and we were talking about Easter brunch. I was waxing philosophic about the cool stuff at brunch that I liked and what a treat it was to go to something like Easter Brunch. My daughter cocked one jaundiced eye at me and disclaimed the "coolness". "Daddy", she said pointedly, "Brunch is just a way of cheating me out of one of my meals. I like Breakfast and Lunch." 

I had no response to that.

Wednesday Dec 03, 2008

Of scotch and flames

Tonight I got a card from a friend of mine. It was a little card, and it was a thank you. In that card this fellow was thanking me for a bottle of scotch and a kind note that I had sent him due to a recent loss. He quoted Albert Schweitzer, and likened me to someone that helped his flame (at little bit) back to life. The card couldn't have come at a better time. I am not up to the praise he gave me, but, it helped a lot that he felt better. And, I felt better by his gesture.

Marine, if you read this, thank you. I needed that in ways you can't begin to imagine.

"In everyone's life, at some time, our inner fire goes out. It is then burst into flame by an encounter with another human being." A. Schweitzer

Monday Nov 10, 2008

Disaster Recovery in Solaris

Mike La Spina, an engineer in Canada has been a member of my virtual team for about 6 months now. Mike is a great representative of the Open Solaris community, a strong advocate for quality (he doesn't really let me get away with anything) and a voice of reason when it comes to VMware, iSCSI and of course Thumpers. He connected with me in an effort to get me to fix a bug. We fixed it, he continues to suggest ways for us together, and that, to me at least, means team work.

Recently Mike contacted me about his DR plan, and I asked him to make sure he pointed me towards the blog when he published. Mike just posted a day or two ago and it is a great read. I would encourage you to take a look at it. Mike continues to drive as a tireless ambassador for Solaris and iSCSI and ZFS. All cool stuff.

Provisioning Disaster Recovery with ZFS, iSCSI and VMware

Sunday Nov 09, 2008

Target cheat sheets...

Last week I presented COMSTAR futures to the DMA (Data Management Ambassadors). We talked about the strength of the COMSTAR framework and a fair bit about the adoption curve. The truth is, hype and pride aside, COMSTAR is a game changer for us. The naive implementation is merely third tier storage or interesting scratch disks. However, as people become more comfortable with the employment of COMSTAR, they are recognizing that the strength of COMSTAR isn't so much that you CAN do these types of things but rather you can essentially deploy world class data services at a fraction of the cost. And, it is easy to do. So the possibility is merely exciting. The savings is revolutionary. 

I also promised to post the matrix of targets... Here it is...

Component S10 Nevada Open Solaris 2008.11 Open Solaris 2009.04
Use Build 100




No, but you can apply on top.

Tuesday Oct 21, 2008

Cool visual metric:

I haven't had a lot of time for blogging lately. Given the collision of work projects that are just about done, I have been hanging on by a fingernail. All that being said, my wife pointed out a cool metric. She took a snap shot of all my blog entries and ran them through It gives a visual representation of what I blog about. It seems pretty accurate to me in terms of attempting to quantify the relative weight of what I am interested in and validation that I talk about what I believe in. The number one word is "Team".

Wednesday Aug 06, 2008

Getting there

14:46:53.401 Starting RD=run1; I/O rate: (vdblite); Elapsed: 60 seconds.
For loops: xfersize=524288 threads=4

 interval                            i/o         MB/sec   bytes    read       resp    resp       resp       cpu%      cpu%
                                        rate       1024\*\*2     i/o     pct         time    max       stddev   sys+usr    sys
14:47:05.071         1    1861.87   930.94  524288 100.00    8.589  22.702    1.302    44.0         43.6
14:47:14.030         2    1863.34   931.67  524288 100.00    8.586  36.584    1.272    44.6         44.0
14:47:24.026         3    1863.05   931.53  524288 100.00    8.585  35.810    1.192    43.9         43.3
14:47:34.026         4    1862.25   931.12  524288 100.00    8.586  25.890    1.293    44.3         43.8
14:47:44.027         5    1864.35   932.18  524288 100.00    8.579  28.986    1.126    44.0         43.6
14:47:54.024         6    1858.23   929.12  524288 100.00    8.607  65.024    1.710    44.6         43.0

Wednesday Jul 02, 2008

Multipathing for Tape

It was a dark and stormy night, the back up windows opened and shut with the regularity of a 90 year old pensioner's clacking of teeth as he snored with the flem filled nervousness of old age and incontinence. The data flowed from the backing store to fixed content archives like a humpback whale strains plankton. The administrator anxiously clacked her gum back and fourth against the two teeth she had recently had crowned with a couple of diamonds (one shaped like an "i" the other clearly an "o"...some of the geeks at work thought it was a one and a zero, but, they don't understand IO) as she watched the progress of her back up. All of a sudden there was a rigid silence as the status window showed zero throughput and the tortured silent scream of failed IOs bounced off the ear buds of the administrator. With manic obsessiveness the administrator chanted her mantra "If an IO fails to complete, is it ever an IO?" and she anxiously counted off the seconds before the back up timed out. ....Over and over again she chanted her mantra as the digital clocked ticked with the slowness and regularity of a clock:

7..8..9..."...fails to complete..." 17...18....19... "is it ever an IO?"....28...29... (occasionally she changed her mantra to "I think I can I think I can" as she took power hits of her RedBull) 47...48...49... And then, with the almost ecstatic relief one feels as a particularly large boil is lanced, the IO kicked back in down the alternate path and the back up proceeded. 

So, with a rather shameless head nod towards the "The Bulwer-Lytton Fiction Contest" I announce that we have integrated multipathing for tape into Solaris build 93. This is the culmination of a lot of work in both the ST driver as well as MPxIO. In particular, we have solved the problem of logical block addressing and error recovery as well as true multi-pathing to multi ported tape drives. Additionally, this is a platform for innovation as well. Now that we have highly available tape back up windows that are natively part of the Solaris operating system (you could do some of this before with FCP2 Error recovery, but it wouldn't help with a bad HBA or cable) we can look at bus saturation so we can attempt to drive down back up times.

A strong head nod towards our experts in tape and MPxIO. Well done Randy and Wayne

Tuesday Jun 17, 2008

Fun in the Sun

Deirdré Straughan, Community-Monger and Videoblogger Extraordinaire came to work with me about 18 months ago. She now works in a different organization doing fun things that rightly play to her talents. We grabbed some time and some equipment and some good ideas and together we build a pretty innovative program around storage software. Which is not really the plot of this post, but, interesting background information.

Anyway, she had a cartoon done of her  several years ago that I thought was pretty fun. In May I got one done as well. I like to think of it as sort of "The Matrix meets Tequila Sunrise" (I have always thought I was a mix of the best parts of Keanue Reeves and Mel Gibson) <big chuckle>. Anyway, here is the picture. The artist was great and very reasonable. Deb doesn't like the white spot on my nose, but, I think I actually have one there, so, it all works.  I like it a lot and Mike was fun to work with.

We now return from this two minutes of narcissism. Had this been a real narcissistic emergency, I would have written some poetry.  These are not the droids you are looking for. Move along.

Friday Jun 13, 2008

Rock star guitar hero

Sumit is still a rock star. Here is his latest guitar riff. Seriously, ZFS, DTRACE and COMSTAR: What more could you want on a phenomenal Friday in June?

Playing to the level of desire

Last night I played a game of hockey against an all girls team. I had a bad feeling when I walked into my locker room and the captain announced that we were going to "take it easy" tonight. Three passes in the zone, no break aways and all that stuff. I was truly aghast and then I heard it was due to us playing an all girls team.

I totally disagree with the philosophy that you play a different game depending on your perception of the competition, unless it is to change the game up in order to win or play better. Dumbing down our game is insulting, not what I signed up for, doesn't do anything but fuel misogynistic tendencies and frankly, makes the other team worse. Will you play as hard as you can if you recognize that the other guy (gal) is just screwing around? Nope.

So, I played hard. I gave my best efforts against the team I played with, and they played hard. The goalie (my buddy) taught them how to check a huge guy and get away with it. He applied his stick in places that he shouldn't have. I ended up getting boarded (hard) by someone that was a foot shorter and 100 Lbs smaller. Good for her. I had a ball and I was a human pinball. (Another point, what ref is going to call boarding on some one I could bench press?) They had a break away on me because I over skated and lost the puck. If I was taking it easy, they never would have had the break away.

It was hockey, they played hard and I honored them by playing my best. There is no shame in defeat. There is only shame in not playing or trying. Play your best game. Always. You can teach and you can make them better players by playing a good clean game and playing it honestly. I think that is life too. People raise to the level of expectation and to their level of desire. 

This seems so clear and so simple to me.  

Thursday Jun 12, 2008

"New Sun code transforms x86 servers into storage arrays"

The guys over at Computerworld wrote a little blurb about COMSTAR:

Open-source Comstar software enables servers to act as block-storage devices

By Brian Fonseca

Anyone out in the world want to share a story, complaint, request or <ahem> praise? Shoot me a note.

For those of you living under a rock, check it all out right here at:

Wednesday Jun 11, 2008


The COMSTAR press is starting to roll out and it is very cool. My team has worked on COMSTAR for more than a year, and, people from Roger Dong's team and of course our test resources and our docs folks have spent a lot of energy making the project and making it successful. Lynn blogs about it here. I expect a fair bit of press over the next few weeks. Deirdre' helped us get the campaign rolling and it is all staring to come together!  My very first Chinese blog was about COMSTAR.



« July 2016