Make sure to watch the video in HD! The button is on the bottom right (HD). God bless Google's YouTube upgrade! :-)
Watch the Data Center Pulse YouTube Channel for more updates. We will be highlighting each of the vendors technologies as the testing is conducted. The testing is scheduled to wrap up by August 15th. The results will be presented at the Silicon Valley Leadership Group Energy Summit slated for end of September, 2009. Additional Chill off details can be followed at the official Chill Off Site on http://datacenterpulse.org.
On Wednesday September 24, 2008 we hosted the second AFCOM Northern California chapter meeting in Sun's Santa Clara, CA auditorium. I'm the board VP for this chapter that started in April 2008. The focus for this meeting was the Chill Off 2 and the first Data Center Pulse round table.
CHILL OFF 2
After Jim Gammond (membership), Eric Stromberg (President) and Maricel Cerruti (Communications) concluded chapter business, I presented a quick summary of the findings from the first chill off we hosted in June 2008. Then I gave a summary of the approach for Chill Off 2. Next we brought up a panel of subject matter experts to discuss the details between each other and the 80+ profesionals that attended. The panel I moderated included:
Mike Ryan, GDS Sr Staff Engineer - Sun Microsystems
Olivier Sanche, Sr Director - Data Center Operations for eBay
Phil Hughes, CEO of Clustered Systems
Bill Tschudi, Lawrence Berkeley National Labs (LBNL)
Mike was there to represent the vendors from the last chill off and the testing environment. He was the technical lead for the first chill off. Bill will be conducting the testing again this year as an independent party. Olivier has agreed to provide the workload definition that answers two important questions for ebay data center investments/operations. Phil is a new vendor in the test with a passive cooling solution that removes all the fans from the servers. We started with a few questions from me and then opened it up to the crowd for questions and input. It was a very lively session with most of the dialog coming from the audience which represented the entire data center community from consultants to engineers to manufacturers to end users. We received excellent feedback and were able to achieve what we set out to do, have the community infulence the direction of the chill off 2.
Some of the learnings/changes that will be made from this session:
Possibly have a mixture of hardware/vendors (blades, 1U, 2U, etc) in the test rather than all one form factor (sun x4100) - or do both?
2N testing down to N to compare efficiency losses.
Fault insertion that shows both equipment failures and humans faults such as opening the door of a container for maintenance, leaving a hole in an APC HACS, etc. Also, simulate moves adds and changes and their affect on the test environment.
Mix the work load in the test, not serial tests. So HPC, Web and enterprise all in one test as the mixed workload.
Add results from a tuned/contained raised floor/crac based DataCenter to the chart. Show the real differences, not just theoretical, against all the solutions. Current chart shows a traditional open raised floor/crac installation.
Standard communication protocol for data collection? How easily and effectively do these solutions tie into a BMS?
Compare main PDU, in-rack PDU wireless sensors, env monitors and server sensor efficiency. Include mixture of vendors that can meter down to the plug level and see how acurate they are compared to the internal server readings. Take temp and humidity readings across the equipment and the cooling devices to see how accurate they are.
A fully isolated environment so there are no questions about other in-room conditions affecting the tests. Also have the ability to watch the affect of raised water temps and raised inlet temp, on each solution.
TCO of each solution? This would be difficult, but would be very interesting data.
Remove all the server fans in the container and compare that to the regularly loaded test. In other words only use the containers in-line cooling fans to move air rather than the servers.
Do a full 3D CFD model of the solutions and compare that to what is actually seen. Future Facilities.
The presentations with the updates based on the session is coming soon. I would appreciate feedback on any area in the test/presentation. If there are companies that would be interested in including their products, services, support, expertise or time, please email me. email@example.com
DATA CENTER PULSE ROUNDTABLE
After a break for networking, we started what I hope to be a regular occurance at data center chapter meetings like this all over the world. Earlier in September, Mark Thiele and I started a new exclusive group that only includes datacenter owners and operators called Data Center Pulse. You can see the details in my earlier blog entry.
The panel we assembled was quite impressive. We had data center owners from Cisco, Apple, VMware, eBay, Stanford and the CEO of IDS, a new startup company putting datacenter co-lo space on container ships.
I asked a number of questions that had come up in our Data Center Pulse group through likedin, some others I had prepared and questions that came out naturally during the dialog/debate.
Check out the specific questions and responses in this round table session through the Data Center Pulse Blog. (click on blog)
In the end we ran out of time with the amount of discussion and healthy debate. After we were finished, I really felt that we should have filmed the session for the rest of the community to watch. Later, an industry friend of mine suggested that we hold these kinds of roundtable discussions on satelite radio so everyone could benefit from them. I thought that was a great idea, but we should try through a differnet medium. Video podcasts or YouTube. I travel quite a bit and could film these sessions and have interviews with different DC owners all over the globe. So, starting next week, Mark and I will be filming the first episode introducing the group. We have identified about 10 sessions we want to film including an upcoming trip I'm taking to Barcelona, Spain where I hope to have another DC Pulse roundtable session.
I'd be very interested in input from DC professionals around the globe. Is this something you would watch? Is it something you would want to participate in as a panelist? Would you be interested in letting us tour your DC to share the challenges and your solutions/learnings first hand with the community? Other ideas? We'd love to hear them.
On April 2, 2007 I met Ray Pfiefer from the Silicon Valley Leadership Group (SVLG). He was soliciting help in datacenter demonstration projects for SVLG. On May 21, 2007 I signed us up to host a demonstration project to compare cooling products for datacenters. We were undergoing a major consolidation and had the perfect environment to be able to host these tests. At that point I didn't realize how much we had signed up for.
Over the next year we coordinated the testing of five differant modular, closely coupled cooling solutions. It was dubbed the "Chill-Off". Brian Day, my chief of staff, was the program manager, Mike Ryan assisted in all of the mechanical/electrical details, Serena Devito coordinated all server and network needs, and Eric Garcia was the technican that racked, reracked and put the sweat into getting the tests setup. Power Assure did the server characterization and load testing. Modius did the measurement of over 1500 points, collecting over 40 million measurements. Tim Xu from Lawrence Berkeley National Labs (LBNL) was the independent party that defined the testing methodology then executed the tests and reported the results. The participants were Liebert XD, APC InRow, Rittal LCP+, IBM/Vette Cool Blue and Spraycool chip-level cooling. For this test we decided not to use new, high efficient equipment. We used older V20 Sun servers to create a more hostile environment for the solutions.
My goal in hosting this test was to get to find out how efficient these devices really were. We design datacenters all over the world and talk to customers and peers building datacenters almost every day. I was tired of hearing the marketing hype, opinions and posiitioning of these solutions. I wanted an independent test that could be shared with the industry so we all could make our own call on which one will solve the problem.
The effort payed off. On June 26, 2008, we hosted the SVLG Data Center Energy Summit on our Santa Clara, CA campus. We had over 300 datacenter customers and industry professionals show up to hear the results of this test as well as 10 others. I had purposely not looked at any of the results during the test. I waited until the draft reports were finished and reviwed them just like my peers and customers would. I then took what I gleaned from the reports and created a simplified presentation of the findings. I told the paricipating vendors that I wanted to spice the presentation up. The people didn't come to hear me talk, they came to get content to help them make informed decisions. So, I decided to present a quick history, have Tim describe the testing methodology, show a single results slide comparing the vendors and then conclusions. I got through this as quickly as I could and then called representatives from each of the companies on stage for a panel debate. I prepared a list of questions to ask them that would give them a chance to defend their results and spark debate. It was a blast. To give you an example, I asked Rittal why their numbers were so much higher than the others. I asked Liebert and APC to tell me why I would buy one of their products since they tied. I also asked all the vendors why we wouldn't just raise the inlet temperatures of the all the IT equipment to 80+ F degrees, use economizers and eliminate the need for their modular products. That one was fun. All of the vendors took my questions and attempts to increase the debate very well. I was hoping for more fireworks, but it was a great session. The audience was very receptive to the format and the questions.
You can get a copy of my presentation here (pdf format). You can get the final Accenture/SLVG report here. All of the case studies can be found here including our Modular Cooling Test, the Chill Off. In the end, I found a number things that stood out the most for me in this test.
All of the modular cooling solutions are more efficient than traditional datacenter designs.
As an industry, we need to keep pushinng the inlet temperatures higher. Our server fans did not kick into higher speed even with an inlet temperature of 80 degrees F.
If we can get these temperatures higher, it opens a huge amount of the world to economizers.
The collaboration and effort from all involved was incredible, even from from fierce competitors.
Lastly, I realized that we have just started. We need to continue these collaborative tests and challenge each other to push the envelope. At the end of the presentation & panel, I put the challenge out to our competitors and partners. Lets get more equipment and more solutions side by side in Chill-off 2. Containers, ducting, mixed vendor hardware loads and locations, highly optimized equipment via virtualization, raised air and water temperatures, 30kw loads, plus what ever we can come up with. I want to drive more independent content so that everyone who is contemplating datacenter construction or retrofits will be able to make informed decisions.
I will be soliciting input via a roundtable at the next AFCOM Northern California chapter meeting. Stay tuned for details. If you are not an AFCOM member, please contact Craig Easley (firstname.lastname@example.org) to get set up. I'm looking forward to an even better event next year.
PS. I gave a tour to some reporters and industry analysts the morning of the SVLG event. One of the reporters posted a video of the tour on youtube. Check out the two parts of the tour here.
PSS. Deborah Grove, from Grove Associates, interviewed some of the presentors during the SVLG event. See the summary video here.