Tuesday Jun 16, 2009

Paris in the Clouds: A CloudCamp Paris report

CloudCamp logoLast week, Eric Bezille invited me to Paris for a couple of Cloud Computing related meetings and to help out with CloudCamp Paris. Paris in the clouds, what a nice experience!

This was also a great opportunity to try out the audio recording features of my LiveScribe Pulse pen. This pen not only can record what you write (on special dot paper), it can also record what has been said while you write, creating links between the words you write and the points in time of the audio recording. Very cool. You can then tap on the words in your notebook and the pen will play back the associated audio. Great for conferences, and I wish I had had this pen during my university times :). You can also export your notes including the audio as a flash movie and share them on the net, which is what I'm going to do below.

Intro Session and Lightning Talks

The CloudCamp was kicked off by a representative of Institut Telecom, the location sponsor of CloudCamp Paris. Sam Johnston gave a short and sweet introduction to Clouds, providing some definitions, examples and also some contrarian views, finishing with a short video on how easy it is to set up your account in the cloud.

A series of lightning talks by the sponsors gave us some interesting news, insights and context for the conference:

  • Eric Bezille from Sun showed us what's behind Sun's cloud activities.
  • Arvid Fossen from Aserver.com talked about how they provide datacenters as a service to their clients. Wanna have your own cloud? Go buy it as a turnkey solution!
  • Matthew Hugo (Not sure if I got that name right...) from Runmyprocess.com showed some nice examples of integration between different cloud services.
  • Josh Fraser, VP of Business Development at Rightscale showed some impressive examples of how the cloud can neatly adjust to your business demand curve.
  • Peter Martin from Orange Business Services showed us some pictures of his kids who use clouds based services today (Facebook anyone?), pointing out that when they'll grow up to be CEOs, CIOs and decision makers, they're most likely not going to operate their own datacenters. Food for thought for the sceptics who think Cloud Computing is just a temporary hype or not ready (yet) for prime time: Just wait 'til your kids grow up. It may happen sooner than that, though, given the enthusiasm of the more than 100 people in the room...
  • Finally, Owen Garrett from Zeus provided a really good reason for using a software load balancer: Take back control of your application!

Here are two pencasts with audio and notes taken during the above lightning talks. The first one covers the intro until and including the Rightscale talk, the second one starts with the Orange talk and finishes with the Zeus talk.

The Unpanel

I've been to a couple of unconferences before, but this was my first unpanel. Dave Nielsen asked the attendees about who thought they were an expert on Cloud Computing. A couple of hands went up and whoosh - there you have seven experts for a panel :). Then he asked the group to provide seven questions for the panel to answer, after which each of the panelists got to answer one. For each question, the group was asked whether there was potential for some more discussion on that topic, so we also had a good basis for creating some spontaneous sessions during the conference part. Listen to the whole unpanel session on the right.

Cloud Architecture Session

After the introductory sessions and the unpanel, it was time for the breakouts. There were four of them: Cloud Security (moderated by Luc Wijns from Sun), Cloud Architecture, Open Clouds and Cloud Business Opportunities. Sébastien Pahl from DotCloud and I moderated the Cloud Architecture session. After some introductory slides, Sébastien explained his work on creting portable cloud-based services (including leverating Solaris Containers). (Sébastien, let me know when you have your slides online...). We then let the group share their questions, answeres, thoughts and discussion points. We talked about scaling MySQL in the cloud, or perhaps it would be better to leave the traditional relational model and use a simple key/value alternatives such as CouchDB. Developers asked whether they'll be able to use their IDEs with the cloud (hint: Check out NetBeans...) or whether they need to throw it all away and learn everything from scratch. How much should developers care about scalability? Isn't that something the cloud should provide? What about different APIs? Does it make sense to write your own abstraction layer? Message queues were also a popular topic and we noticed that RESTful interfaces are everywhere. I liked the final statement of one attendee most: Maybe clouds are forcing us to rethink a lot of our developer concepts so we can actually sit down and start writing clean code for a change!

Here's the audio recording from the architecture session. I tried to write down some notes after they have been discussed so you can try and skip to the pieces you're most interested in. The audio is a bit low volume, but still quite intelligible.

Wrapping It All Up

After the breakouts, a surprisingly large number of attendees were still there despite being late into the evening to gather and listen to the summaries of the different sessions. Here's the recording, including some notes to help you navigate.

All in all, this was a great event. A big thank you to Eric and his team in Paris and the sponsors for setting this up! More than ever, it became clear to me how significant the trend towards cloud computing is and how many talented people are part of this community, driving the future of IT into the sky.

Update: Eric now published his own summary with a lot of background information. It's a great read, so check it out!

Monday Jun 08, 2009

DevDusk June 2009 in Munich

Last week, DevDusk June 2009 took place at the Sun office in Munich.

What is DevDusk, you ask?

To me, it's the ultimate geek after-work party: Every once in a while, developers gather after their work day and chat about current cool technologies. Think after work mini-unconference. There are DevDusks in Frankfurt and in Munich, and this time we were lucky to sponsor the latest incarnation out of the Sun Startup Essentials program.

Every good geek event starts with food & beer!After some food and beers, we had a nice variety of talks:

  • Wolfgang Stief talked about one of the coolest hobbies: Collecting and restoring old computers. Not just C-64s and Commodore Amigas, no, the real stuff: Control Data Mainframes, Crays, etc., including a project to build a full tube-based computer. He shared a lot of funny stories involving the many obstacles in transporting, reparing and operating huge digital beasts. Check out the cray-cyber.org website and Wolfgang's collection of photos on Flickr.

  • Wolfram Kriesing from uxebu.com introduced us to EventNinja, a clever way to leverage free Cloud services (Google Docs, Yahoo! Pipes) to create fully functional, intelligent and customizable widgets, without the need to operate any server infrastructure by yourself other than a simple static web server. Very cool and a glimpse of a whole generation of clever, light-weight distributed cloud widgets. I'm working on a similar thing myself, more on that in a future blog post.

  • And yes, I got to present something too. I used the opportunity to introduce the group to my personal definition of Cloud Computing, the Sun Cloud, highlighting it's REST APIs and encouraging the audience to play around with Zembly while they're waiting for the Sun Cloud to become publicly available.

Slides, links and other material are available from the DevDusk Munich event page, feel free to check them out (some may be in Germany, but what's a little language barrier to tech people anyway?). Also, check out Gabi's blog entry on this event as well as related Twitter comments.

Wednesday Apr 15, 2009

Top 3 Cloud Computing Principles

The Sun always shines behind the clouds

As with every new topic in IT, people are wondering about the same questions: Hype or reality? Didn't we kinda have this before? What's in it for me? What's so special about it? Cloud computing is no exception and I've had the privilege to discuss this topic with a number of very bright people over the last couple of months.

To separate the wheat from the chaff, here are the top three key principles of cloud computing that struck me as making this topic very relevant, interesting and definitely the way of the future:

#1: Abstraction

For decades, IT providers have tried to standardize their operations so they can concentrate on optimizing their IT.

But this is in contrast to what IT developers and users want: They want their special version of Apache, with the newest version of PHP and "sorry, but I can't live without these 5 plugins in exactly that versions".

So much for standardization, and thus we ended up with dozens of different versions of the same services, hundreds of different services that we grew up over time with ("Of course we need the foo service, our company can't live without it! No, we can't re-implement it, that would be too expensive, you'll have to continue operating it!"). This is why compute centers today tend to look like Frankenstein's lab instead of the clean infrastructure we'd really like to have, from an architecture perspective.

Cloud computing has found a way to break out of this: A cloud gives you just a few basic, but well-defined services and that's it. Take it or leave it. "Do you like our simple, RESTful foo interface? Fine, use it!", or: "Oh, you want your own special custom version? Sorry, we don't have it. Go away." It's that simple.

This is obviously good for cloud providers, because they now can optimize the bejeezus out of their infrastructure and provide nice, massive scale, low-cost, simple to administer services, which is every IT provider's dream come true.

The new thing here is that now the developers have realized this is good for them, too (and Amazon's success is a testimony to that effect): They can now use whatever version of their software they want, on whatever OS they want and get as many updates as they want, without having to ask their IT provider.

Granted, now the burden of managing the software falls onto the developer/user, but in the end this is a win-win for both, because both sides know exactly what to expect from the other, the rules are clear, and the interface between provider and developer/user is well-defined. Of course, low service costs to the developer always helps, but we'll get to that later.

So the key point here is that well-defined abstraction layers between clouds and developers/users are the grease that lets both sides operate efficiently and completely independent of each other.

#1.1: Layers of Abstraction in Clouds

There are three layers of abstraction in clouds:

  • Application as a Service (AaaS): This is what the end-user gets when they use a service like GMail, DropBox (please make an OpenSolaris version, thanks), the myriads of Facebook apps, SmugMug or even Adobe's online photoshop web service. AaaS services are very popular and there's really no reason to start a new application any other way today.
  • Platform as a Service (PaaS): The abstraction layer here is some kind of developer environment, but the details of implementation (OS, Hardware, etc.) are completely hidden. You just get a programming language and some APIs/Libraries and off you go. This is what Zembly gives you (check it out and create your own Facebook app in minutes), or the Google App Engine. This is the development model of the future: Develop against the cloud, no need to know the details behind it.
  • Infrastructure as a Service (IaaS): These are the Amazon S3s, EC2s, etc. and we recently introduced our own version of IaaS as the Sun Cloud (featuring open interfaces and a lot of Sun technology goodness under the hood.) In this model, you get access to a virtual server or virtual storage, treat them like real machines, but the physical details of what machine is in what rack or which disks you use are hidden from you.

Most discussions around clouds center around IaaS, but remember that the basic principle of abstraction applies to the other two as well. Also, many AaaS offerings are either implemented on top of a PaaS or IaaS offerings on someone else's cloud, so we already see a whole ecosystem of cloud components working together in a pyramid like fashion, building on top of each other.

The nice thing about the Sun Cloud here is that it'll open up the abstraction layer. Just like programming environments, file server or web protocols, there's a lot of value in open standards and interfaces. That's what Sun's cloud offering is about, so our "open grease" between cloud providers and developers will enable freedom of choice, better interoperability and bigger, open cloud market for all. Check out "RESTing on the Cloud with Open APIs" for a discussion on the Sun Cloud APIs.

#2: Automation

Virtual Datacenter

Again, this may seem like nothing new, because IT operators have tried to automate as much as possible within their datacenters forever. From our own history of Sun MC through N1 and now xVM Ops Center to other people's Tivoli's, OpenView and whathave you, we've seen a lot in data center automation, but none of these went the whole way of providing true one-click setup or tear-down of a complete server over the public internet.

Automation in the cloud means the developer/user is in complete, automatic control over their resources. No human interaction whatsoever, even from a developer/user perspective. Need more servers? Let the load-balancer tell the cloud how many more to provide. No need to wait for someone to unpack and cable your machine, no need to wait for your IT department to find the time to install. Everything is automatic.

Again, this is a win-win for both sides. While full automation reduces cost and complexity for the cloud provider, it puts the developer/user in control. Now you can reduce your time to market for your next rollout because you can do it yourself, fully automatic, and you don't need to call anybody, rely on someone else to set up stuff for you, or wait days until some minor hardware/software installation is completed.

The Sun Cloud brings automation to the next level: With its Virtual Datacenter Technology, you'll be able to automate a full virtual datacenter in the cloud out of standard components, not just individual machines.

#3: Elasticity

In the nineties, people bought large, expensive, scalable servers and waited for them to fill up over time as their companies grew. This was of course highly inefficient because most of the time you didn't use most of your server. After the dot-com bust, people became smarter and started scaling horizontally. That allowed you to add capacity to your datacenter in smaller chunks and on an as-needed basis. But what if you need a lot of capacity on one day (because your startup got Techcrunched), but the next day you're back to humble levels of usage, because it's the weekend or the wrong season or there's a major recession coming up? As an extreme case: What if you ran the Olympics website and the games are just over?

That's when elasticity comes in very handy: You can easily scale up your cloud usage, but you can just as easily scale it down again. One day you have 500 web servers, 50 app servers and 10 database servers, the next day you could easily go back to the old 50:5:2 ratio. And you only pay for what you use, never for what could have been.

On a technical level, elasticity is a direct outcome of automation, our #2 principle outlined above.

But the real invention here is in the business model of cloud providers: By multiplexing their resources over a large number of customers, they can level out differing capacity needs, so that they get good resource utilization on a big scale, no matter how much or little resources individual users actually use. And by giving their customers transparent access to this model, they enable them to take advantage of a fully elastic pay-per-use-no-strings-attached model that makes a cloud service so attractive.

And this is what your traditional hoster never gave you before: Whenever you wanted some service from an old-school hoster, you'd have to sign a contract that looks like a mobile phone contract with lots of fine print and whatnot. It's easy to scale up, but then you have to commit to some usage period (like 24 months) and usually it's hard or downright impossible to scale down the size of the service you got. You could easily get stuck.

Cloud computing changes everything with its "look Ma, I can scale like Google!" model: Everybody with a credit card can operate a large datacenter for whatever time they want (and have credit for), and shut it down whenever they like.

The Sun Cloud will expand the business possibilities of the cloud model: You can choose to be the cloud (and we'll help you build it), you can choose to build the cloud (for others, out of our cloud components), you can build your own cloud (we'll help you build that, too) or you can just use it (the Sun Cloud). Just like we believe in open standards, we also believe in partnering, so no matter what your cloud business model is, Sun can help.


A lot of people discuss a lot of aspects of clouds these days, but to me, it's just the three principles above that really count.

You can use them as a litmus test for clouds: Where's the abstraction layer? Is it open? Is it fully automated? Where's the API? What if I scale down, not up? What's the cost model? If one of the above principles are missing, it's probably not a cloud. If they are there, it's most probably a cloud.

Or you can use these three principles to figure out if your internal IT operations are ready for the cloud: Can you implement your service by exclusively using a cloud API? Would you be able to encapsulate your current service inside a virtual machine, then redeploy elsewhere? How about using a PaaS model for developing your next app? Do you really want to afford your own IT infrastructure if you can just rent it like a taxi? What services would need to be re-implemented, and why? These are all good questions to ask when discussing clouds with colleagues and vendors.

But remember that cloud computing is not going to end hunger, bring world peace and cure cancer, all at once and today: Some services fit the cloud model very well (hint: Everything that looks like a web service also looks like a good candidate), some don't (If it's still on a mainframe, forget it). The answer is almost always a mixture, and it will become more interesting as public and private clouds start to interoperate, much like intranets and the internet interoperate today.

Useful Cloud Resources

There's a lot to learn about clouds and a lot of bright people are blogging about it. Here are a few points to start from:

  • Play a bit on Zembly.com. This is a great IDE for web apps, offered as a PaaS in the cloud. Translation: Log in and create your own Facebook/Meebo/iPhone app in your browser in minutes, the social way.
  • Check out the Innovating@Sun blog entry on "RESTing on the Cloud with Open APIs" to learn about what Hal Stern and Tim Bray have to say on the Sun Coud's RESTful APIs. Also, the Virtual Datacenter Demo is very impressive, and there are a number of other interesting videos on that site.
  • Glenn Brunette has a lot to say about security and clouds, certainly a hot topic. His Immutable Service Containers are the way to go for securely deploying web services in large infrastructures, including clouds.
  • If you understand German, there are some excellent German blogs to read. Check out Ralf Zenses' Blog, or Jan Brosowski's (who offers a slightly different definition of cloud principles) or the Serverwolken blog.
  • If you prefer to read English, don't despair. Check out the Sun Cloud Blog, or Alka Gupta's Blog with many interesting articles. Marc Hamilton also let's you look at cool hardware building blocks for the cloud as well as HPC clouds.
  • There's a great whitepaper from Berkeley University called Above the Clouds, a real must-read. Also, there's a great Cloud Computing Guide on the Sun Cloud page, well worth the small hassle to register for it.
  • Finally, if you think you've read enough, then relax by watching a cool video from our partner rPath: Cloud Computing in Plain English.

What are your cloud principles? What aspects of cloud computing are important to you? What important cloud aspect am I missing that would warrant its own principle? Feel free to add your own comment on cloud computing to this post!

Thursday Mar 26, 2009

Cloud Computing in 6 Minutes

Yesterday I visited Sun's European Education & Research Conference in Berlin where my colleague Manuel and I ran a session on Web 2.0 and Cloud Computing. Web 2.0 companies have really pioneered the use of cloud computing for their businesses, taking advantage of the low entry cost and high elasticity that clouds provide. These are really good things if you only have a few hundred or so users on one day, then all of a sudden you face hundreds of thousands of them, just because somebody featured your company on Techcrunch or some famous VC twittered about your service. So the two subjects go really well together so our session room was quite packed and we had some good discussions with attendees afterwards.

Sun Campus Ambassadors Alper Celik and Gökhan Dogan from KTH University in Sweden were busy interviewing a lot of people during the conference with their digital camera, and both Manuel and I got our few minutes of YouTube fame with them. Here's Manuel talking about Web 2.0:

And here's yours truly, trying to explain Cloud Computing in about 6 minutes:

Curious about Cloud Computing? Check out the Sun Cloud or start developing Web Services inside the Cloud from the comfort of your web browser the easy way using Zembly.

Alper and Gökhan were really busy, they published a bunch of other interviews on YouTube the very same day. Just search YouTube for "European Education and Research Conference" and you'll find more than a dozen of their interviews.

Gökhan also participated in his university's WaterWell project that used Sun SPOT technology to create a wireless sensor network that monitors water quality. Here's Gökhan explaining his project:

With a generation of students that show this kind of motivation, I'm not really worried about how to come out of this recession :).


Tune in and find out useful stuff about Sun Solaris, CPU and System Technology, Web 2.0 - and have a little fun, too!


« July 2016