Virtual Machine (VM) template technical design
By Gabor Puhalla on Jun 22, 2009
The development of a VM template is similar in essence to the development of any other SW product. You collect your business, functional and other requirements, develop, release and sustain. No black magic, pretty straightforward, isn't it?
This is the second entry in a series that's objective is to share our experience with building VM templates. Rather then describing a general approach or process, we are sharing our experience through the specific example of building the Sun Glassfish Web Space Server 10 VM Template that we announced earlier this month.
The one specific angle you need to consider is that you use the VM template as a distribution vehicle to deliver your software, not an automated installer based bundle or a zip distro containing files that you need to copy to your system. You need to understand, what virtualization platforms your customers use, what use cases drive them to use VM templates, what download size your VM template can afford, what sort of out-of-the-box experience you could/should provide, what legal/licensing considerations you need to take in account, etc. We will not attempt to describe the complete spectrum of questions around delivering software products in form of VM templates. We want to focus on technical design aspects and hence, let us assume that most of non-technical requirements necessary to start the development are given. To get a sense for what requirements we considered, when creating the Sun GlassFish Web Space Serve 10 VM Template, you could take a look here.
So, let us take a closer look at the technical design and some of the requirements that helped forming it. I will start the explanation from the bottom-up.
Hardware resources and the hypervisor
Determining VM formats and the HW resource allocation was a balancing exercise among 1) HW needs of the installed software, 2) HW possibilities of the intended audience and 3) the resulting download size.
Given we were building an evaluation oriented VM template, we wanted to make sure that users are able to open it on both server (VMware ESX) and desktop (VirtualBox, VMware Workstation/Player/Server/Fusion) virtualization platforms. We decided to consider cloud platforms at a later point in the time, because of the project's time and resources constraints. You should hear from us about it in the future though.
Most of the latest desktop machines and notebooks have 2 GB memory (well, some of them have 4GB, but it is still far from being standard), 1 CPU (potentially with couple of cores) and hundreds of GB of HD. That made us to limit the memory assignment by 1GB and CPU assignment by 1 CPU. Users are welcome to increase the HW allocation, if they have the capacity for it. It can improve the performance.
We knew that we didn't want the download size to exceed 1GB significantly. That would make downloads difficult. This implied us limiting the initial size of the HD to 4GB, which can grow dynamically up to 20GB once deployed.
We expected that most of our users will have DHPC server (through a wireless access point at home or through other means) in their network that can assign an IP to the VM template instance dynamically. That is, how we arrived to bridged networking. Setting up a static IP or NAT networking is possible too, but they may require manual user actions, which we wanted to avoid for the default scenario.
The operating system
HW resources assigned to the OS were implied by the overall VM resource allocation and resource allocation of the layered software. The 256 MB of memory isn't explicitly assigned to the OS; this is the
rough amount that it implicitly ends up with along the memory
consumption of the layered software. The goal we had was to get an optimal performance of Web Space Server 10 and other layered software within the constrains of HW resources assigned to the VM. 2 GB SWAP and the ZFS file system cache limitation up to 64 MB are explicit assignments determined by experimental work.
The other important aspect we had to consider is the download size. The default installation of OpenSolaris consumes several GB of disk space, which is several GB even in a compressed form. A reduced size image was in order to make our project successful. We decided to build an OpenSolaris JeOS (Just Enough Operating System) that includes just those OpenSolaris packages that are needed for the kind of production oriented deployment that we targeted with our VM template. You can take a look at Rudolf Kutina's blog to learn about the background of the JeOS we built and used. More on this in future blogs.
HW resources assigned were, specifically memory was determined on an experimental basis (just as in case of the OS). SW components that we included and that we needed to consider are:
GlassFish domain with Web Space Server 10 and samples deployed for evaluation use and a GlassFish Admin Console for this evaluation domain. These domains are started by default.
GlassFish domain with Web Space Server 10 (without samples) for production use and a GlassFish Admin Console for this production domain. These are turned off by default to save HW resources. They are prepared for users, who would consider moving to production and who don't need Web Space Server samples deployed. Increasing the HW resource alocation is strongly encouraged for users, who plan moving to production with the template.
MySQL to store Web Space Server 10 data.
Apache domain to support the integration of VM template internals. Specifically to host the start-up page of the template that is one of the cornerstones of the out-of-the-box experience of the template.
Webmin to enable web-based administration of OpenSolaris and MySQL. This is a functionality that people moving to production would appreciate.
The specification of the technical design as well as the overall development effort was an experimental effort that resulted to a specific constellation of HW and SW. We don't expect that you will reuse this spec for your appliances as it is. We hope, however, that seeing considerations and challenges that we went through may help you in defining your own technical design. As always, feedback about your experience or suggestions for improvement are welcome.
And, so I don't take all the credit for this work, let me tell that most of this design was built based on Chris Kampmeier's and Rudolf VirtualGuru Kutina's work. My role was in consolidating their inputs. I hope, you find it useful.
Find the next entry in the series here.