Friday Mar 15, 2013

New OVM Server for SPARC WP

I've been working mainly on Engineered Systems, more specifically SPARC SuperCluster for the past year, and it occurred to me that the way that we use layered virtualisation within the SSC is not very heavily publicised, and although you could derive these configurations from the OVM Server for SPARC documentation, it wasn't clear that this was considered a valid way of configuring a T-Series based server.

One of the ways that the SSC achieves such high levels of performance and efficiency, is that all the domains are built as root domains, which means that there is zero virtualisation overhead within these domains. Implementing Root Domains with Oracle VM Server for SPARC co-written by me, Mikel Manitius and Jeff Savit explains how this is achieved, and how to use these techniques on your own T4 based servers, to get the same kind of benefits.

 Happy Reading!

Wednesday Jun 06, 2012

JET now available on OTN

I know some of you have been waiting patiently, so I'm pleased to announce that the JET bundle is now available for download on the Oracle Technology Network.

I've migrated most of the content from the old Sun wiki site, and got the download in a single handy location on OTN.

Download JET now

The version available is the current latest, which is 4.9.4. This version contains a number of updates, the most significant of which is the ability to specify slot locations instead of the traditional cXtYdZsN nomenclature. This is pretty useful when trying to Jumpstart multiple servers with SAS2.0 based HBAs, as they will have the WWN embedded in the cXtYdZsN name, and it's pretty difficult to guess what that will be until you've booted the server.

The JetSDS and JetZFS modules have also been updated to use the slot terminology.

Happy JETing,

Monday Oct 25, 2010

JET 4.8 Now available

 It has been a long while. We've had about 6 internal releases of JET 4.7.1, .2, .3 etc, but we've recently gone to 4.8

The main highlight is that JET 4.8 has the new sysidcfg variables required for the unattended installation of Solaris 10 09/10. It also knows about the "new" Solaris 10 Recommended patch structure.

The best way to get it is to click the link at the top of the wiki site.

JET 4.8 Changes:

As mentioned above the main JET change is the addition of a new base_config_sysidcfg variable to allow the auto-registration stuff to be configured. 

Other changes:


All the modules have been updated to be ksh93 compliant, for those of you who are using OpenSolaris/Nevada/Solaris 11 Express Jumpstart servers.

Recommended Patches

The Solaris 10 Recommended patches (since around this time last year) have had a different directory structure, a better installation script, and a "password" to ensure that you've read the "README". JET understands this new structure and uss the installer script if present. A new base_config variable: base_config_recpatch_passcode has been added to supply the password.

Assorted Bug fixes

I've fixed an issue with the hosts file with DHCP Sparc hosts, check_client for custom checks f and m scripts, nfsmapid is started in the miniroot so nfsv4 shared files have the right perms, search items are now first in resolv.conf, use of correct awk in get_matrix_byname, added clearer fdisk comments, fixed an update_template bug, stopped n-1 custom from forcing another boot, made setupdumpdevice work only in a global zone, further refined run_sshkeygen, updated pkginfo to install only in "THISZONE".

Menu.lst enhancements

There's now a bit more flexibility in dealing with the menu.lst file. JET has always had the capability of simply appending a file to the menu.lst file using base_config_menulst_append. I've added 3 more variables to allow even more control:

1: base_config_grub_append: lets you add arguments to the boot line in the menu.lst file.

2: base_config_grub_timeout: change the default timeout from 2 seconds to whatever you set here.

3: base_config_grub_singleuser: add a boot option in menu.lst to boot net single user.

Custom Files enhancements:

You can now optionally specify owner:group:mode for each files that custom_files copies. Under normal circumstances JET simply duplicates the perms of the file on the JET server. In implementing this, I discovered an interesting bug with NFS permissions in the net-booted Solaris, which meant I had to work around that by manually starting nfsmapid in the Jumpstart phase to ensure correct and consistent permisssions.


Implemented the auto_reg functionality. Probably easier to simply paste the comments in the Template here:

# From Solaris 10 Update 9 onwards, a new sysidcfg keyword exists: auto_reg
# This allows automatic Solaris registration to take place.
# auto_reg can be all, noproxy, anon, none or disable
# "all" requires ALL additional fields to be filled in
# "noproxy" requires only oracle_user and oracle_pw fields to be filled in
# "anon" requires all the http_proxy fields to be filled in
# "none" or "disable" do not need any additional fields.


Other Modules

Some minor changes/updates have been made to some of the modules JetSDS, JetZFS, JetZONES. Mainly bugfixes and error reporting enhancements, so no huge functionality changes, but you might want to scan through the comments in the templates as I have rewritten some of them to make some of the existing functionality clearer.


As usual, we maintain backwards compatibility, so your existing templates will continue to work. To make use of the new functionality, you'll need to refresh your templates. All new clients (created with make_template) will automatically have the new variables. (Have a scan through sample.template in /opt/SUNWjet/Templates to see what's changed.)

In case anyone asks. Upgrading JET is as simple as pkgrming your existing JET modules, and then pkgadding the new ones. JET won't remove any of your existing configuration, Template or Client files.


Friday Jun 18, 2010

The Cloud Holy Grail

I went to the Oracle Enterprise Architects Club yesterday which was titled Enterprise Cloud: Hype or Reality, with talks from 3 perspectives: Customer (I have a private Cloud), Partner (I build and run Public/Private Clouds for customers), and Oracle (We have a full stack to deliver PAAS).

It helped me clear up in my own mind that the Cloud is really only a cloud to the consumer. To the cloud provider, it is not cloudy at all, but a very specific set of scoped services delivered by a H/W and S/W stack. i.e. The consumer does not need to know how or what sits inside the cloud, simply that it delivers a reliable service using self-service provisioning with predictable and transparent pricing.

It is the job of the Cloud provider to build an infrastructure that supports the user expectation of a Cloud service. For some reason Cloud architectures seem to immediately become a VMWare sales guy's dream. The definition of Cloud as a set of systemic qualities by its nature has no specification for HOW that could be delivered, and there are countless ways of building a technology stack that delivers Cloud services. I think I could make a strong case for a Cloud Infrastructure being composed of a pair of clustered M9000-64s with thousands of Solaris Containers connected to an array of S7410s.  It would exhibit the same systemic qualities as any of the other cloud architectures out there.

The point is, the technology stack bit is easy, and your architectural choice is about where you want the orchestration of multi-tenancy to live. In the middleware layer with a Weblogic grid? In the DB layer with an Oracle RAC grid? In the O/S layer with Solaris Containers? Or in the Server layer with OVM? My answer? All of the above. Doing ONLY the last is hugely inefficient. Having an agile and efficient infrastructure is key to minimising your running costs.

The 2 hard parts are self-service and billing. In the IAAS and SAAS worlds, the delivered service is sufficiently constrained to make this solvable.

The self-service delivery of PAAS requires a much more rich and complex set of interfaces for the user to be able to articulate and deploy the required "service". This could be simplified by providing pre-configured models and simply allowing the customer to select the ones they want. 

The HARDEST part of delivering PAAS is the cost model. This is simple economics: The cloud provider has an investment in infrastructure that needs to be paid for over its lifetime, and has running costs that tend to be relatively stable whether customers are using the service or not. The cloud provider has to provide a commercial proposition to the customer that delivers the service cheaper than if the customer did it himself, while still being able to cover his costs and make a profit.  Obviously in the IAAS world I can simply charge per allocated resource, and in the SAAS world I can simply charge per transaction or per user. What is the PAAS model? Transaction? Resource? User? A combination of them all? Do I need to have a reservation charge as well as a usage charge? How does/can the customer compare the cloud cost against his own? 

The solution to the PAAS billing problem? Make it a Private Cloud and you can skip the billing piece. :-)

In summary, the Holy Grail of PAAS is to come up with a commercial model that allows customers to use the service at a lower cost while still allowing the Cloud provider to make a profit. When someone comes up with an answer, please let me know. 


Thursday Dec 17, 2009

Jumpstarting over multiple subnets

The question of Jumpstart and subnets is always coming up. I've finally documented the issue and the ways of "solving" it on the JET wiki site. As with any problem, there is no single easy answer, but a number of different solutions that can be combined in a number of ways.


Saturday May 16, 2009

Solaris Virtualization Epiphany

I was giving my standard pitch covering Sun Virtualisation Technologies which essentially entails me talking at length about the following slide. I kind of make the point that it is a means not an end, and that really what you're trying to do is run more applications on the same piece of hardware, and the trick is to try and do it most efficiently.

I've done this talk countless times with a pretty wide variety of customers, and I tend to start talking from the right, and move to the left, explaining how the "layer of additional non app resource utilisation" grows from right to left. (not true for Dynamic System Domains on the far left though.)

It finally occurred to me yesterday that the Unix and Windows world have an utterly different view of Virtualisation, and the need for Virtual Machines.

In the Unix world, application co-existence is second nature to most sys-admins. Virtualisation is all about application isolation. i.e. You START with all the apps on a single O/S instance, and you apply more and more constraints: resource management --> containers/zones --> virtual machines --> dynamic system domains.

In the Windows world, application co-existence is so rare that it isn't even considered. Virtualisation is all about application consolidation. (and the primary way to do it is to create lots of virtual machines, one per app).

The problem is simply viewed from completely different angles.

It's not just an application co-existence issue either. There's a scalability problem. A lot of "other" O/S's can't scale to fit the larger more powerful Intel/AMD multi-core servers available today, and the ONLY way to make use of them is to carve them up into smaller machines that Windows or Linux can digest. Solaris, on the other hand can scale up to hundreds of CPUs, and hundreds of Gigabytes of memory.

From the slide above, Sun/Solaris provides a wide choice of technologies to allow multiple applications to be consolidated onto a single server. Don't let the limitations of "other" O/S's blinker you into thinking that a Virtual Machine is the only answer. 

(I'm not saying that Virtual Machines is the WRONG answer btw, it's ONE of the answers, but it shouldn't be the only one that is considered.)


Friday May 08, 2009

JET 4.7 Now available

 So, I've been working on JET 4.7 for a while now. A couple of weeks ago I released it internally to Sun, and no-one's complained about any showstopping bugs yet, so I've released a fresh bundle to the downloads site. The best way to go straight to it is to follow the link at the top of the wiki site.

JET 4.7 Changes:

The MAJOR JET change is that I've kind of merged JetNAS functionality into the main SUNWjet module. This isn't an exact replication of the JetNAS code, and it does things a little differently, but in short you can choose to place the NFS functionality somewhere different from your JET server.

Other JET changes are covered by bugids: 6835016, 685027, 685023, 685019, 685030.

In summary:

6835016:  JET needs to support alternative servers for NFS media. The JET User Guide has already been updated to cover this functionality. Additionally, I've written a short article about it on the JET wiki site.

685027:  JET jetjump.xml script does not stop graphical login from starting. Tidied it up. graphical login no longer starts until J/S is complete.

685023: JET should not create /dhcp directory. We used to do this to shorten the dhcp macro. This is no longer necessary, so we've removed the complication of having it.

685019: JET ksh scripts do not work when using OpenSolaris. There were a couple of constructs in our ksh code that ksh93 choked on. We've updated them so that they are compatible with ksh and ksh93. This allows all you OpenSolaris people to install JET on your OpenSolaris based Jumpstart server. (Note, AI is required to "jumpstart" OpenSolaris.)

685030: JET error message when suitable server is not found could be more meaningful. Well, this is just an attempt to reduce the e-mail traffic. Probably the first error, people new to JET see. We've updated the error message to tell you what to do to fix it.

JET 4.6 - 4.7 Change Summary:

For all you "external" people, the last externally available version was 4.6. Here are the changes from 4.6 - 4.7:

4.6.1: Ability to compress the rpool, and specify /var for zfs boot builds.

4.6.2: 6798648: JET needs to workaround Bug: 6528699: This is to stop Jumpstart from labelling ALL your disks and blowing away all your data. Also added a tweak to the ssh scripts to generate the keys properly.

4.6.3: Added an additional bootadm update-archive to the end of the finish.common script. We were getting some situations where the first reboot had a stale boot-archive, possibly due to a race condition in SPARC S10 Update 6 and above builds.

JET bundled Modules:

We've updated some/most of the bundled products. ALL of them (if required) have been updated to make them ksh93 compatible. Some of them have had more substantial changes:


This module has had the most substantial rewrite. We've gone against ALL JET precedent and completely changed the structure of the ldom.conf file, so it's not backward compatible (for once). We needed to do that so that we could provide a much easier mechanism to make use of the ldm 1.3 functionality. Sorry.... but you'll like the new one better.


I've updated JetFLASH so that is actually checks the flar (if it can), to see that it is the right architecture, and the right boot filesystem for the build. (Yes, zfs based flash is coming, but it is a different format to the ufs based one). Bruce added some code to clean up service tags if required.


Quite a few changes in this module. You can now update your rpool properties, I've provided more verbose progress messages (so you can see exactly what it is doing), added code to deal with converting EFI disks back to SMI (it does it for you if required), and finally added a strange hack so that you can optionally use ::: instead of = in your variables to get around a xVM OC gui restriction.


Fixed a bug where make_template was in the wrong place, and added some code to boot the zone before trying to add stuff to the rootpath/var dir. (because it might not be there unless the zone is booted). (Thanks to Mike Hendon for pointing it out). (plus a bunch of new functionality like support for branded zones)


So just a reminder, the following modules have NEW template variables. Old templates (other than the ldom based ones) will continue to work, but if you want to make use of the new functionality (with the new comments that describe them), you'll need to refresh your templates. (Have a scan through sample.template in /opt/SUNWjet/Templates to see what's changed.)


Wednesday Apr 15, 2009

JET/Jumpstart without NFS

I've been working on JET 4.7 (current released JET version is 4.6.3).

The main change is that I've now provided support to allow the Solaris Media and the /opt/SUNWjet directories to be hosted by a server (or servers) other than the JET server. This effectively removes the requirement for the JET server to provide NFS services.

Along the way I was able to tidy up and consolidate a lot of code, particularly the client_allocation_\* methods. I've added a few functions, and used the variables more consistently throughout. For those of you doing DHCP/PXE based builds, I've removed the /dhcp hack that was used to overcome the 255 char limitiation of the DHCP macros.

Full details on the wiki site

I'm hoping to release it by the end of April.


Tuesday Jan 06, 2009

FOLLOWUP: Solaris Cluster on a laptop using VirtualBox, iSCSI and a quorum server

A few months ago I wrote  Solaris Cluster on a laptop using VirtualBox, iSCSI and a quorum server, which detailed some of the hoops that I needed to jump through to get it to work. The good news is that the new features in VirtualBox 2.0 (and now 2.1) have gotten rid of 2 of those hoops:

  1.  I no longer need to play with vnics and that crazy script as Host Interface networking simply works out of the box
  2. I don't need to install the 32bit internal only SC package, as VirtualBox now supports 64bit guests.

I'm in the process of rebuilding my laptop cluster making use of the above new features.



Monday Jan 05, 2009

JET 4.6 externally available

As a follow-on to my previous post, JET 4.6 is now available externally. Check the JET wiki which has a link to the download.

This is mainly a bug fix release, and has been given a major number increase because it's the JET version that will be in the next release of xVM OpsCenter. (and we like to release the OpsCenter releases at major version numbers).

In terms of changes, I've fixed a couple of bugs with turning ntp on, and multiple disks in zfs pools for ZFS disks. This version also has tweaks in place to workaround some DHCP issues when installing S10 U6 and Solaris Nevada.


Tuesday Nov 04, 2008

JET 4.4.7 available externally

As a follow-on to my previous post, JET 4.4.7 is now available externally. Check the JET wiki which has a link to the download.

Also have a look at the JET User Guide which has now been posted on the wiki. I expect this to grow over time, but its a conversion from the original user guide with updates for correctness.


Tuesday Oct 21, 2008

JET 4.4.7 released

I've just released JET 4.4.7 internally. Bruce is in the process of putting together a new bundle including 4.4.7 for the external download, so it'll arrive in couple of weeks. If you need it NOW just ask nicely. :-)


The latest  version currently available externally is 4.4, and we've added a bunch of features since then. Here's a brief overview of what's new/changed/fixed:


  1. 6660631: Disabling UFS logging no longer works.
  2. 6673213: Selecting dhcp should automatically select grub if required.
  3. 6675467: JET needs to workaround Nevada installation bug 6633017.
  4. 6315828: JET "shutup_sendmail" variable does not work on Solaris 10.
  5. 6526561: Typo in make_client, missing r on JS_error line 130.


  1. 6656549: Add ability to do fdisk commands in the profile.


  1. 6694299: SUNWjet fails to install on system with super long group.
  2. 6690411: base_config_profile_fdisk needs to be ingored for sparc.
  3. 6698860: "rarp" service should not be enable by JET (on dhcp builds)


  1. 6706581: JET needs to support ZFS root installs. This is a pretty big one, with the inclusion of 5 new base_config variables. Might require a separate blog article to explain, but as usual, the comments in the template should get you on the way.


  1. 6735314: JET incorrectly sets NFSMAPID_DOMAIN to "dynamic".
  2. 6735318: JET should support noatime mount options.
  3. 6740625: JET client_environment should never output a warning message.


Never released.


  1. 6675467: (again, as we need to apply workaround for Sol10 U6 as well)
  2. 6761787: JET custom_files should allow the overwritten file to be backed up. (inspiration from Christopher Hubbell)

Summary of Changes

A lot of the changes are bug fixes, but a number of them begin to make use of some of the new Solaris features, or leverage new ways of doing things. The most important of these is the ability for JET to install ZFS boot Solaris. This applies to Nevada build 90 and above, as well as the imminently to be released Solaris 10 Update 6. Other things that are kinda cool is the ability for JET to fdisk your disk for you when doing an install, as well as set noatime mountpoint options (which more and more people seem tobe doing by default now.


Wednesday Sep 24, 2008

belenix, opensuse and ubuntu on Nevada with VirtualBox

I was in the process of playing with some stuff using VirtualBox, and it occurred to me that this was a pretty cool screenshot, so I did, and here it is.


Wednesday Sep 17, 2008

Using Sun Secure Global Desktop

Over the past few weeks/months, I've been involved (sometimes peripherally) with setting up our demo kit in the UK Customer Briefing Centre. One of my design goals was to ensure seamless access to our demos from anywhere in the world or more realistically anywhere in the UK, and more specifically, at any customer site. 

Within the CBC network (which is not connected to our internal Sun network) we set up a Sunray Server also running SGD. We initially had some hiccups getting the firewall traversal to work, and had to resort to using VNC which was adequate, but painful as we were tunnelling through ssh, so there were a bunch of steps required to ssh in, start vncserver, then re-ssh in to tunnel the right port, then start vncviewer and after all of that the screen refresh rate wasn't stellar.

However, once we got SGD working it was absolutely awesome.

If you want to have a play with how it works, simply go to Sun's SDG demo site . Unfortunately MyDesktop doesn't work (for obvious reasons) for anonymous users, but it will let you play with the webtop functionality.

For our purposes, we used the MyDesktop functionality. Running a desktop session inside the CBC from anywhere in the world was as easy as pointing a browser at the right place, logging in and clicking on the Desktop icon. Within seconds, as if by magic, a full screen desktop takes over your current screen (you can switch back to local using Alt-TAB). I'd show you a picture of it, but it just looks like a normal desktop and feels remarkably snappy even if it is remote.

The reason we were doing all of this is that we  (The UK Systems Practice) have been running some "Discovery Days" around LDOMS and ZFS which have live demos of the technology. The cool part is that we get to showcase the SGD technology at the same time!


Helford River

So I got the boat down, and managed to go sailing in and around the Helford river in Cornwall. It was absolutely awesome. On the day the picture was taken, the wind was pretty light and so I decided to brave going out single handed. Even got the asymmetric out on the downwind leg.




« December 2016