Leveraging a disaster recovery site for development - Part 2

In our previous post, we introduced the idea of using Disaster Recovery (DR) sites as a private cloud for hosting virtual development and testing environments. The solution we developped for an ISV partner of ours in the Healthcare sector looks like this.


The solution is based on the Zones and ZFS features of Oracle Solaris --available from Solaris 10 and up. Solaris Zones (a.k.a. Containers) are an operating system level virtualization technology that provides a complete, light-weight and isolated run-time environment for applications. ZFS is a new kind of filesystem that provides simple administration, transactional semantics and immense scalability; it also embeds advanced fonctionnalities in compression, encryption and snapshot, typically expected from proprietary high-end storage systems.

A ZFS snapshot is a consistent point-in-time image of a filesystem. A clone is a writable copy of a snapshot. Solaris creates ZFS clones quickly using no additional disk space to start with, because data is not duplicated on disk unless/until the cloned image changes that data. Combined with Solaris Zones, ZFS Clone allows to provision almost instantly a Virtual Image (VM) of Solaris. This is what is at work here.

So, how does the overall solution work exactly? Data is replicated between the production system and the DR site. The replicated data is stored on the high-performance fiber-channel drives, but is also duplicated, once, on low-cost high-capacity SATA drives, managed by the same array head. A Solaris Zone --we call it the “golden image”-- is created with the software stack needed by the developers fully pre-installed and configured: Web server, application server, database instance, compilers, version controlling system, you name it. This one-time installation and configuration work will be used as a template for all development environments to come, thanks to the cloning mechanism.

For each new development VM, the golden image is simply cloned with a copy of the production data. When logging into the VM, the developer views it as a fully-independent Solaris server, with its unique hostname and IP address, its own database server running and fed with fresh production data that can be read/updated/deleted with no restriction --remember, it is only a clone. A major point is that, once a template is set --many different templates can be created--, cloning and booting a new virtual server is a matter of seconds. As for the storage footprint, it will start from being almost null, since a clone will only store changes that occurs during the lifetime of the server.

Finally, when an environment is no longer needed, deleting it completely without leaving any trace is again a matter of seconds. For a developer, this is a huge benefit; you can try something completely crazy, no worrying about ruining your environment and data. Create a server, test it, break it, throw it away and create a fresh new server, all day, all you want!

To summarize, the combination of the Solaris Zones and ZFS technologies fully addresses the requirements of our specs, and more :

  • Solaris Zones assure a total isolation between the development environments and the replicated production data, and between each development environment. To play it safer, the addition of low-cost SATA drives to store the test data assures a hardware isolation from the production replicated data.
  • The creation/deletion of a development environment is extremely fast, can be scripted, and can be based on different templates with different software stacks and/or configurations. Once a Solaris zone is created and configured, it can be used as a template and cloned to create new zones. Different templates can coexist.
  • Fresh data is constantly available to each development environment, with minimum storage footprint. With ZFS Clone, we work around the (impossible) need to duplicate large production databases, which is long and storage-space consuming.
  • A simple script can shutdown all zones (other than the global zone) is a handful of seconds, giving back all available resources (CPU, memory, network) to the DR system. It gives a quick and easy way to stop all development environments if a disaster occurs on the primary production site, so the DR site can take over.
  • Solaris Zones can be detached, exported and attached to another hardware server as long as you stay on the same hardware architecture (SPARC or x64) and OS level. This gives the ability to save/export a development environment to another server.

Two last points are worth mentioning:

  • In this specific case, all environments were set to use the same Oracle 11gR2 database binaries, installed in the global zone and shared by all local zones. However the isolation offered by a zone allows the installation of various versions of a same software in different zones. We can therefore imagine many other use cases; e.g. an ISV can keep a copy of each of its customer's environment in a zone for support reasons, or perform Q&A on new versions of third-party software.
  • Functional tests may pass successfully, without giving indications on how a new or updated functionality will perform once in production. One last benefit from using the DR site, which has often the same horsepower than the production site, is to do performance tests of a new or updated algorithm or SQL query. However, if the given test is susceptible to change the data, a snapshot should be taken before, to be able to rollback to the initial state, and obviously the replication should be stopped before and restarted after the test.

Now, what could Solaris Zones and ZFS do for you? If you're a Solaris ISV, let's talk!


Post a Comment:
  • HTML Syntax: NOT allowed

How open innovation and technology adoption translates to business value, with stories from our developer support work at Oracle's ISV Engineering.



« July 2016