Friday Jul 31, 2015

Secure Remote RESTful Administration with RAD

I've written before about the work we've been doing to provide a set of programmatic interfaces to Oracle Solaris using RAD. This allows developer and administrators to administer systems remotely over C, Java, Python and REST based interfaces. For anyone wanting to get their hands dirty, I've written a useful article: Getting Started with the Remote Administration Daemon on Oracle Solaris 11.

One of the areas I didn't tackle in this initial article was providing secure REST based administration interfaces over TLS. Thanks to the help of Gary Pennington, we now have a new article: Secure Remote RESTful Administration with RAD. In this article we'll use the automatically generated self-signed certificates, but this could be easily changed to point to certificates that have been signed by a Certificate Authority.

With the various announcements that we've been making recently about Oracle joining the Open Container Initiative and bringing Docker into Oracle Solaris, we're in a great position of being able to design a platform to handle the next wave of cloud deployment and delivery - whether that's traditional enterprise applications or micro services. We see the huge advantage of streamlining IT operations and facilitating methodologies such as DevOps, and it's time to take Oracle Solaris into that next wave.

Monday Jul 13, 2015

Periodic and scheduled services with SMF

With the release of Oracle Solaris 11.3 Beta last week, we've introduced a metric ton of new features. I'm really excited by the direction Oracle Solaris has been taking ad we continue to modernise the platform, include software administrators and developers are using on other platforms, and generally ensure we're ready to support the next generation of applications and infrastructure. If you've not really been following along, I'd strongly suggest you download Oracle Solaris 11.3 and have a play.

Back in 2005, we took the brave step to move away from /etc/init.d and introduced the Service Management Facility (SMF) as the main way to manage application and system services. SMF provided us with automatic service dependencies, central logging, structured configuration management, reliable application restart in the event of hardware or software failures as part of the overall fault management architecture in Oracle Solaris, and a much, much easier way of administering services. Better still, we converted all the system services over to SMF straight away and improved startup performance as we could now graph service dependencies and identify issues. You can under estimate the significance of this work, especially if you've read the turbulent history of systemd.

That was then, and this is now. One of the exciting enhancements in Oracle Solaris 11.3 relates to SMF, the introduction of the periodic and scheduled services. In another bold move, we're hoping to knock cron off it's block. There's no doubt cron is a foundation of scheduling in UNIX and Linux environments, and will be for years to come. But with scheduled SMF services we take all the ability of cron and combine them with all the benefits of SMF.

Creating an SMF periodic service is easy, with a simple addition to your SMF manifest to describe a periodic method (or using svcbundle):

<periodic_method
    period='600'
    delay='30'
    jitter='60'
    exec='/usr/local/bin/db_check'
    timeout_seconds='0'>
    <method_context>
        <method_credential user='oracle' group='dba' />
    </method_context>
</periodic_method>
In the above snippet, we can see that we're executing /usr/local/bin/db_check every 10-11 minutes (as indicated by a jitter attribute of 60 seconds) with a maximum of 30 seconds delay after the service has been transitioned to the online state. We've also given it a method credential to run the script as the oracle user with dba group. The svc:/system/svc/periodic-restarter:default service instance will be responsible for restarting this service periodically.

Scheduled services are services that are run at a specific time, perhaps at an off-peak time. Similarly these are easy to create with a simple addition to your SMF manifest (or again by using svcbundle):

<scheduled_method
    interval='day'
    hour='2'
    minute='0'
    exec='/usr/local/bin/db_backup'
    timeout_seconds='0'>
    <method_context>
        <method_credential user='oracle' group='db' />
    </method_context>
</scheduled_method>
In the above snippet, we can see that we're executing /usr/local/bin/db_backup every day at 2am (as indicated by the hour and minute attributes). In this case the frequency is set as a default value of 1, meaning that we will run this every day. Like the previous example, we have given it a method credential to run the script as the oracle user with dba group. The svc:/system/svc/periodic-restarter:default service instance is also responsible for ensuring this services runs to its defined schedule.

One of the outstanding gaps with the Image Packaging System (IPS) was the ability to associate cron jobs during package install time by locating . Some other platforms have solved this with the introduction of /etc/cron.d using a process of self-assembly of the system's cron entries. We don't support this ability with the cron version included in Oracle Solaris 11. But now using periodic or scheduled services, administrators can simply install their SMF manifests into /lib/svc/manifest/site and restart the svc:/system/manifest-import:default service instance. You can achieve this with an IPS manifest fragment that uses an IPS actuator similar to the following:

file lib/svc/manifest/site/db-backup.xml \
    path=lib/svc/manifest/site/db-backup.xml owner=root group=sys \
    mode=0444 restart_fmri=svc:/system/manifest-import:default

So take the plunge and move your cron entries over to SMF today - you'll not regret it! Our plan is to convert the existing system cron entries over in future releases. For more information, see the following chapters in the excellent Oracle Solaris 11.3 Product Docs:

Wednesday Jul 08, 2015

Remote Administration with RAD and Oracle Solaris 11

As organisations look for increased agility in their IT operations, many are turning towards more cloud like environments with shared compute, network and storage, and the ability for self-service users to quickly provision new virtualised environments on demand. With this increased virtualization sprawl, it's imperative to have a set of tools to allow administrators to effectively manage these environments, ensure they remain highly available, secure and observable.

There's hundreds of tools that have been created to help administrators manage their environments more effectively. Many tools such as Puppet and Chef, have inspired administrators to shift legacy enterprise management models over towards more rapid, agile and 'dev ops' like models. In Oracle Solaris 11, we've worked hard to modernise the operating system to adapt to this change and transform it into a highly capable cloud platform. We've included tools like Puppet as a response to customer demand, but we've also created our own - in this case RAD.

RAD (or Remote Administration Daemon) provides a set of programmatic interfaces to allow administrators to manage Oracle Solaris 11 subsystems using Python, C, Java, and RESTful APIs. RAD is also intended for developers as a complete development framework for creating their own custom interfaces to manage systems. I've written a getting started article that covers the basics of RAD, including some examples of using a few of the Oracle Solaris RAD modules. RAD is a very strategic technology for us because it provides a standardised set of interfaces to allow Oracle and other 3rd parties to write their own management interfaces on top of RAD. In fact we've already used RAD extensively in our port of OpenStack to Oracle Solaris.

Getting Started with the Remote Administration Daemon on Oracle Solaris 11.

Oracle Solaris 11.3 Beta Now Available!

We've done it again! Oracle Solaris 11.3 beta has been released today! The beta program is a great opportunity to download the latest release, try it out, and give us some feedback.

We've crammed in hundreds of new features into this release including some of my favourites: an updated OpenStack distribution (Juno), live migration support for Oracle Solaris Kernel Zones and hosting them over NFS using shared storage, bigger compression ratios with LZ4 support in the ZFS file system, PVLAN support, REST APIs and additional RAD modules (see here), Hiera to allow easy variable substitution in your Puppet manifests, faster Oracle Database 12c startups and SGA resize with Optimised Shared Memory, and everything that goes into supporting Oracle's next generation systems based on the SPARC M7 processor including Application Data Integrity (ADI) that helps prevent illegal memory access during a malicious attack.

There's a lot more, so I'd encourage you to check out the Oracle Solaris 11.3 Beta What's New and see for yourself.

Wednesday Apr 29, 2015

Managing Oracle Solaris systems with Puppet

This morning I gave a presentation to the IOUG (Independent Oracle Users Group) about how to manage Oracle Solaris systems using Puppet. Puppet was integrated with Oracle Solaris 11.2, with support for a number of new resources types thanks to Drew Fisher. The presentation covered the challenges in today's data center, some basic information about Puppet, and the work we've done to integrate it as part of the platform. Enjoy!

Wednesday Feb 25, 2015

New Solaris articles on Oracle Technology Network

I haven't had much time to do a bunch of writing for OTN, but here's a few articles that have been published over the last few weeks that I've had a hand in. The first is a set of hands on labs that we organised for last year's Oracle Open World. We walked participants through how to create a complete OpenStack environment on top of Oracle Solaris 11.2 and a SPARC T5 based system with attached ZFS Storage Appliance. Once created, we got them to create a golden image environment with the Oracle DB to upload to the Glance image repository for fast provisioning out to VMs hosted on Nova nodes.

The second article I teamed up with Ginny Henningsen to write. We decided to write an easy installation guide for Oracle Database 12c running on Oracle Solaris 11, covering some of the tips and tricks, along with some ideas for what additional things you could do. This is a great complement to the existing white paper, which I consider an absolute must read for anyone deploying the Oracle Database on Oracle Solaris.

Enjoy!

Friday Aug 15, 2014

Mirroring IPS repositories

Out of the many changes introduced in packaging with the Oracle Solaris 11.2 release, one of really good ones was the introduction of a repository mirroring service. This provides administrators with an easy, automated way of mirroring repository contents. For example, let's say you had a package repository set up locally that was serving the clients in your data center. While we provide a few different ways to sync up the contents of this repository with the Oracle Solaris 11 support repository hosted by Oracle through the pkgrecv utility or incremental ISO images, it's a pretty manual process. Now it's a case of simply configuring and starting an SMF service, svc:/application/pkg/mirror:default.

I've written a short article on this new IPS feature - How to Set Up a Repository Mirroring Service with the Oracle Solaris 11 Image Packaging Service.

You'll also notice that I also include a sneaky mention of pkg exact-install, another new feature that allows administrators to essentially reset a system to a known software boundary. Bart Smaalders has already covered this in a great blog post.

Friday Aug 01, 2014

Oracle Solaris 11.2 Available

We got there in the end, and today, Oracle Solaris 11.2 has been officially made available.

This is the most significant release I've had the pleasure to work on, and we've made huge strides in terms of overall usability, performance, and functionality. There are some really incredible base technologies included in this release - everything from independent kernel versions and patching with Oracle Solaris Kernel Zones, fast and portable clone and disaster recovery images with Unified Archives, simple to use compliance framework built on OpenSCAP, and open cloud infrastructure with OpenStack. With 21 months of hard work, Oracle Solaris 11.2 represents a huge milestone as it shifts from an enterprise-grade operating system, to a comprehensive cloud platform.

Of course we're still ensuring we're the best platform for enterprise applications and ensuring an engineered solution for Oracle - those go without saying.

Download Oracle Solaris 11.2 today!

Secure, compliant application deployment with Oracle Solaris 11

One of the really exciting features that was introduced in Oracle Solaris 11.2 is called Unified Archives. Unified Archives provide system cloning and disaster recovery capabilities for the platform. Built on the foundations of Oracle Solaris ZFS, an archive can quickly be taken on a live running system thanks to snapshot and cloning. A single archive can be created for a complete system that includes a number of virtual environments. Once captured, it can be deployed using Automated Installer or using the existing zonecfg(1M) and zoneadm(1M) utilities during Oracle Solaris Zone creation. Thanks to integration with the IPS packaging system, an archive can be partially deployed with complete flexibility - across different systems of the same architecture, or using physical-to-virtual or virtual-to-physical transforms. They're completely flexible. Jesse Butler, the architect for Unified Archives, has already covered a lot of the basics in two blog posts: Introducing Unified Archives in Oracle Solaris 11.2 and Cloning Zones with Unified Archives.

Unified Archives are a pretty critical piece of the overall application lifecycle. Combined with Oracle Solaris Zones, Immutable Zones (read-only VMs), and our new compliance framework, we have a very nice set of technologies that can be combined to really aid developers and administrators in creating and deploying compliant application environments, from development through to test and eventually production. I've written an article that helps explain how you can achieve this, and greatly cut down the cost of ensuring certified and compliant applications and reducing the cost of human error or security exploits.

Take a look at How to Ensure Secure, Compliant Application Deployment with Oracle Solaris 11.

Monday Jul 21, 2014

Understanding IPS versioning

During the lead up to Oracle Solaris 11.2 GA, I noticed that I had written an article back last year that never got published about understanding IPS package versioning. If you haven't yet had a chance to look at Oracle Solaris 11, one of the really great changes that we introduced was completely replacing the packaging mechanism from the rather legacy SVR4 packaging system to the network based Image Packaging System. IPS relies on the fact that ZFS is the underlying file system using a feature called ZFS Boot Environments, allowing us to take advantage of snapshots and clones while updating systems. This means that administrators can perform a system update while still having the old environment to fallback to if something goes wrong. There was a similar concept in Oracle Solaris 10, but was quite primitive by comparison.

And so to the document in question. IPS uses a pretty comprehensive versioning system to allow it to calculate how to go about performing a system update, or indeed any individual software package. We use a series of package constraints on the system to ensure that administrators are updating their software to a well known, and tested state. By contrast, Oracle Solaris 10 essentially let you update or apply any patches you wanted, often leading our customers down a very un-tested path. It's useful to understand this versioning system at times so I've written a useful article that covers some of this.

Take a read of Understanding Oracle Solaris 11 Package Versioning.

Thursday Jun 12, 2014

Interactive manifest editing with the Automated Installer Manifest Wizard

Oracle Solaris 11.2 adds a new Automated Installer (AI) Manifest Wizard to allow administrators to more easily create AI manifests for use in provisioning new client systems in the data center. The AI Manifest Wizard is a web web based interface that steps administrators through the basics of the AI manifest - target disks and layout selection, additional ZFS pools and datasets, IPS publisher and package selection, and the creation of any Oracle Solaris Zone virtual environments. The end result is an AI manifest without having to directly edit XML, and this can then be associated with an appropriate AI service.

To get started, check out How To Create an Automated Installer Manifest with an Interactive Wizard

Wednesday Jun 04, 2014

Getting Started with Puppet on Oracle Solaris 11

One of the exciting enhancements with Oracle Solaris 11.2 has been the introduction of Puppet. While upstream Puppet did have some rudimentary support for Oracle Solaris 11, Drew Fisher and Ginnie Wray worked tirelessly to add enhance the Oracle Solaris Puppet offering. We've talked to customers over the past few years and asked them what their problems were and what technologies they were using, particularly for configuration management. Puppet came up time and time again, and it made a huge amount of sense bringing it as a 1st class citizen in the Oracle Solaris platform.

So what is Puppet, and why is it useful? To quote from PuppetLabs, the guys who are responsible for creating Puppet:

Puppet is a declarative, model-based approach to IT automation, helping you manage infrastructure throughout its lifecycle, from provisioning and configuration to orchestration and reporting. Using Puppet, you can easily automate repetitive tasks, quickly deploy critical applications, and proactively manage change, scaling from 10s of servers to 1000s, on-premise or in the cloud.

What's more, with Puppet support for Oracle Solaris, administrators can now manage a completely heterogeneous data center from a single or series of Puppet masters. Better still, it's an excellent tool when combined with our new compliance framework to ensure you're meeting your compliance regulations. We're not stopping there of course, and we'll enhance our offerings over time, and work with PuppetLabs to get some of this support upstream (or into the Puppet Forge). So if you've heard some of the buzz around Puppet and never quite got started, and have some Oracle Solaris real estate that you'd love to manage, check out the Getting Started with Puppet on Oracle Solaris 11 guide.

Monday May 26, 2014

Oracle Solaris at the OpenStack Summit in Atlanta

I had the fortune of attending my 2nd OpenStack summit in Atlanta a few weeks ago and it turned out to be a really excellent event. Oracle had many folks there this time around across a variety of different engineering teams - Oracle Solaris, Oracle ZFSSA, Oracle Linux, Oracle VM and more. Really great to see continuing momentum behind the project and we're very happy to be involved.

Here's a list of the highlights that I had during the summit:

  • The operators track was a really excellent addition, with a chance for users/administrators to voice their opinions based on experiences. Really good to hear how OpenStack is making businesses more agile, but also equally good to hear about some of the continuing frustrations they have (fortunately many of them are new and being addressed). Seeing this discussion morph into a "Win the enterprise" working group is also very pleasing.
  • Enjoyed Troy Toman's keynote (Rackspace) about designing a planet scale cloud OS and the interoperability challenges ahead of us. I've been following some of the discussion around DefCore for a bit and while I have some concerns, I think it's mostly heading in the right direction. Certainly seems like there's a balance to strike to ensure that this effects the OpenStack vendors in such a way as to avoid negatively impacting our end users.
  • Also enjoyed Toby Ford's keynote (AT&T) about his desire for a NVF (Network Function Virtualization) architecture. What really resonated was also his desire for OpenStack to start addressing the typical enterprise workload, being less like cattle and more like pets.
  • The design summit was, as per usual, pretty intense for - definitely would get more value from these if I knew the code base a little better. Nevertheless, attended some really great sessions and got a better feeling of the roadmap for Juno.
  • Markus Flierl gave a great presentation (see below) at the demo theatre for what we're doing with OpenStack on Oracle Solaris (and more widely at Oracle across different products). Based on the discussions that we had at the Oracle booth, there's a huge amount of interest there and we talked to some great customers during the week about their thoughts and directions in this respect.
  • Undoubtedly Atlanta had some really good food. Highlights were the smoked ribs and brisket and the SweetWater brewing company. That said, I also loved the fried chicken, fried green tomatoes and collared greens, and wonderful hosting of "big momma" at Pitty Pat's Porch. Couldn't quite bring myself to eat biscuits and gravy in the morning though.
  • Visiting the World of Coca-Cola just before flying out. A total brain washing exercise, but very enjoyable. And very much liked Beverly (contrary to many other opinions on the internet) - but then again, I'd happily drink tonic water every day of the year...

Looking forward to Paris in November!

Tuesday May 06, 2014

Improved SMF Docs in Oracle Solaris 11.2

While there has been a ton of blogs posted about some of the new features of Oracle Solaris 11.2, one of those blogs in particular pleases me more than many others is the improved SMF documentation. While documentation doesn't exactly sound particularly exciting, it's absolutely crucial given the increasing importance that SMF is playing as a foundation for Oracle Solaris. Alta Estad has done a phenomenal job at improving the SMF documentation and accurately represents the hard work the SMF engineering team have been doing here.

One of the really exciting enhancements in SMF is actually a feature, unbeknownst to many, that has been available in the ZFS storage appliance for some time - SMF Stencils. Stencils are a way of taking advantage of the SMF configuration repository without having to rewrite your application to use libscf. Essentially by using a stencil you can manage your application configuration within SMF and have it automatically mirrored out to the traditional configuration file using svcio. This gives us a much improved way of managing configuration in a more structured sense, and ultimately a much better upgrade and auditing experience. In fact we've taken advantage of SMF stencils while integrating the Puppet into Oracle Solaris 11.2 (as detailed here).

So without further ado, check out Introduction to the Service Management Facility.

Wednesday Apr 30, 2014

Oracle Solaris 11.2 Announced

This morning in New York City, we announced Oracle Solaris 11.2 with Mark Hurd, John Fowler and Markus Flierl all taking the stage. I think many will probably completely fail to appreciate the amount of work that's gone into this release and some of the ground breaking technologies that have been included. They've been trying to bury us for years with the 'Solaris is dead' moniker. But here we still are, continuously engineering and steadily delivering - much like our SPARC processor and system roadmap.

If you had to sum up this release, I think the tagline for the release is pretty true to form. Engineered for cloud. It's the one driving force that's always been there since the start of Solaris development - our engineers always deliver a truly engineered system that's been architected from really solid foundations. Oracle Solaris 11 brings a number of new features that I think will be really game changing:

  • Unified Archives - Unified Archives are a replacement for Solaris Flash Archives that were available in Oracle Solaris 10. It's essentially a new archive format that's built on the OVF compliant format, but taking advantage of ZFS streams. Unified Archives provide you with the ability to capture environments of a running system either for cloning within a cloud environment or for backup and disaster recovery purposes. Unlike your standard golden image that you've been used to, Unified Archives provide an ability to capture both bare-metal and virtualized environments (or a combination of both) and deploy to either bare metal or virtual with powerful image transforms. This is especially useful in a typical development, test and product lifecycle where you want to be able to capture and quickly deploy a golden image as you go through that process. But how many developers have access to a SPARC M6 system? With Unified Archives you don't need to, and developers can capture a SPARC T5 system but deploy to a larger system when it really matters.
  • Kernel Zones - Kernel Zones are a feature of Oracle Solaris Zones to allow you to have a completely independent and isolated virtual environment with a standalone kernel. Our customers have been asking for this for a while, and with the rise of highly tenanted cloud environments, the ability to run different kernel versions side by side is crucially important. This allows administrators to easily migrate workloads for system maintenance or re-balance them completely unconstrained by the version of the global zone.
  • Compliance - We've been hearing this from our customer base for a while. Compliance requirements aren't going away anytime soon, and in fact they're getting harder and harder with multiple different software versions to manage in a virtualized, cloud computing world. With Oracle Solaris 11.1 we integrated support for the industry standard SCAP. Now we're taking this a step further by providing a set of tools to automatically monitor and report compliance with a number of built-in policies to help our customers through that pain.
  • Software Defined Networking - Oracle Solaris 11 has included a number of network virtualization capabilities for quite some time. The addition of virtual network interfaces (VNICs) has led to an unprecedented level of flexibility in network topology construction, and resource management them seamlessly within software. With Oracle Solaris 11.2 we're continuing to expand, and deliver, on Software Defined Networking (SDN) with the introduction of Elastic Virtual Switch. Elastic Virtual Switch provides administrators with the ability to seamlessly manage a series of virtual switches across multiple nodes in cloud from a single source. A new socket level API provides developers the ability to write purpose built application level flows to resource manage network traffic from the VM directly through to the network hardware and backend storage.
  • OpenStack - And finally, but not least, we've adopted the industry standard cloud computing infrastructure software, OpenStack. Oracle Solaris 11.2 is a full OpenStack distribution, and we've integrated the foundations of Oracle Solaris into the various OpenStack services - Nova compute virtualization, Neutron networking, Cinder and Swift block and object level storage, Glance image management, Keystone identity management, and of course that unified management interface Horizon that provides our customers with the ability to manage a heterogenous set of data center resources. We've been hearing from more and more customers interested in deploying OpenStack in their enterprise and we're very excited at being interoperable in this space and providing infrastructure such as a OpenStack Unified Archive to really help our customers get started in shifting their business to cloud computing.

But this list is just the start. Take a read of the Oracle Solaris 11.2 What's New Guide and see the vast array of other new integrations including extended Oracle Solaris support for Puppet, read-only root environments Immutable Global Zones, 3rd party configuration management with SMF stencils, seamless IPS repository mirroring, and secure end-to-end deployment using Automated Installer.

If you're one of the people who has dismissed Oracle Solaris, do take a look at Oracle Solaris 11.2. We've come a long, long way. These are enterprise ready technologies that you won't have seen before, and we really can do a lot better to manage your business IT needs.

About

To learn more about Oracle Solaris 11, check out an extensive list of resources including technical articles, cheat sheets and screencasts on Oracle Technology Network

Search

Categories
Archives
« August 2015
SunMonTueWedThuFriSat
      
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
     
Today