Tuesday Sep 01, 2015

Applying read-only protection with Immutable Zones

A while back, I wrote an article that focus on how you can achieve a secure and compliant application deployment from development, through test and production. This article took advantage of a number of Oracle Solaris technologies including Oracle Solaris Zones, Unified Archives, Immutable Zones, Automated Installer and the integrated compliance framework.

We've had a number of customers get really excited by Immutable Zones and being able to lock down their environments. Not only does this provide an additional layer of security, but also protects against the potential cost of human error and ensures that organisations can meet their compliance requirements routinely. Darren Moffat and Casper Dik have already written great blog entries on how to do this, but I've also recently published another How-To article on Applying read-only protection with Oracle Solaris Immutable Zones. In this article we cover immutable non-global and global zones, and show how we can make administrative changes such as applying critical security fixes using the Trusted Path. Hope you find it useful!

Friday Jul 31, 2015

Secure Remote RESTful Administration with RAD

I've written before about the work we've been doing to provide a set of programmatic interfaces to Oracle Solaris using RAD. This allows developer and administrators to administer systems remotely over C, Java, Python and REST based interfaces. For anyone wanting to get their hands dirty, I've written a useful article: Getting Started with the Remote Administration Daemon on Oracle Solaris 11.

One of the areas I didn't tackle in this initial article was providing secure REST based administration interfaces over TLS. Thanks to the help of Gary Pennington, we now have a new article: Secure Remote RESTful Administration with RAD. In this article we'll use the automatically generated self-signed certificates, but this could be easily changed to point to certificates that have been signed by a Certificate Authority.

With the various announcements that we've been making recently about Oracle joining the Open Container Initiative and bringing Docker into Oracle Solaris, we're in a great position of being able to design a platform to handle the next wave of cloud deployment and delivery - whether that's traditional enterprise applications or micro services. We see the huge advantage of streamlining IT operations and facilitating methodologies such as DevOps, and it's time to take Oracle Solaris into that next wave.

Monday Jul 13, 2015

Periodic and scheduled services with SMF

With the release of Oracle Solaris 11.3 Beta last week, we've introduced a metric ton of new features. I'm really excited by the direction Oracle Solaris has been taking ad we continue to modernise the platform, include software administrators and developers are using on other platforms, and generally ensure we're ready to support the next generation of applications and infrastructure. If you've not really been following along, I'd strongly suggest you download Oracle Solaris 11.3 and have a play.

Back in 2005, we took the brave step to move away from /etc/init.d and introduced the Service Management Facility (SMF) as the main way to manage application and system services. SMF provided us with automatic service dependencies, central logging, structured configuration management, reliable application restart in the event of hardware or software failures as part of the overall fault management architecture in Oracle Solaris, and a much, much easier way of administering services. Better still, we converted all the system services over to SMF straight away and improved startup performance as we could now graph service dependencies and identify issues. You can under estimate the significance of this work, especially if you've read the turbulent history of systemd.

That was then, and this is now. One of the exciting enhancements in Oracle Solaris 11.3 relates to SMF, the introduction of the periodic and scheduled services. In another bold move, we're hoping to knock cron off it's block. There's no doubt cron is a foundation of scheduling in UNIX and Linux environments, and will be for years to come. But with scheduled SMF services we take all the ability of cron and combine them with all the benefits of SMF.

Creating an SMF periodic service is easy, with a simple addition to your SMF manifest to describe a periodic method (or using svcbundle):

<periodic_method
    period='600'
    delay='30'
    jitter='60'
    exec='/usr/local/bin/db_check'
    timeout_seconds='0'>
    <method_context>
        <method_credential user='oracle' group='dba' />
    </method_context>
</periodic_method>
In the above snippet, we can see that we're executing /usr/local/bin/db_check every 10-11 minutes (as indicated by a jitter attribute of 60 seconds) with a maximum of 30 seconds delay after the service has been transitioned to the online state. We've also given it a method credential to run the script as the oracle user with dba group. The svc:/system/svc/periodic-restarter:default service instance will be responsible for restarting this service periodically.

Scheduled services are services that are run at a specific time, perhaps at an off-peak time. Similarly these are easy to create with a simple addition to your SMF manifest (or again by using svcbundle):

<scheduled_method
    interval='day'
    hour='2'
    minute='0'
    exec='/usr/local/bin/db_backup'
    timeout_seconds='0'>
    <method_context>
        <method_credential user='oracle' group='db' />
    </method_context>
</scheduled_method>
In the above snippet, we can see that we're executing /usr/local/bin/db_backup every day at 2am (as indicated by the hour and minute attributes). In this case the frequency is set as a default value of 1, meaning that we will run this every day. Like the previous example, we have given it a method credential to run the script as the oracle user with dba group. The svc:/system/svc/periodic-restarter:default service instance is also responsible for ensuring this services runs to its defined schedule.

One of the outstanding gaps with the Image Packaging System (IPS) was the ability to associate cron jobs during package install time by locating . Some other platforms have solved this with the introduction of /etc/cron.d using a process of self-assembly of the system's cron entries. We don't support this ability with the cron version included in Oracle Solaris 11. But now using periodic or scheduled services, administrators can simply install their SMF manifests into /lib/svc/manifest/site and restart the svc:/system/manifest-import:default service instance. You can achieve this with an IPS manifest fragment that uses an IPS actuator similar to the following:

file lib/svc/manifest/site/db-backup.xml \
    path=lib/svc/manifest/site/db-backup.xml owner=root group=sys \
    mode=0444 restart_fmri=svc:/system/manifest-import:default

So take the plunge and move your cron entries over to SMF today - you'll not regret it! Our plan is to convert the existing system cron entries over in future releases. For more information, see the following chapters in the excellent Oracle Solaris 11.3 Product Docs:

Wednesday Jul 08, 2015

Remote Administration with RAD and Oracle Solaris 11

As organisations look for increased agility in their IT operations, many are turning towards more cloud like environments with shared compute, network and storage, and the ability for self-service users to quickly provision new virtualised environments on demand. With this increased virtualization sprawl, it's imperative to have a set of tools to allow administrators to effectively manage these environments, ensure they remain highly available, secure and observable.

There's hundreds of tools that have been created to help administrators manage their environments more effectively. Many tools such as Puppet and Chef, have inspired administrators to shift legacy enterprise management models over towards more rapid, agile and 'dev ops' like models. In Oracle Solaris 11, we've worked hard to modernise the operating system to adapt to this change and transform it into a highly capable cloud platform. We've included tools like Puppet as a response to customer demand, but we've also created our own - in this case RAD.

RAD (or Remote Administration Daemon) provides a set of programmatic interfaces to allow administrators to manage Oracle Solaris 11 subsystems using Python, C, Java, and RESTful APIs. RAD is also intended for developers as a complete development framework for creating their own custom interfaces to manage systems. I've written a getting started article that covers the basics of RAD, including some examples of using a few of the Oracle Solaris RAD modules. RAD is a very strategic technology for us because it provides a standardised set of interfaces to allow Oracle and other 3rd parties to write their own management interfaces on top of RAD. In fact we've already used RAD extensively in our port of OpenStack to Oracle Solaris.

Getting Started with the Remote Administration Daemon on Oracle Solaris 11.

Oracle Solaris 11.3 Beta Now Available!

We've done it again! Oracle Solaris 11.3 beta has been released today! The beta program is a great opportunity to download the latest release, try it out, and give us some feedback.

We've crammed in hundreds of new features into this release including some of my favourites: an updated OpenStack distribution (Juno), live migration support for Oracle Solaris Kernel Zones and hosting them over NFS using shared storage, bigger compression ratios with LZ4 support in the ZFS file system, PVLAN support, REST APIs and additional RAD modules (see here), Hiera to allow easy variable substitution in your Puppet manifests, faster Oracle Database 12c startups and SGA resize with Optimised Shared Memory, and everything that goes into supporting Oracle's next generation systems based on the SPARC M7 processor including Application Data Integrity (ADI) that helps prevent illegal memory access during a malicious attack.

There's a lot more, so I'd encourage you to check out the Oracle Solaris 11.3 Beta What's New and see for yourself.

Wednesday Apr 29, 2015

Managing Oracle Solaris systems with Puppet

This morning I gave a presentation to the IOUG (Independent Oracle Users Group) about how to manage Oracle Solaris systems using Puppet. Puppet was integrated with Oracle Solaris 11.2, with support for a number of new resources types thanks to Drew Fisher. The presentation covered the challenges in today's data center, some basic information about Puppet, and the work we've done to integrate it as part of the platform. Enjoy!

Wednesday Feb 25, 2015

New Solaris articles on Oracle Technology Network

I haven't had much time to do a bunch of writing for OTN, but here's a few articles that have been published over the last few weeks that I've had a hand in. The first is a set of hands on labs that we organised for last year's Oracle Open World. We walked participants through how to create a complete OpenStack environment on top of Oracle Solaris 11.2 and a SPARC T5 based system with attached ZFS Storage Appliance. Once created, we got them to create a golden image environment with the Oracle DB to upload to the Glance image repository for fast provisioning out to VMs hosted on Nova nodes.

The second article I teamed up with Ginny Henningsen to write. We decided to write an easy installation guide for Oracle Database 12c running on Oracle Solaris 11, covering some of the tips and tricks, along with some ideas for what additional things you could do. This is a great complement to the existing white paper, which I consider an absolute must read for anyone deploying the Oracle Database on Oracle Solaris.

Enjoy!

Friday Aug 15, 2014

Mirroring IPS repositories

Out of the many changes introduced in packaging with the Oracle Solaris 11.2 release, one of really good ones was the introduction of a repository mirroring service. This provides administrators with an easy, automated way of mirroring repository contents. For example, let's say you had a package repository set up locally that was serving the clients in your data center. While we provide a few different ways to sync up the contents of this repository with the Oracle Solaris 11 support repository hosted by Oracle through the pkgrecv utility or incremental ISO images, it's a pretty manual process. Now it's a case of simply configuring and starting an SMF service, svc:/application/pkg/mirror:default.

I've written a short article on this new IPS feature - How to Set Up a Repository Mirroring Service with the Oracle Solaris 11 Image Packaging Service.

You'll also notice that I also include a sneaky mention of pkg exact-install, another new feature that allows administrators to essentially reset a system to a known software boundary. Bart Smaalders has already covered this in a great blog post.

Friday Aug 01, 2014

Oracle Solaris 11.2 Available

We got there in the end, and today, Oracle Solaris 11.2 has been officially made available.

This is the most significant release I've had the pleasure to work on, and we've made huge strides in terms of overall usability, performance, and functionality. There are some really incredible base technologies included in this release - everything from independent kernel versions and patching with Oracle Solaris Kernel Zones, fast and portable clone and disaster recovery images with Unified Archives, simple to use compliance framework built on OpenSCAP, and open cloud infrastructure with OpenStack. With 21 months of hard work, Oracle Solaris 11.2 represents a huge milestone as it shifts from an enterprise-grade operating system, to a comprehensive cloud platform.

Of course we're still ensuring we're the best platform for enterprise applications and ensuring an engineered solution for Oracle - those go without saying.

Download Oracle Solaris 11.2 today!

Secure, compliant application deployment with Oracle Solaris 11

One of the really exciting features that was introduced in Oracle Solaris 11.2 is called Unified Archives. Unified Archives provide system cloning and disaster recovery capabilities for the platform. Built on the foundations of Oracle Solaris ZFS, an archive can quickly be taken on a live running system thanks to snapshot and cloning. A single archive can be created for a complete system that includes a number of virtual environments. Once captured, it can be deployed using Automated Installer or using the existing zonecfg(1M) and zoneadm(1M) utilities during Oracle Solaris Zone creation. Thanks to integration with the IPS packaging system, an archive can be partially deployed with complete flexibility - across different systems of the same architecture, or using physical-to-virtual or virtual-to-physical transforms. They're completely flexible. Jesse Butler, the architect for Unified Archives, has already covered a lot of the basics in two blog posts: Introducing Unified Archives in Oracle Solaris 11.2 and Cloning Zones with Unified Archives.

Unified Archives are a pretty critical piece of the overall application lifecycle. Combined with Oracle Solaris Zones, Immutable Zones (read-only VMs), and our new compliance framework, we have a very nice set of technologies that can be combined to really aid developers and administrators in creating and deploying compliant application environments, from development through to test and eventually production. I've written an article that helps explain how you can achieve this, and greatly cut down the cost of ensuring certified and compliant applications and reducing the cost of human error or security exploits.

Take a look at How to Ensure Secure, Compliant Application Deployment with Oracle Solaris 11.

Monday Jul 21, 2014

Understanding IPS versioning

During the lead up to Oracle Solaris 11.2 GA, I noticed that I had written an article back last year that never got published about understanding IPS package versioning. If you haven't yet had a chance to look at Oracle Solaris 11, one of the really great changes that we introduced was completely replacing the packaging mechanism from the rather legacy SVR4 packaging system to the network based Image Packaging System. IPS relies on the fact that ZFS is the underlying file system using a feature called ZFS Boot Environments, allowing us to take advantage of snapshots and clones while updating systems. This means that administrators can perform a system update while still having the old environment to fallback to if something goes wrong. There was a similar concept in Oracle Solaris 10, but was quite primitive by comparison.

And so to the document in question. IPS uses a pretty comprehensive versioning system to allow it to calculate how to go about performing a system update, or indeed any individual software package. We use a series of package constraints on the system to ensure that administrators are updating their software to a well known, and tested state. By contrast, Oracle Solaris 10 essentially let you update or apply any patches you wanted, often leading our customers down a very un-tested path. It's useful to understand this versioning system at times so I've written a useful article that covers some of this.

Take a read of Understanding Oracle Solaris 11 Package Versioning.

Thursday Jun 12, 2014

Interactive manifest editing with the Automated Installer Manifest Wizard

Oracle Solaris 11.2 adds a new Automated Installer (AI) Manifest Wizard to allow administrators to more easily create AI manifests for use in provisioning new client systems in the data center. The AI Manifest Wizard is a web web based interface that steps administrators through the basics of the AI manifest - target disks and layout selection, additional ZFS pools and datasets, IPS publisher and package selection, and the creation of any Oracle Solaris Zone virtual environments. The end result is an AI manifest without having to directly edit XML, and this can then be associated with an appropriate AI service.

To get started, check out How To Create an Automated Installer Manifest with an Interactive Wizard

Wednesday Jun 04, 2014

Getting Started with Puppet on Oracle Solaris 11

One of the exciting enhancements with Oracle Solaris 11.2 has been the introduction of Puppet. While upstream Puppet did have some rudimentary support for Oracle Solaris 11, Drew Fisher and Ginnie Wray worked tirelessly to add enhance the Oracle Solaris Puppet offering. We've talked to customers over the past few years and asked them what their problems were and what technologies they were using, particularly for configuration management. Puppet came up time and time again, and it made a huge amount of sense bringing it as a 1st class citizen in the Oracle Solaris platform.

So what is Puppet, and why is it useful? To quote from PuppetLabs, the guys who are responsible for creating Puppet:

Puppet is a declarative, model-based approach to IT automation, helping you manage infrastructure throughout its lifecycle, from provisioning and configuration to orchestration and reporting. Using Puppet, you can easily automate repetitive tasks, quickly deploy critical applications, and proactively manage change, scaling from 10s of servers to 1000s, on-premise or in the cloud.

What's more, with Puppet support for Oracle Solaris, administrators can now manage a completely heterogeneous data center from a single or series of Puppet masters. Better still, it's an excellent tool when combined with our new compliance framework to ensure you're meeting your compliance regulations. We're not stopping there of course, and we'll enhance our offerings over time, and work with PuppetLabs to get some of this support upstream (or into the Puppet Forge). So if you've heard some of the buzz around Puppet and never quite got started, and have some Oracle Solaris real estate that you'd love to manage, check out the Getting Started with Puppet on Oracle Solaris 11 guide.

Monday May 26, 2014

Oracle Solaris at the OpenStack Summit in Atlanta

I had the fortune of attending my 2nd OpenStack summit in Atlanta a few weeks ago and it turned out to be a really excellent event. Oracle had many folks there this time around across a variety of different engineering teams - Oracle Solaris, Oracle ZFSSA, Oracle Linux, Oracle VM and more. Really great to see continuing momentum behind the project and we're very happy to be involved.

Here's a list of the highlights that I had during the summit:

  • The operators track was a really excellent addition, with a chance for users/administrators to voice their opinions based on experiences. Really good to hear how OpenStack is making businesses more agile, but also equally good to hear about some of the continuing frustrations they have (fortunately many of them are new and being addressed). Seeing this discussion morph into a "Win the enterprise" working group is also very pleasing.
  • Enjoyed Troy Toman's keynote (Rackspace) about designing a planet scale cloud OS and the interoperability challenges ahead of us. I've been following some of the discussion around DefCore for a bit and while I have some concerns, I think it's mostly heading in the right direction. Certainly seems like there's a balance to strike to ensure that this effects the OpenStack vendors in such a way as to avoid negatively impacting our end users.
  • Also enjoyed Toby Ford's keynote (AT&T) about his desire for a NVF (Network Function Virtualization) architecture. What really resonated was also his desire for OpenStack to start addressing the typical enterprise workload, being less like cattle and more like pets.
  • The design summit was, as per usual, pretty intense for - definitely would get more value from these if I knew the code base a little better. Nevertheless, attended some really great sessions and got a better feeling of the roadmap for Juno.
  • Markus Flierl gave a great presentation (see below) at the demo theatre for what we're doing with OpenStack on Oracle Solaris (and more widely at Oracle across different products). Based on the discussions that we had at the Oracle booth, there's a huge amount of interest there and we talked to some great customers during the week about their thoughts and directions in this respect.
  • Undoubtedly Atlanta had some really good food. Highlights were the smoked ribs and brisket and the SweetWater brewing company. That said, I also loved the fried chicken, fried green tomatoes and collared greens, and wonderful hosting of "big momma" at Pitty Pat's Porch. Couldn't quite bring myself to eat biscuits and gravy in the morning though.
  • Visiting the World of Coca-Cola just before flying out. A total brain washing exercise, but very enjoyable. And very much liked Beverly (contrary to many other opinions on the internet) - but then again, I'd happily drink tonic water every day of the year...

Looking forward to Paris in November!

Tuesday May 06, 2014

Improved SMF Docs in Oracle Solaris 11.2

While there has been a ton of blogs posted about some of the new features of Oracle Solaris 11.2, one of those blogs in particular pleases me more than many others is the improved SMF documentation. While documentation doesn't exactly sound particularly exciting, it's absolutely crucial given the increasing importance that SMF is playing as a foundation for Oracle Solaris. Alta Estad has done a phenomenal job at improving the SMF documentation and accurately represents the hard work the SMF engineering team have been doing here.

One of the really exciting enhancements in SMF is actually a feature, unbeknownst to many, that has been available in the ZFS storage appliance for some time - SMF Stencils. Stencils are a way of taking advantage of the SMF configuration repository without having to rewrite your application to use libscf. Essentially by using a stencil you can manage your application configuration within SMF and have it automatically mirrored out to the traditional configuration file using svcio. This gives us a much improved way of managing configuration in a more structured sense, and ultimately a much better upgrade and auditing experience. In fact we've taken advantage of SMF stencils while integrating the Puppet into Oracle Solaris 11.2 (as detailed here).

So without further ado, check out Introduction to the Service Management Facility.

About

To learn more about Oracle Solaris 11, check out an extensive list of resources including technical articles, cheat sheets and screencasts on Oracle Technology Network

Search

Archives
« September 2015
SunMonTueWedThuFriSat
  
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today