Wednesday Apr 20, 2016

Oracle Solaris VMs in the Oracle Cloud Compute Service

Over the past few months I've been helping to get Oracle Solaris available in the Oracle Cloud. In particular, the Oracle Cloud Compute Service is an Infrastructure as a Service (IaaS) offering that allows users to create VMs on demand using a self-service web portal. Not unlike OpenStack, users can choose from a variety of shapes based on the number of virtual CPUs (defined by Oracle CPUs) and memory to allocate to the VM. There are a number of pricing options from dedicated environments that are single tenant, unmetered and avoid any possible noisy neighbour effect, to shared, multi-tenant environments.

Creating a VM on the service is super easy and I've walked through a typical user experience at the following screencast:

To be consistent with the existing experience using the compute service with Oracle Linux, we've pre-configured a default opc user that you can use to log into your VM environment using your SSH public keys. We've made sure though that's we're also consistent with Oracle Solaris security best practices, so this user will be assigned the System Administrator privilege, and have access to the root role with an expired password of solaris_opc. We've also changed the default IPS publisher from the Oracle Solaris release repository to the support repository - since all users of the compute service will get a support entitlement to access the latest Support Repository Updates and file bugs against Oracle Solaris at no additional cost.

There's a couple of gotchas that are worth noting - in particular when you create a bootable storage volume separately to creating your VM instance. This is because the compute cloud service also supports the ability to create ephemeral storage volumes (storage volumes that are created and destroyed as you start and stop your VM). If you're wanting to update the VM with the latest SRU, you'll need to be careful during your instance creation, as shown in the following screencast:

So take the Oracle Cloud Compute Service for a spin! It's now extremely easy to launch new Oracle Solaris VMs there that you can use to run your application workloads and take advantage of all the unique, enterprise grade technologies in the platform.

Playing around with the Oracle Storage Cloud Service

As part of my work to bring Oracle Solaris into the Oracle Cloud Compute Service, I've had the opportunity to spend a little time playing around with the Oracle Storage Cloud Service. Put simply, this service provides secure, scalable object storage similar to other storage services out there such as Amazon S3 or Dropbox.

The documentation is pretty useful. Like most storage services, there's support for a variety of different interfaces - from a REST API, to a Java client library, and a number of 3rd party tools like OpenStack Swift.

In the rest of the blog, I'll demonstrate some of the basic operations you can do - for convience I'll log into a VM running Oracle Solaris in the Oracle Cloud:

$ ssh opc@X.X.X.X
Last login: Tue Mar 22 01:27:35 2016 from 121-73-133-192.
Oracle Corporation      SunOS 5.11      11.3    December 2015
opc@fa922d:~$ uname -a
SunOS fa922d 5.11 11.3 i86pc i386 i86pc
opc@fa922d:~$ virtinfo
NAME            CLASS     
xen             current   
non-global-zone supported 
opc@fa922d:~$ pkg info entire
          Name: entire
       Summary: entire incorporation including Support Repository Update (Oracle Solaris
   Description: This package constrains system package versions to the same
                build.  WARNING: Proper system update and correct package
                selection depend on the presence of this incorporation.
                Removing this package will result in an unsupported system.  For
                more information see:
      Category: Meta Packages/Incorporations
         State: Installed
     Publisher: solaris
       Version: 0.5.11 (Oracle Solaris
 Build Release: 5.11
Packaging Date: January 14, 2016 09:32:35 PM 
          Size: 5.46 kB
          FMRI: pkg://solaris/entire@0.5.11,5.11-

We'll start by using the REST API. To find the service endpoint, we can log into the web portal and look at the REST Endpoint field as shown in the image below:

This gets us something like (which will include the identity domain that you're assoicated with). Once we have this, we can make a calls to the API based on a variation of this URL. We'll need to authenticate with the service first and get back a token that we'll use in subsequent REST calls. For that we'll need to set two headers: X-Storage-User and X-Storage-Pass, that is a combination of our identity domain, username, and password. In my case my identity domain is compute-pm and username is Let's start by using curl with the verbose flags and defaulting to HTTP 1.0 (scroll the entries below to see the full content):

opc@fa922d:~$ curl -v -O -H "X-Storage-User:" -H "X-Storage-Pass: XXXXXXXX"
*   Trying
* Failed to set TCP_KEEPALIVE on fd 4
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* Connected to ( port 443 (#0)
* successfully set certificate verify locations:
*   CAfile: none
  CApath: /etc/openssl/certs
* TLSv1.2, TLS handshake, Client hello (1):
} [174 bytes data]
* TLSv1.2, TLS handshake, Server hello (2):
{ [85 bytes data]
* TLSv1.2, TLS handshake, CERT (11):
{ [2861 bytes data]
* TLSv1.2, TLS handshake, Server finished (14):
{ [4 bytes data]
* TLSv1.2, TLS handshake, Client key exchange (16):
} [262 bytes data]
* TLSv1.2, TLS change cipher, Client hello (1):
} [1 bytes data]
* TLSv1.2, TLS handshake, Finished (20):
} [16 bytes data]
* TLSv1.2, TLS change cipher, Client hello (1):
{ [1 bytes data]
* TLSv1.2, TLS handshake, Finished (20):
{ [16 bytes data]
* SSL connection using TLSv1.2 / AES128-SHA
* Server certificate:
*        subject: C=US; ST=California; L=Redwood Shores; O=Oracle Corporation; CN=*
*        start date: 2015-08-07 00:00:00 GMT
*        expire date: 2016-10-05 23:59:59 GMT
*        subjectAltName: matched
*        issuer: C=US; O=VeriSign, Inc.; OU=VeriSign Trust Network; OU=Terms of use at (c)10; CN=VeriSign Class 3 Secure Server CA - G3
*        SSL certificate verify ok.
> GET /auth/v1.0 HTTP/1.1
> User-Agent: curl/7.40.0
> Host:
> Accept: */*
> X-Storage-User:
> X-Storage-Pass: XXXXXXXX
< HTTP/1.1 200 OK
< Date: Tue, 22 Mar 2016 02:31:44 GMT
< X-Auth-Token: AUTH_tk04c63cb4928e2aacb26f4506d6c9a331
< X-Storage-Token: AUTH_tk04c63cb4928e2aacb26f4506d6c9a331
< X-Storage-Url:
< Content-Length: 0
< Server: Oracle-Storage-Cloud-Service
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
* Connection #0 to host left intact

We can see that we've successfully completed the authentication and the service has passed by a token indicated by the X-Auth-Token response header.

Within the storage service, we have a heirarchy of containers and objects. Containers is a user created resource and can hold any number of objects. An object is created typically when a file is uploaded to the service. Containers cannot be nested, but meta-data can be attached to both containers and objects to make searching and managing easier. To list the containers associated with my identity domain we call another API as follows:

opc@fa922d:~$ curl -0 -X GET -H "X-Auth-Token: AUTH_tk04c63cb4928e2aacb26f4506d6c9a331"

We can list objects in the container by specifying the container we would like to query:

opc@fa922d:~$ curl -0 -X GET -H "X-Auth-Token: AUTH_tk04c63cb4928e2aacb26f4506d6c9a331"

Let's try and download this Oracle Solaris image as follows:

opc@fa922d:~$ curl -O -X GET -H "X-Auth-Token: AUTH_tk04c63cb4928e2aacb26f4506d6c9a331" > s11_3_4_5_0-System.img.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 1395M  100 1395M    0     0  88146      0  4:36:43  4:36:43 --:--:--  203k

Let's create a new container:

opc@fa922d:~$ curl -0 -X PUT -H "X-Auth-Token: AUTH_tk04c63cb4928e2aacb26f4506d6c9a331"
opc@fa922d:~$ curl -0 -X GET -H "X-Auth-Token: AUTH_tk04c63cb4928e2aacb26f4506d6c9a331"

Let's push a new file into this container and verify it's there:

opc@fa922d:~$ curl -O -X PUT -H "X-Auth-Token: AUTH_tk04c63cb4928e2aacb26f4506d6c9a331"
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1626    0     0  100  1626      0   3504 --:--:-- --:--:-- --:--:--  8424
opc@fa922d:~$ curl -0 -X GET -H "X-Auth-Token: AUTH_tk04c63cb4928e2aacb26f4506d6c9a331"

So rather than a raw set of HTTP requests, let's use something a little nicer. The Oracle Storage Cloud Service is compatible with the OpenStack Swift service, and so we can use the Python based swift client. We'll first need to install it into our VM:

opc@fa922d:~$ su -
Oracle Corporation      SunOS 5.11      11.3    December 2015
root@fa922d:~# pkg install swiftclient
           Packages to install: 84
       Create boot environment: No
Create backup boot environment: No

DOWNLOAD                                PKGS         FILES    XFER (MB)   SPEED
Completed                              84/84     9622/9622    39.2/39.2  943k/s

PHASE                                          ITEMS
Installing new actions                   11229/11229
Updating package state database                 Done
Updating package cache                           0/0
Updating image state                            Done
Creating fast lookup database                   Done
Updating package cache                           1/1
root@fa922d:~# exit
opc@fa922d:~# which swift

To use the client, we'll need to set some environmental variables. Rather than setting the OS_* variables, we'll have to use the legacy ST_* ones for the 1.0 versioned API as follows:

opc@fa922d:~$ export ST_AUTH=
opc@fa922d:~$ export
opc@fa922d:~$ export ST_KEY=XXXXXXXX

Now let's list the containers:

opc@fa922d:~$ swift list

And list the objects of new-container:

opc@fa922d:~$ swift list new-container

We can get information about our file:

opc@fa922d:~$ swift stat new-container new-file
                  Account: Storage-computepm
                Container: new-container
                   Object: new-file
             Content Type: application/octet-stream;charset=UTF-8
           Content Length: 1626
            Last Modified: Tue, 22 Mar 2016 03:07:45 GMT
                     ETag: c3eaad86f6572bf9e52a9b0c16aa905c
            Accept-Ranges: bytes
                   Server: Oracle-Storage-Cloud-Service
               Connection: keep-alive
              X-Timestamp: 1458616064.51712
               X-Trans-Id: tx1f7a06307a7744909fe6a-0056f0b9b1ga
X-Last-Modified-Timestamp: 1458616064.51712

We can download a file as follows:

opc@fa922d:~$ swift download new-container new-file
new-file [auth 0.118s, headers 0.181s, total 0.181s, 0.026 MB/s]

Or upload a file into our container:

opc@fa922d:~$ swift upload new-container another-new-file
opc@fa922d:~$ swift list new-container

And finally, delete our container:

opc@fa922d:~$ swift delete new-container
opc@fa922d:~$ swift list

Attaching and Detaching Storage Volumes in Oracle Cloud Compute Service

If you haven't spent much time on Oracle Solaris in the past few years you may be completely unaware that it has a completely different file system and volume manager. Gone are the days of struggling with disk configuration and setting up file systems. We have ZFS now and it's *amazing*. ZFS is a complete file system and volume manager with integrated data services such as encryption, compression and software RAID. It designed for the next generation data requirements such as big data in the cloud.

If you've signed up to the Oracle Cloud Compute Service you'll now be able to launch Oracle Solaris based VMs in this environment - an enterprise platform on demand. At some stage you'll probably want to create new storage volumes and attach them into your running instances. This is incredibly simple using ZFS, but to help you with the basics I've recorded two screencasts that might be of use.

Attaching Storage Volumes to Oracle Solaris VMs in Oracle Cloud Compute Service

Attaching and detaching storage volumes in Oracle Solaris VMs in Oracle Cloud Compute Service

So take an Oracle Solaris VM for a spin and let us know how you get on!

Friday Oct 09, 2015

DevOps on Oracle Solaris 11

There's no doubting the popularity of the DevOps movement these days. We're seeing it in many of our customers as the need to move faster in their business becomes more important. More often than not, it's being combined with the move to cloud computing and self-service everything. The traditional application development model with infrastructural and organizational silos is dead....well, almost!

DevOps promotes an open culture of collaboration, merging these silos into central teams with a more agile development methodology. Everything from development to production is automated as much as possible, allowing applications to be continuously developed, built, and released into production. In this environment everything is monitored and measured allowing for faster feedback back into the development cycles, with many incremental changes over short time periods. While the key to success for a DevOps environment is really the work environment itself, we've certainly seen some changes to tools that have made such an agile methodology much, much easier.

Many folks connect DevOps with Linux on commodity x86 based systems in the cloud. Not necessarily so! In my latest technical article Automated Application Development and Deployment with DevOps on Oracle Solaris 11, I put a simple application pipeline together to demonstrate a typical DevOps environment on Oracle Solaris 11. In this article, we'll take a look at Git distributed version control, Apache Maven build automation tool, Jenkins continuous integration server, and Puppet configuration management. I'll also show some integration with IPS using a Maven IPS plugin to automatically generate new packages that can be quickly deployed on a successful test run.

Let me know what you think!

Wednesday Dec 11, 2013

Oracle joins OpenStack Foundation

Oracle is now joining the OpenStack Foundation and planning on using the OpenStack technology in a wide variety of its products, including Oracle Solaris. This is pretty exciting generally and I think that OpenStack shows a huge amount of promise and ties together functionality that customers have been looking to have for quite some time. While we've had the concept of a unified compute, storage and networking layer for a while now when you provision Oracle Solaris Zones, being able to do this at scale across multiple nodes with both a management interface, and set of CLI and RESTful APIs is very appealing - and having Oracle Solaris as a rock solid OS foundation we're looking forward to adding a lot of value there. We're looking forward to contributing upstream (on Nova, Cinder and Neutron drivers initially, and then in other areas where we can help out).

I was fortunate enough to attend the OpenStack Summit in Hong Kong. It was an eye opening summit in many ways, mostly from the sheer volume of different technical projects and approach to solving problems in the data center. Not only that, it was also great to be able to catch up with friends of communities past.

Update: Already some folks are asking 'where's the code?' Oracle, as many can imagine, has policies and procedures around open source community contribution - especially when they involve contributor agreements. Needless to say, it's taken a bit of time to get this approved internally. Hopefully we'll see some Oracle folks more involved in the community at some stage in the short term.


Friday Mar 22, 2013

Oracle Live Webcast - SPARC, March 26th

If you've ever wondered where Oracle has been putting its R&D lately, here's an event that you'll not want to miss. On Tuesday 26th March at 1pm PT, we'll be announcing a set of new Oracle SPARC based servers with the world's fastest microprocessor. Both Larry Ellison and John Fowler will be on stage with this live webcast.

While I can't obviously divulge what we'll be talking about in any detail, I've been very fortunate enough to see the engineering going into SPARC and Oracle Solaris day by day, month by month as we optimize right across the Oracle stack, from applications through to disk. Few other IT vendors have the opportunity to do anything close to this level of integration, and the results are going to be a game changer. I'll be live tweeting the event over on our ORCL_Solaris account, and Larry will be doing the same over on Facebook. If you've ever been in doubt with Oracle's intent around SPARC and Oracle Solaris, I think this will change your mind.

Register now!.

Friday Aug 10, 2012

Oracle Solaris at Oracle Open World

I've just been booking flights for this years Oracle Open World which is happening just over a month away on the 30th September through to the 4th October. For anyone who's been to the conference before, it's a pretty overwhelming experience which so much happening at the Moscone center and other locations across the city.

Oracle Solaris will have another strong presence at the conference once again with some really great content lined up. Here's some of my picks:

General Session: Oracle Solaris 11 Strategy, Engineering Insights, and RoadmapThis general session, hosted by Markus Flierl, head of Oracle Solaris Core Engineering, goes through the strategy and roadmap for Oracle Solaris. It covers how Oracle Solaris 11 is being deployed in cloud computing and the unique optimizations in Oracle Solaris 11 for the Oracle stack. The session also offers a sneak peak at the latest technology under development in Oracle Solaris and what customers can expect to see in the coming updates. Flierl will be joined by special guests from the core Oracle Solaris team and customers deploying Oracle Solaris 11.
Oracle Solaris 11 Panel: Insights and Directions from Oracle Solaris Core EngineeringThis session, featuring a panel of core engineers responsible for a wide range of key Oracle Solaris technologies, covers the current state of Oracle Solaris 11 and gives insights into the future direction of Oracle Solaris technologies and products. Come learn how Oracle Solaris can benefit your environment today and what will continue to make Oracle Solaris the best platform for deploying enterprise applications going forward, with even more features for performance, serviceability, cloud computing, deep Oracle stack optimizations, and ease of administration. The session also provides insights from customers deploying Oracle Solaris 11 in a range of data center environments.
Cloud Formation: Implementing IaaS in Practice with Oracle SolarisBe it public or private, IaaS or PaaS, using a cloud infrastructure has become the way to build out not only tomorrow’s but also today’s data center. Although there are many parts to a cloud solution, you are going to be left with a whole bunch of problems to solve if you don’t have an effective OS built for the cloud. In this session, learn how Oracle Solaris, the first cloud OS, provides the features you need built-in—no effort required. Be it elastic infrastructure, unrivaled security, superfast installation and deployment, nonstop availability, or crystal-clear observability, you will learn how Oracle Solaris provides it. Besides the “what,” also learn the “how” from a customer study on how Oracle Solaris is used in the cloud today to implement the Oracle stack.
Oracle Solaris 11 Serviceability Best PracticesOracle Solaris 11 provides many facilities that can help identify when a problem has occurred, provide information about it, and make it easier to collect the information required to solve it. This session provides insight into the the best way to install, configure, and update your Oracle Solaris systems. The focus is on optimizing the lifecycle of your Oracle Solaris infrastructure to make it easier to maintain.
Customer Insight: Oracle Solaris on Oracle Exadata, Oracle Exalogic, and SPARC SuperClusterIn this session, you’ll hear from existing customers what benefits they have realized from using the Oracle stack on Oracle Exadata and Oracle’s SPARC SuperCluster and from using Oracle Solaris on those engineered systems. Oracle Solaris brings lightweight OS virtualization (the Oracle Solaris Zones feature), enterprise reliability and scale, and many other key features of engineered systems for rapid ROI. Learn how Oracle Solaris and Oracle engineered systems can bring value to your organization.
Oracle Solaris 11 Security and Data Management: Reduce Risk and Deliver Secure ServicesOracle Solaris 11 ZFS and Oracle Solaris security features provide a comprehensive, secure data management environment for the data center and the cloud, combining guaranteed data integrity from disk platter to application and cryptographic data security. This session details the technologies that keep your data safe, using real-world examples. The discussion includes data integrity from application to disk, zones and application action whitelisting, and ZFS encryption. It also covers Oracle Key Manager integration and discusses best practices included in Oracle Optimized Solutions.
Large-Scale Installation and Deployment of Oracle Solaris 11 Hands on LabThis hands-on lab builds awareness of and instills confidence for deploying Oracle Solaris 11 in large installations. It addresses challenges associated with rolling out, and subsequently maintaining, traditional Oracle Solaris installations across the enterprise. And it subsequently provides an overview of installation and package management innovations that address these specific challenges. Further, it reviews the built-in tools provided for transitioning to Oracle Solaris 11 as well as specific technical projects in Oracle Solaris development that have become the foundation for solidifying Oracle Solaris 11 installation and lifecycle management capabilities.
Virtualizing Your Oracle Solaris 11 Environment (Hands on Lab)The cloud and virtualization are core to Oracle Solaris 11. Come to this hands-on lab to see what that means. Learn all about the Oracle Solaris Zones feature of Oracle Solaris, about the new network virtualization technology, and how they both integrate.

The times and locations are still to be announced so check back to the Oracle Open World 2012 page frequently. There's a bunch of other pretty exciting events also - the Oracle Open World Music Festival, the Oracle Appreciation Event and Oktoberfest by the Bay!. All in all, should be a great week!

Thursday Jul 12, 2012

What's new in Oracle Solaris 11?

As product management lead for the Oracle Solaris 11 release back in November 2011, one of the many responsibilities I had was to compile a list of some of the many new features that we've integrated. With well over 15,000 individual putbacks to just the Oracle Solaris kernel alone, compiling a list of interesting features was a pretty big challenge.

Oracle Solaris 10 introduced an incredible number of game changing innovations - the ZFS file system with its ability to prevent against silent data corruption, built in volume management and snapshot and cloning; DTrace, the dynamic tracing framework that allows anyone to safely get a much more accurate view at precisely what is happening on a system at any time; SMF, the service management facility that ensures maximum availability of critical applications and system services with automatic restart should any hardware or software failures occur; and Zones, built-in OS virtualization allowing administrators a way of carving up virtual environments with almost no performance overhead.

Oracle Solaris 11 builds on those strong foundations, delivering a set of features that provide huge value to administrators dealing with complex problems in the data center. We've torn apart some of the assumptions that were made years ago and modernized the operating system to meet the challenges of today - everything from the initial installation and management of software, to setting up virtualized environments in a multi-tenanted cloud environment. The integration with this release has been one of our higest priorities - taking best advantage of the technologies that we've introduced and present them to the administrator to dramatically help in their day-to-day operations. I'll be talking more about these features in future blog posts, but for now, take a look at some of the new features we've introduced.

What's New in Oracle Solaris 11

Welcome to Under the Microscope

Welcome to the "Under the Microscope" blog. So you've recently decided to take a look at Oracle Solaris 11, the latest version of the #1 enterprise UNIX operating system? Well, we're here to help you find your way. I'm Glynn Foster, Oracle Solaris product manager working in Wellington, New Zealand. I've been working at Oracle since 2010, having joined them as part of the Sun Microsystems acquisition. I'll be taking you on a tour of some of the features of the operating system, some useful resources you can use to help you get started, and making sure you're aware of the latest news and events that are happening. So sit back, grab a beer, and start figuring out what the hype is all about. Cheers!

To learn more about Oracle Solaris 11, check out an extensive list of resources including technical articles, cheat sheets and screencasts on Oracle Technology Network


« May 2016