Thursday Sep 13, 2012

To encryption=on or encryption=off a simple ZFS Crypto demo

I've just been asked twice this week how I would demonstrate ZFS encryption really is encrypting the data on disk.  It needs to be really simple and the target isn't forensics or cryptanalysis just a quick demo to show the before and after.

I usually do this small demo using a pool based on files so I can run strings(1) on the "disks" that make up the pool. The demo will work with real disks too but it will take a lot longer (how much longer depends on the size of your disks).  The file hamlet.txt is this one from

# mkfile 64m /tmp/pool1_file
# zpool create clear_pool /tmp/pool1_file
# cp hamlet.txt /clear_pool
# grep -i hamlet /clear_pool/hamlet.txt | wc -l

Note the number of times hamlet appears

# zpool export clear_pool
# strings /tmp/pool1_file | grep -i hamlet | wc -l

Note the number of times hamlet appears on disk - it is 2 more because the file is called hamlet.txt and file names are in the clear as well and we keep at least two copies of metadata.

Now lets encrypt the file systems in the pool.
Note you MUST use a new pool file don't reuse the one from above.

# mkfile 64m /tmp/pool2_file
# zpool create -O encryption=on enc_pool /tmp/pool2_file
Enter passphrase for 'enc_pool': 
Enter again: 
# cp hamlet.txt /enc_pool
# grep -i hamlet /enc_pool/hamlet.txt | wc -l

Note the number of times hamlet appears is the same as before

# zpool export enc_pool
# strings /tmp/pool2_file | grep -i hamlet | wc -l

Note the word hamlet doesn't appear at all!

As a said above this isn't indended as "proof" that ZFS does encryption properly just as a quick to do demo.

Wednesday Jul 04, 2012

Delegation of Solaris Zone Administration

In Solaris 11 'Zone Delegation' is a built in feature. The Zones system now uses fine grained RBAC authorisations to allow delegation of management of distinct zones, rather than all zones which is what the 'Zone Management' RBAC profile did in Solaris 10.

The data for this can be stored with the Zone or you could also create RBAC profiles (that can even be stored in NIS or LDAP) for granting access to specific lists of Zones to administrators.

For example lets say we have zones named zoneA through zoneF and we have three admins alice, bob, carl.  We want to grant a subset of the zone management to each of them.

We could do that either by adding the admin resource to the appropriate zones via zonecfg(1M) or we could do something like this with RBAC data directly:

First lets look at an example of storing the data with the zone.

# zonecfg -z zoneA
zonecfg:zoneA> add admin
zonecfg:zoneA> set user=alice
zonecfg:zoneA> set auths=manage
zonecfg:zoneA> end
zonecfg:zoneA> commit
zonecfg:zoneA> exit

Now lets look at the alternate method of storing this directly in the RBAC database, but we will show all our admins and zones for this example:

# usermod -P +'Zone Management' -A alice

# usermod -A alice

# usermod -P +'Zone Management' -A bob
# usermod -A bob

# usermod -P +'Zone Management' -A carl
# usermod -A carl
# usermod -A carl
# usermod -A carl

In the above alice can only manage zoneA, bob can manage zoneB and zoneC and carl can manage zoneC through zoneF.  The user alice can also login on the console to zoneB but she can't do the operations that require the authorisation on it.

Or if you have a large number of zones and/or admins or you just want to provide a layer of abstraction you can collect the authorisation lists into an RBAC profile and grant that to the admins, for example lets great an RBAC profile for the things that alice and carl can do.

# profiles -p 'Zone Group 1'
profiles:Zone Group 1> set desc="Zone Group 1"
profiles:Zone Group 1> add profile="Zone Management"
profiles:Zone Group 1> add
profiles:Zone Group 1> add
profiles:Zone Group 1> commit
profiles:Zone Group 1> exit
# profiles -p 'Zone Group 3'
profiles:Zone Group 1> set desc="Zone Group 3"
profiles:Zone Group 1> add profile="Zone Management"
profiles:Zone Group 1> add
profiles:Zone Group 1> add
profiles:Zone Group 1> add
profiles:Zone Group 1> commit
profiles:Zone Group 1> exit

Now instead of granting carl  and aliace the 'Zone Management' profile and the authorisations directly we can just give them the appropriate profile.

# usermod -P +'Zone Group 3' carl

# usermod -P +'Zone Group 1' alice

If we wanted to store the profile data and the profiles granted to the users in LDAP just add '-S ldap' to the profiles and usermod commands.

For a documentation overview see the description of the "admin" resource in zonecfg(1M), profiles(1) and usermod(1M)

Tuesday May 01, 2012

Podcast: Immutable Zones in Oracle Solaris 11

In this episode of the "Oracle Solaris: In a Class By Itself" podcast series, the focus is a bit more technical. I was interviewed by host Charlie Boyle, Senior Director of Solaris Product Marketing. We talked about a new feature in Oracle Solaris 11: immutable zones. Those are read-only root zones for highly secure deployment scenarios.

See also my previous blog post on Enctypted Immutable Zones.

Wednesday Feb 29, 2012

Solaris 11 has the security solution Linus wants for Desktop Linux

Recently Linus Torvalds was venting (his words!) about the frustrating requirement to keep giving his root password for common desktop tasks such as connecting to a wifi network or configuring printers.

Well I'm very pleased to say that the Solaris 11 desktop doesn't have this problem thanks to our RBAC system and how it is used including how it is tightly integrated into the desktop.

One of the new RBAC features in Solaris 11 is location context RBAC profiles, by default we grant the user on the system console (ie the one on the laptop or workstation locally at the physical keyboard/screen) the "Console User" profile.  Which on a default install has the necessary authorisations and execution profiles to do things like joining a wireless network, changing CPU power management, and using removal media.   The user created at initial install time also has the much more powerful "System Administrator" profile granted to them so they can do even more without being required to give a password for root (they also have access to the root role and the ability to use sudo).

Authorisations in Solaris RBAC (which dates back in main stream Solaris to Solaris 8 and even further 17+ years in Trusted Solaris) are checked by privileged programs and the whole point is so you don't have to reauthenticate.  SMF is a very heavy user of RBAC authorisations.  In the case of things like joining a wireless network it is privileged daemons that are checking the authorisations of the clients connecting to them (usually over a door)

In addition to that GNOME in Solaris 11 has been explicitly integrated with Solaris RBAC as well, any GNOME menu entry that needs to run with elevated privilege will be exectuted via Solaris RBAC mechanisms.  The panel works out the least intrusive way to get the program running for you.  For example if I select "Wireshark" from the GNOME panel menu it just starts - I don't get prompted for any root password - but it starts with the necessary privileges because GNOME on Solaris 11 knows that I have the "Network Management" RBAC profile which allows running /usr/sbin/wireshark with the net_rawaccess privilege.   If I didn't have "Network Management" directly but I had an RBAC role that had it then GNOME would use gksu to assume the role (which might be root) and in which case I would have been prompted for the role password.  If you are using roleauth=user that password is yours and if you are using pam_tty_tickets you won't keep getting prompted.

GNOME can even go further and not even present menu entries to users who don't have granted to them any RBAC profile that allows running those programs - this is useful in a large multi user system like a Sun Ray deployment.

If you want to do it the "old way" and use the CLI and/or give a root password for every "mundane" little thing, you can still do that too if you really want to.

So maybe Linus could try Solaris 11 desktop ;-)

Monday Feb 20, 2012

Solaris 11 Common Criteria Evaluation

Oracle Solaris 11 is now "In Evaluation" for Common Criteria at EAL4+.  The protection profile is OSPP with the following extended packages: AM - Advanced Management  EIA - Extended Identification and Authentication, LS - Label Security, VIRT - Virtualization.  For information on other Oracle products that are evaluated under Common Criteria or FIPS 140 please see the general Oracle Security Evalutions page.

Please email for all inquiries regarding Oracle security evaluations, I can't answer questions about the content of the evaluation on this blog or directly by email to me.

Friday Feb 03, 2012

What Free/Open Source software is Solaris 11 still missing

Note this is not a commitment from Oracle to deliver anything as a result of your answers, nor is it an official survey of any kind.

Okay first my dirty little secret... my family home desktop machine runs Windows 7.  Earlier this week I had a need to check the MD5 or SHA256 checksum on an iso image I'd downloaded.  On Solaris I'd just run 'digest -a sha256' or sha256sum on Solaris or any Linux distro.  But on Windows 7 the best I could come up with was code it up in Java myself or install the GNU versions via Cygwin.

So that got me thinking, the Solaris 11 repository has a lot more "upstream" Free/Open Source tools and frameworks than any other release of Solaris ever had.  We have Python (which is really a core part of Solaris 11 now), Ruby loads of the GNU runtime and development toolchains and much much more.   However many common Linux distributions still have more than we do but some of that isn't target at server use cases.

So what Free/Open Source software is Solaris 11 still missing that you use to run your business on your Solaris servers?

Even if you don't have Solaris 11 installed you can quickly search for packags at

Please add details in the comments.

Again note this is not a commitment from Oracle to deliver anything as a result of your answers, nor is it an official survey of any kind, just my curiosity.  I will of course log the relevant bugs for viable things if any come up.

Update 1: Thanks for all the submissions so far, some great suggestions in there - keep them coming and don't worry about looking for duplicates in others comments (in fact I'd rather things were listed my multiple people since it shows more interest in a given component).

Update 2: comments are moderated (site requirement), submitting multiple times unfortunately sometimes results in you being told your comment is spam but I still see it and will approve it. Thanks for your patience.

Tuesday Dec 20, 2011

How low can we go ? (Minimised install of Solaris 11)

I wondered how little we can actually install as a starting point for building a minimised system. The new IPS package system makes this much easier and makes it work in a supportable way without all the pit falls of patches and packages we had previously.

For Solaris 11 I believe the currently smallest configuration we recommend is the solaris-small-server group package.

Note the following is not intended to imply this is a supported configuration and is illustrative only of a short amount of investigative work.

First lets look at a zone (it is easier since there are no driver issues to deal with): I discovered it is possible to get a 'working' zone by choosing a single package to install in the zone AI manifest and that package is: pkg:/package/pkg

That resulted in 175M being reported as being transferred by pkg. Which results in 255M on disk of which about 55M is /var/pkg. I had 140 'real' packages (ie excluding the incorporations). We have 71 online SMF services (nothing in maintenance) with 96 known SMF services. Around 23 processes are running (excluding the ps I used to check this and the shell I was logged in on).

I have discovered some potential packaging that could result in this being a little bit smaller but not much unless a break up of pkg:/system/core-os was done.

Now onto the bare metal install case. This was a little harder and I've not gotten it to where I wanted yet.

Ignoring drivers the only thing I needed on an x86 system was: pkg:/package/pkg and pkg:/system/boot/grub

Which is good and not really different from the zones case. However that won't produce a bootable system - even though it produces one that will install!

To get it to boot I took the list of all the network and storage drivers from the solaris-small-server group package.  I removed all the wlan drivers and also any drivers I knew to be SPARC only.   My list of packages in the AI manifest had 113 driver packages in it. That got me a bootable system, though not one minimized with respect to drivers.

We have a few more processes in the global zone (again ignoring the ps and my shell) this time I counted 32.  This came from 89 online services. Again ignoring the incorporation packages  I had 161 packages installed of which 73 were in the pkg:/driver namespace.

The disk space footprint is much bigger at total of 730M - but remember I've likely installed drivers that I might not need. This time /var/pkg is 174M of that.

Tuesday Nov 22, 2011

HOWTO Turn off SPARC T4 or Intel AES-NI crypto acceleration.

Since we released hardware crypto acceleration for SPARC T4 and Intel AES-NI support we have had a common question come up: 'How do I test without the hardware crypto acceleration?'.

Initially this came up just for development use so developers can do unit testing on a machine that has hardware offload but still cover the code paths for a machine that doesn't (our integration and release testing would run on all supported types of hardware anyway).  I've also seen it asked in a customer context too so that we can show that there is a performance gain from the hardware crypto acceleration, (not just the fact that SPARC T4 much faster performing processor than T3) and measure what it is for their application.

With SPARC T2/T3 we could easily disable the hardware crypto offload by running 'cryptoadm disable provider=n2cp/0'.  We can't do that with SPARC T4 or with Intel AES-NI because in both of those classes of processor the encryption doesn't require a device driver instead it is unprivileged user land callable instructions.

Turns out there is away to do this by using features of the Solaris runtime loader ( First I need to expose a little bit of implementation detail about how the Solaris Cryptographic Framework is implemented in Solaris 11.  One of the new Solaris 11 features of the linker/loader is the ability to have a single ELF object that has multiple different implementations of the same functions that are selected at runtime based on the capabilities of the machine.  The alternate to this is having the application coded to call getisax() and make the choice itself.  We use this functionality of the linker/loader when we build the userland libraries for the Solaris Cryptographic Framework (specifically, and the unfortunately misnamed due to historical reasons

The Solaris linker/loader allows control of a lot of its functionality via environment variables, we can use that to control the version of the cryptographic functions we run.  To do this we simply export the LD_HWCAP environment variable with values that tell to not select the HWCAP section matching certain features even if isainfo says they are present. 

For SPARC T4 that would be:

export LD_HWCAP="-aes -des -md5 -sha256 -sha512 -mont -mpmul" 

and for Intel systems with AES-NI support:

export LD_HWCAP="-aes"

This will work for consumers of the Solaris Cryptographic Framework that use the Solaris PKCS#11 libraries or use interfaces directly.  It also works for the Oracle DB and Java JCE.  However does not work for the default enabled OpenSSL "t4" or "aes-ni" engines (unfortunately) because they do explicit calls to getisax() themselves rather than using multiple ELF cap sections.

However we can still use OpenSSL to demonstrate this by explicitly selecting "pkcs11" engine  using only a single process and thread. 

$ openssl speed -engine pkcs11 -evp aes-128-cbc
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
aes-128-cbc      54170.81k   187416.00k   489725.70k   805445.63k  1018880.00k

$ LD_HWCAP="-aes" openssl speed -engine pkcs11 -evp aes-128-cbc
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
aes-128-cbc      29376.37k    58328.13k    79031.55k    86738.26k    89191.77k

We can clearly see the difference this makes in the case where AES offload to the SPARC T4 was disabled. The "t4" engine is faster than the pkcs11 one because there is less overhead (again on a SPARC T4-1 using only a single process/thread - using -multi you will get even bigger numbers).

$ openssl speed -evp aes-128-cbc
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
aes-128-cbc      85526.61k    89298.84k    91970.30k    92662.78k    92842.67k

Yet another cool feature of the Solaris linker/loader, thanks Rod and Ali.

Note these above openssl speed output is not intended to show the actual performance of any particular benchmark just that there is a significant improvement from using hardware acceleration on SPARC T4. For cryptographic performance benchmarks see the postings.

Thursday Nov 17, 2011

OpenSSL Versions in Solaris

Those of you have have installed Solaris 11 or have read some of the blogs by my colleagues will have noticed Solaris 11 includes OpenSSL 1.0.0, this is a different version to what we have in Solaris 10.  I hope the following explains why that is and how it fits with the expectations on binary compatibility between Solaris releases.

Solaris 10 was the first release where we included OpenSSL libraries and headers (part of it was actually statically linked into the SSH client/server in Solaris 9).  At time we were building and releasing Solaris 10 the current train of OpenSSL was 0.9.7. 

The OpenSSL libraries at that time were known to not always be completely API and ABI (binary) compatible between releases (some times even in the lettered patch releases) though mostly if you stuck with the documented high level APIs you would be fine.   For this reason OpenSSL was classified as a 'Volatile' interface and in Solaris 10 Volatile interfaces were not part of the default library search path which is why the OpenSSL libraries live in /usr/sfw/lib on Solaris 10.  Okay, but what does Volatile mean ?

Quoting from the attributes(5) man page description of Volatile (which was called External in older taxonomy):

         Volatile interfaces can change at any time and  for  any

         The Volatile interface stability level allows  Sun  pro-
         ducts to quickly track a fluid, rapidly evolving specif-
         ication. In many cases, this is preferred  to  providing
         additional  stability to the interface, as it may better
         meet the expectations of the consumer.

         The most common application of this taxonomy level is to
         interfaces that are controlled by a body other than Sun,
         but unlike specifications controlled by standards bodies
         or Free or Open Source Software (FOSS) communities which
         value interface compatibility, it can  not  be  asserted
         that  an incompatible change to the interface specifica-
         tion would be exceedingly rare. It may also  be  applied
         to  FOSS  controlled  software  where  it is deemed more
         important to track the community  with  minimal  latency
         than to provide stability to our customers.

         It also common  to  apply  the  Volatile  classification
         level  to  interfaces in the process of being defined by
         trusted or  widely  accepted  organization.   These  are
         generically  referred  to  as draft standards.  An "IETF
         Internet draft"  is  a  well  understood  example  of  a
         specification under development.

         Volatile can also be applied to experimental interfaces.

         No assertion is made regarding either source  or  binary
         compatibility  of  Volatile  interfaces  between any two
         releases,  including  patches.  Applications  containing
         these  interfaces might fail to function properly in any
         future release.

Note that last paragraph!  OpenSSL is only one example of the many interfaces in Solaris that are classified as Volatile.  At the other end of the scale we have Committed (Stable in Solaris 10 terminology) interfaces, these include things like the POSIX APIs or Solaris specific APIs that we have no intention of changing in an incompatible way.  There are also Private interfaces and things we declare as Not-an-Interface (eg command output not intended for scripting against only to be read by humans).

Even if we had declared OpenSSL as a Committed/Stable interface in Solaris 10 there are allowed exceptions, again quoting from attributes(5):

         4.   An interface specification which  isn't  controlled
              by  Sun  has been changed incompatibly and the vast
              majority of interface consumers  expect  the  newer

         5.   Not  making  the  incompatible  change   would   be
              incomprehensible  to our customers. 

In our opinion and that of our large and small customers keeping up with the OpenSSL community is important, and certainly both of the above cases apply.

Our policy for dealing with OpenSSL on Solaris 10 was to stay at 0.9.7 and add fixes for security vulnerabilities (the version string includes the CVE numbers of fixed vulnerabilities relevant to that release train).  The last release of OpenSSL 0.9.7 delivered by the upstream community was more than 4 years ago in Feb 2007.

Now lets roll forward to just before the release of Solaris 11 Express in 2010. By that point in time the current OpenSSL release was 0.9.8 with the 1.0.0 release known to be coming soon.  Two significant changes to OpenSSL were made between Solaris 10 and Solaris 11 Express.  First in Solaris 11 Express (and Solaris 11) we removed the requirement that Volatile libraries be placed in /usr/sfw/lib, that means OpenSSL is now in /usr/lib, secondly we upgraded it to the then current version stream of OpenSSL (0.9.8) as was expected by our customers.

In between Solaris 11 Express in 2010 and the release of Solaris 11 in 2011 the OpenSSL community released version 1.0.0.  This was a huge milestone for a long standing and highly respected open source project.  It would have been highly negligent of Solaris not to include OpenSSL 1.0.0e in the Solaris 11 release. It is the latest best supported and best performing version.  


In fact Solaris 11 isn't 'just' OpenSSL 1.0.0 but we have added our SPARC T4 engine and the AES-NI engine to support the on chip crypto acceleration. This gives us 4.3x better AES performance than OpenSSL 0.9.8 running on AIX on an IBM POWER7. We are now working with the OpenSSL community to determine how best to integrate the SPARC T4 changes into the mainline OpenSSL.  The OpenSSL 'pkcs11' engine we delivered in Solaris 10 to support the CA-6000 card and the SPARC T1/T2/T3 hardware is still included in Solaris 11.

When OpenSSL 1.0.1 and 1.1.0 come out we will asses what is best for Solaris customers. It might be upgrade or it might be parallel delivery of more than one version stream.  At this time Solaris 11 still classifies OpenSSL as a Volatile interface, it is our hope that we will be able at some point in a future release to give it a higher interface stability level.

Happy crypting! and thank-you OpenSSL community for all the work you have done that helps Solaris.

Wednesday Nov 09, 2011

Completely disabling root logins on Solaris 11

Since Solaris 8 it has been possible to make the root account a role.  That means you can't login directly as root (except in single user mode) but have to login as an authorised user first and assume (via su) the root role.  This still required the root account to have a valid and known password as it is needed for the su step and for single user access.

With Solaris 11 it is possible to go one step further and completely disable all need for a root password even for access in single user mode.

There are two complementary new features that make this possible.  The first is the ability to change which password is used when authenticating to a role.  A new per role property called roleauth was added, if it isn't present the prior behaviour of using the role account password is retained, if roleauth=user is set instead then the password of the user assuming the role is used.

The second feature was one that existed in the Solaris 11 Express release which changed how the sulogin command worked, prior releases all just asked for the root password.  The sulogin program was changed to authenticate a specific user instead so now asks for a username and the password of that user.  The user must be one authorised to enter single user mode by being granted the 'solaris.system.maintenance' authorisation - and obviously be one that can actually connect to the system console (which I recommend is protected by "other means" eg ILOM level accounts or central "terminal server")

The following sequence of commands takes root from being a normal root account (which depending on how you install Solaris 11 it maybe, or it might already be a role) and granting the user darrrenm the ability to assume the root role and enter single user mode.

# usermod -K type=role root
# usermod -R +root -A +solaris.system.maintenance darrenm
# rolemod -K roleauth=user  root
# passwd -N root

Note that some of the install methods for Solaris 11 will have created an initial user account that is granted the root role and has been given the "System Administrator" profile, in those cases only the last two steps are required as the equivalent of the first two will already have been done at install time for the initial non root user.

Note that we do not lock (-l) the root account but instead ensure it has no valid password (-N) this is because the root account does still have some cron jobs that we ideally want to run and if it was locked then the PAM module would prevent cron from running those jobs.

When root is a role like this you authenticate to the system first as yourself, in this case the user darrenm logs in first.  Once darrenm has logged in we use su(1M) to be come root - just like we would have if root wasn't a role.  The main difference here is that the password given to su(1M) in the above config is darrenm's password.

If you boot the system in single user mode (boot -s) you will be asked for a username and password, we give the username of darrenm and darrenm's password. Once you do that you get a # prompt that is truely root in single user mode.  The distinction here is we have an audit trail and know it was darrenm that authenticated and we have no separate root password to manage.

In some deployment cases there may not be any locally defined accounts, in those cases it is necessary to allow the root to allow direct login on the system console in multiuser mode.  This is achived by adding the following to /etc/pam.conf, and also give the root account a valid password.

login account required

By having that entry we do not have active for console login so the root account will be able to login directly.  The assumption here is that access to the system console is sufficiently secured (and audite) by means external to the Solaris instance.  For example the ILOM of the system is on an access restricted management network that has specific user accounts for SSH access to the ILOM.  You may also want to only give out that root password in emergency cases.  This will allow direct root login only on the console but require that users authenticate to root using their own password when using su.


If you have made root as role and you want to go back to a traditional direct login capability for root you can do so by simply running:

 # rolemod -K type=normal root

Update 1 to answer the first question: Basically exactly the same as if the password was locked, expired or forgotten if you just used root directly.  Failed account locking is not enabled by default.  As for forgetting who was the authorised account that isn't a problem Solaris can fix on its own that is part of your administative procedures.  You can have any number of authorised users and the userattr, roles, profiles commands can be used tell you who they are and manage them.

Update 2 to make it clearer how you use this in multi-user and single user.

Update 3 add information on how to allow root on console.

Password (PAM) caching for Solaris su - "a la sudo"

I talk to a lot of users about Solaris RBAC but many of them prefer to use sudo for various reasons.  One the common usability features that users like is the that they don't have to continually type their password.  This is because sudo uses a "ticket" system for caching the authentication for a defined period (by default 5 minutes).

To bring this usability feature to Solaris 11 I wrote a new PAM module (pam_tty_tickets) that provides a similar style of caching for Solaris roles. 

By default the tickets are stored in /system/volatile/tty_tickets (/var/run is a symlink to /system/volatile now). 

When using su(1M) the user you currently are is set in PAM_USER and PAM_AUSER is the user you are becoming (ie the username argument to su or root if one is not specified).  The PAM module implements the caching using tickets, the internal format of the tickets is the same as what sudo uses. The location can be changed to be compatible with sudo so the same ticket can be used for su and sudo.

To enable pam_tty_tickets for su put the following into /etc/pam.conf (the module is in the pkg:/system/library package so it is always installed but not configured for use by default):

su      auth required 
su      auth sufficient
su      auth requisite
su      auth required 

So what does it now look like:

braveheart:pts/3$ su -
root@braveheart:~# id -a
uid=0(root) gid=0(root) groups=0(root),1(other),2(bin),3(sys),4(adm),5(uucp),6(mail),7(tty),8(lp),9(nuucp),12(daemon)
darrenm@braveheart:~# exit
braveheart:pts/3$ su -

If you want to enable it in the desktop for gksu then you need to add a similar set of changes to /etc/pam.conf with the service name as "embedded_su" with the same modules as is  listed above.  The default timeout matches the sudo default of 5 minutes, the timeout= module option allows specifying a different timeout.

[ NOTE: The man page for pam_tty_tickets was mistakenly placed in section 1 for Solaris 11, it should have been in section 5. ]

Update for Solaris 11.1, now that we have /etc/pam.d/ support it is recommended that instead of updating /etc/pam.conf the following lines be placed into /etc/pam.d/su

auth sufficient
auth definitive
auth requisite
auth required
auth required

User home directory encryption with ZFS

ZFS encryption has a very flexible key management capability, including the option to delegate key management to individual users.  We can use this together with a PAM module I wrote to provide per user encrypted home directories.  My laptop and workstation at Oracle are configured like this:

First lest setup console login for encrypted home directories:

    root@ltz:~# cat >> /etc/pam.conf<<_EOM
    login auth     required create
    other password required

The first line ensures that when we login on the console bob's home directory is created with as an encrypted ZFS file system if it doesn't already exist, the second one ensures that the passphrase for it stays in sync with his login password.

Now lets create a new user 'bob' who looks after his own encryption key for is home directory, note that we do not specify '-m' to useradd so that pam_zfs_key will create the home directory when the user logs in.

root@ltz:~# useradd bob
root@ltz:~# passwd bob
New Password: 
Re-enter new Password: 
passwd: password successfully changed for bob
root@ltz:~# passwd -f bob
passwd: password information changed for bob

We have now created the user bob with an expired password. Lets login as bob and see what happens:

    ltz console login: bob
    Choose a new password.
    New Password: 
    Re-enter new Password: 
    login: password successfully changed for bob
    Creating home directory with encryption=on.
    Your login password will be used as the wrapping key.
    Last login: Tue Oct 18 12:55:59 on console
    Oracle Corporation      SunOS 5.11      11.0    November 2011
    -bash-4.1$ /usr/sbin/zfs get encryption,keysource rpool/export/home/bob
    NAME                   PROPERTY    VALUE              SOURCE
    rpool/export/home/bob  encryption  on                 local
    rpool/export/home/bob  keysource   passphrase,prompt  local

Note that bob had to first change the expired password. After we provided a new login password a new ZFS file system for bob's home directory was created. The new login password that bob chose is also the passphrase for this ZFS encrypted home directory. This means that at no time did the administrator ever know the passphrase for bob's home directory. After the machine reboots bob's home directory won't be mounted anymore until bob logs in again.  If we want bob's home directory to be unmounted and the key removed from the kernel when bob logs out (even if the system isn't rebooting) then we can add the 'force' option to the module line in /etc/pam.conf

If users login with GDM or ssh then there is a little more configuration needed in /etc/pam.conf to enable pam_zfs_key for those services as well.

root@ltz:~# cat >> /etc/pam.conf<<_EOM
gdm     auth requisite
gdm     auth required 
gdm     auth required 
gdm     auth required  create
gdm     auth required 

root@ltz:~# cat >> /etc/pam.conf<<_EOM
sshd-kbdint     auth requisite
sshd-kbdint     auth required 
sshd-kbdint     auth required 
sshd-kbdint     auth required  create
sshd-kbdint     auth required 

Note that this only works when we are logging in to SSH with a password. Not if we are doing pubkey authentication because the encryption passphrase for the home directory hasn't been supplied. However pubkey and gssapi will work for later authentications after the home directory is mounted up since the ZFS passphrase is supplied during that first ssh or gdm login.

Immutable Zones on Encrypted ZFS

Rather that just discussing the new Immutable Zones feature of Solaris 11, I'm going to show how it can be combined with ZFS file system encryption as part of a defense in depth deployment.

Lets assume as part of our security threat model we need to protect data written to disk but we also want to protect the system from malicious or accidental tampering with (system binaries and configuration) during runtime.

Deploying our application in a Solaris Zone allows us to provide both of those, even for a user that has gained root access inside the zone.  We will use two new features of Solaris 11 to do this.  Firstly ZFS encryption to provide protection of the data written to disk and secondly the new 'file-mac-profile' mandatory write access feature that gives us Immtuable Zones.

Normally we can let 'zoneadm install' create the ZFS file system for the zone for us, but it is also perfectly happy using a file system that already exists. We can use that to our advantage to enable encryption for the Zone. So lets first setup our encrypted dataset and put a zone on it.  Note in this case the encryption keys are stored outside of the zone and aren't managed by or visible to the zone users (even root).

# pktool genkey keystore=file keytype=aes keylen=128 outkey=/zones/key
# zfs create -o encryption=on -o keysource=raw,file:///zones/key rpool/zones/ltz
# zonecfg -z ltz 'create ; set zonepath=/zones/ltz'
# zoneadm -z ltz install
zoneadm -z ltz install
/zones/ltz2 must not be group readable.
/zones/ltz2 must not be group executable.
/zones/ltz2 must not be world readable.
/zones/ltz2 must not be world executable.
changing zonepath permissions to 0700.
Progress being logged to /var/log/zones/zoneadm.20111018T123039Z.ltz2.install
       Image: Preparing at /zones/ltz/root.

 Install Log: /system/volatile/install.4194/install_log
 AI Manifest: /tmp/manifest.xml.14a4hi
  SC Profile: /usr/share/auto_install/sc_profiles/enable_sci.xml
    Zonename: ltz2
Installation: Starting ...

              Creating IPS image
              Installing packages from:
DOWNLOAD                                  PKGS       FILES    XFER (MB)
Completed                              167/167 32062/32062  175.8/175.8

PHASE                                        ACTIONS
Install Phase                            44311/44311 

PHASE                                          ITEMS
Package State Update Phase                   167/167 
Image State Update Phase                         2/2 
Installation: Succeeded

        Note: Man pages can be obtained by installing pkg:/system/manual


        Done: Installation completed in 230.518 seconds.

  Next Steps: Boot the zone, then log into the zone console (zlogin -C)

              to complete the configuration process.

Log saved in non-global zone as /zones/ltz/root/var/log/zones/zoneadm.20111018T123039Z.ltz.install

Note the first four lines zoneadm ensured that the zonepath had the correct secure permissions but was otherwise perfectly happy with our pre-created encrypted ZFS dataset.

So at this point we just boot the zone and connect to the console and finish of the system configuration (since I didn't supply a manifest the system will be waiting to be told its name and network config.  Once that is done we can login and have a look at what local ZFS filesystems we have:

# zlogin ltz
# zfs list
rpool                    423M   447G    33K  /rpool
rpool/ROOT               423M   447G    33K  legacy
rpool/ROOT/solaris       423M   447G   358M  /
rpool/ROOT/solaris/var  61.5M   447G  59.6M  /var
rpool/export              68K   447G    35K  /export
rpool/export/home         33K   447G    33K  /export/home

Notice that we have separate datasets for / and /var as well as /export and /export/home. These will all be encrypted because rpool inside this zone is really the ZFS dataset that is underneath /zones/ltz in the global zone, so lets look and check:

# zfs get encryption,keysource rpool/ROOT/solaris
NAME                PROPERTY    VALUE                  SOURCE
rpool/ROOT/solaris  encryption  on                     inherited from $globalzone
rpool/ROOT/solaris  keysource   raw,file:///zones/key  inherited from $globalzone

Notice that the source is that we inherited this from the globalzone. That has dealt with our on disk protection without needing the admin users inside the zone to do anything additional.  

It also worth now pointing out a third new feature, note that the ZFS dataset names in side the zone look just like they do in the global zone, rpool/ROOT/solaris. This is very different to what we had in Solaris 10 where the zone saw parts of the ZFS namespace that were only applicable in the global zone.  This namespace virtualisation provide both additional security and makes p2v and v2v transitions of Zones much easier. Note only have we virtualised the dataset names but we have hidden any global zone paths when they appear in the property source, we only know that this was set my a global zone admin.

Now lets move on to the Immutable Zones part. In the zone configuration, which is held in the global zone only, we specify what 'file-mac-profile' we want.

# zonecfg -z ltz 'set file-mac-profile=fixed-configuration'
# zoneadm -z ltz reboot

Now lets 'do some damage':

root@ltz:~# touch /etc/foo
touch: cannot create /etc/foo: Read-only file system
root@ltz:~# touch /var/tmp/foo
root@ltz:~# touch /tmp/foo
root@ltz:~# pkg install emacs

pkg install: Could not complete the operation on /var/pkg/lock: read-only filesystem.
root@ltz:~# rm /usr/bin/vi
rm: /usr/bin/vi not removed: Read-only file system

root@ltz:~# useradd alice

UX: useradd: ERROR: Cannot update system - login cannot be created.

So we can't create users or remove binaries or add new ones out side of /var/tmp and /tmp, maybe we can disable SMF services permanently:

root@ltz:~# svcadm disable ssh
root@ltz:~# svcs ssh
disabled 13:21:37 svc:/network/ssh:default

Finally 'some damage' - but lets reboot...

root@ltz:~# svcs ssh
STATE          STIME    FMRI
online         13:23:19 svc:/network/ssh:default

The service restarted again on reboot; but we said disable it permanently.  What happened here was that it got disabled in the running SMF but because we couldn't persist the changes back to the on disk SMF database its permanent state didn't change and it came back on line after a reboot.

Thats nice but we need to maintain the Zone still and ensure it gets updated with security fixes so how do we write to it ?

Only from the global zone is it possible to transition the zone into a 'read-write' state.  We do this by passing '-w' or '-W' to the zoneadm boot command.  Note that this is an argument interperted by zoneadmd in the global zone and is not interpreted inside the zone at all, there is thus no way for a privileged user inside the zone to request that we reboot read-write by passing '-w' as an argument to reboot(1M).

# zoneadm -z ltz boot -w
# zlogin -C ltz
[NOTICE: Read-only zone rebooting read-write]

In this case we will get a login prompt and we can login and do everything to the zone we normally could - it is just as if 'file-mac-profile' hadn't been set. When packages are added or updated for a zone with a file-mac-profile it transiently reboots read-write (this can be forced manually with '-W') automatically; this is so any package self-assembly can be done to do config file upgrades etc, we would see this on the console:

[NOTICE: This read-only system transiently booted read/write]
[NOTICE: Now that self assembly has been completed, the system is rebooting]
[NOTICE: Zone rebooting]

At this point the zone is back to being protected just like it was above.

Two very simple to use new features in Solaris 11 that can be used separately or together to give us protection of the zones environment both on disk and at runtime.

My 11 favourite Solaris 11 features

  1. ZFS on disk encryption: zfs create -o encryption=on [ With pam_zfs_key PAM module for per-user key management]
  2. Immutable Zones: zonecfg -z myzone set file-mac-profile=fixed-configuration
  3. New package system - with cryptographically signed packages [ pkg(5) ] and multiple signature support
  4. Root as a role by default & authentication with user password with authentication cacheing [pam_tty_tickets ]
  5. Network virtualisation dladm(1M) & bandwidth control flowadm(1M)
  6. Automatic VNICs for Zones - one line zone creation: zonecfg -z myzone 'create ; set zonepath=/zones/myzone'
  7. IPfilter SMF integration - per service firewall rules
  8. New basic privileges: file_read/file_write/net_access
  9. Default root shell is bash (I'd personally prefer zsh but bash is good enough)
  10. 'man -k' works by default
  11. sudo with Solaris Audit support and priv_exec removal for NOEXEC

Monday Nov 07, 2011

Solaris 11 launch: Wednesday 9th November

I'm going to be in New York with some of the other senior Solaris engineers as part of the Solaris 11 launch on Wednesday 9th November.  I've got a few technical blog entries stacked up that I'll be pushing out over the week on the following topics:

  1. Completely disabling root logins on Solaris 11
  2. User home directory encryption with ZFS & PAM
  3. Password caching for Solaris su
  4. Immutable Zones on Encrypted ZFS
I've got some other topics I'm working on as well but they may end up being published in other forms, such as on OTN.



« April 2014