Monday Feb 11, 2013

Serial Console with VirtualBox on Solaris host

First make sure you have nc(1) available it is in the pkg:/network/netcat package.

Then configure COM1 serial port in the VM settings as a pipe.  Tell VirtualBox the name you want for the pipe and get it to create it.

You can also set up the serial port from the CLI using the VBoxManage command, here my VM is called "Solaris 11.1 Text Only".

$ VBoxManage modifyvm "Solaris 11.1 Text Only" --uart1 0x3F8 4 --uartmode1 server /tmp/solaris-11.1-console.pipe

 

Start up the VM and in a terminal window run nc and the ttya output of the VM will appear in the terminal window.

$ nc -U /tmp/solaris-11.1-console.pipe
SunOS Release 5.11 Version 11.1 64-bit
Copyright (c) 1983, 2012, Oracle and/or its affiliates. All rights reserved.


Thursday Jan 24, 2013

Compliance reporting with SCAP

In Solaris 11.1 we added the early stages of our (security) Compliance framework.  We have (like some other OS vendors) selected to use the SCAP (Security Content Automation Protocol) standard from NIST.  There are a number of different parts to SCAP but for Compliance reporting one of the important parts is the OVAL (Open Vulnerability Assesment Language) standard.  This is what allows us to write a checkable security policy and verify it against running systems.

The Solaris 11.1 repository includes the OpenSCAP tool that allows us to generate reports written in the OVAL language (as well as other things but I'm only focusing on OVAL for now).

OVAL is expressed in XML with a number of generic and OS/application specific schema.  Over time we expect to deliver various sample security policies with Solaris to help customers with Compliance reporting in various industries (eg, PCI-DSS, DISA-STIG, HIPAA).

The XML in the OVAL langauge is passed to the OpenSCAP tool for evaluation, it produces either a simple text report of which checks passed and which failed or an XML results file and an optional HTML rendered report.

Lets look at a simple example of an policy written in OVAL.  This contains just one check, that we have configured the FTP server on Solaris to display a banner.  We do this in Solaris 11 by updating /etc/proftpd.conf to add the "DisplayConnect /etc/issue" line - which is not there by default.   So in a default Solaris 11.1 system we should get a "fail" from this policy.

The OVAL for this check was generated by a tool called "Enhanced SCAP Editor (eSCAPe)" which is not included in Solaris.  It could well have been hand edited in your text editor of choice. In a later blog posting I'll attempt to explain more of the OVAL language and give some more examples, including some Solaris specific ones but for now here is the raw XML:

<?xml version="1.0" encoding="UTF-8"?>
<oval_definitions xmlns="http://oval.mitre.org/XMLSchema/oval-definitions-5" 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
xmlns:oval="http://oval.mitre.org/XMLSchema/oval-common-5" 
xmlns:oval-def="http://oval.mitre.org/XMLSchema/oval-definitions-5" 
xmlns:independent-def="http://oval.mitre.org/XMLSchema/oval-definitions-5#independent" 
xsi:schemaLocation="http://oval.mitre.org/XMLSchema/oval-definitions-5 
   oval-definitions-schema.xsd http://oval.mitre.org/XMLSchema/oval-definitions-5#independent 
   independent-definitions-schema.xsd http://oval.mitre.org/XMLSchema/oval-common-5 oval-common-schema.xsd">

  <generator>
    <oval:product_name>Enhanced SCAP Editor</oval:product_name>
    <oval:product_version>0.0.11</oval:product_version>
    <oval:schema_version>5.8</oval:schema_version>
    <oval:timestamp>2012-10-11T10:33:25</oval:timestamp>
  </generator>
  <!--generated.oval.base.identifier=com.oracle.solaris11-->
  <definitions>
    <definition id="oval:com.oracle.solaris11:def:840" version="1" class="compliance">
      <metadata>
        <title>Enable a Warning Banner for the FTP Service</title>
        <affected family="unix">
          <platform>Oracle Solaris 11</platform>
        </affected>
        <description>/etc/proftpd.conf contains "DisplayConnect /etc/issue"</description>
      </metadata>
      <criteria operator="AND" negate="false" comment="Single test">
        <criterion comment="/etc/proftpd.conf contains &quot;DisplayConnect /etc/issue&quot;" 
          test_ref="oval:com.oracle.solaris11:tst:8400" negate="false"/>
      </criteria>
    </definition>
  </definitions>
  <tests>
    <textfilecontent54_test 
        xmlns="http://oval.mitre.org/XMLSchema/oval-definitions-5#independent" 
        id="oval:com.oracle.solaris11:tst:8400" version="1" check="all" 
        comment="/etc/proftpd.conf contains &quot;DisplayConnect /etc/issue&quot;" 
        check_existence="all_exist">
      <object object_ref="oval:com.oracle.solaris11:obj:8400"/>
    </textfilecontent54_test>
  </tests>
  <objects>
    <textfilecontent54_object 
        xmlns="http://oval.mitre.org/XMLSchema/oval-definitions-5#independent" 
        id="oval:com.oracle.solaris11:obj:8400" version="1" 
        comment="/etc/proftpd.conf contains &quot;DisplayConnect /etc/issue&quot;">
      <path datatype="string" operation="equals">/etc</path>
      <filename datatype="string" operation="equals">proftpd.conf</filename>
      <pattern datatype="string" 
         operation="pattern match">^DisplayConnect\s/etc/issue\s$</pattern>
      <instance datatype="int" operation="greater than or equal">1</instance>
    </textfilecontent54_object>
  </objects>
</oval_definitions>

We can evaluate this policy on a given host by using OpenSCAP like this:

$ oscap oval eval ftp-banner.xml 
Definition oval:com.oracle.solaris11:def:840: false
Evaluation done.

As you can see we got the expected failure of the test, but that output isn't very useful, lets instead generate some html output:

$ oscap oval eval --results results.xml --report report.html ftp-banner.xml
Definition oval:com.oracle.solaris11:def:840: false
Evaluation done.
OVAL Results are exported correctly.

Now we have a report.html file which looks like a bit like this:

OVAL Results Generator Information
Schema Version Product Name Product Version Date Time
5.8  cpe:/a:open-scap:oscap 
2013-01-24 14:18:55 
OVAL Definition Generator Information
Schema Version Product Name Product Version Date Time
5.8  Enhanced SCAP Editor  0.0.11  2012-10-11 10:33:25 

System Information
Host Name braveheart 
Operating System SunOS 
Operating System Version 11.1 
Architecture i86pc 
Interfaces
Interface Name net0 
IP Address 192.168.1.1 
MAC Address aa:bb:cc:dd:ee:ff 
OVAL System Characteristics Generator Information
Schema Version Product Name Product Version Date Time
5.8  cpe:/a:open-scap:oscap 
2013-01-24 14:18:55 
Oval Definition Results



 True  



 False  



 Error  



 Unknown  



 Not Applicable  



 Not Evaluated  
OVAL ID Result Class Reference ID Title
oval:com.oracle.solaris11:def:841 true compliance
Enable a Warning Banner for the SSH Service 
oval:com.oracle.solaris11:def:840 false compliance
Enable a Warning Banner for the FTP Service 

As you probably noticed write away the report doesn't match the OVAL I gave above because the report is actually from a very slightly larger OVAL file which checks the banner exists for both SSH and FTP.  I did this purely to cut down on the amount of raw XML above but also so the report would show both a success and failure case.


Tuesday Jan 08, 2013

Solaris 11.1 FIPS 140-2 Evaluation status

Solaris 11.1 has just recently been granted CAVP certificates by NIST, this is the first formal step in getting a FIPS 140-2 certification.  It will be some months yet before we expect to see the results of the CMVP (FIPS 140-2) evaluation.  There are multiple certificates for each algorithm to cover the cases of SPARC and x86 with and with out hardware acceleration from the CPU.

References:
AES 2310, 2309, 2308
3DES 1457, 1456, 1455
DSA 727, 726
RSA 1193, 1192, 1191
ECDSA 375, 375, 374
SHS (SHA1, 224, 256, 384, 512) 1994, 1993, 1992
RNG 1153, 1152, 1151, 1150
HMAC 1424, 1423, 1422




Monday Dec 03, 2012

Using Solaris pkg to list all setuid or setgid programs

$ pkg contents -a mode=4??? -a mode=2??? -t file -o pkg.name,path,mode

We can also add a package name on the end to restrict it to just that single package eg:

 
  

$ pkg contents -a mode=4??? -a mode=2??? -t file -o pkg.name,path,mode core-os

PKG.NAME       PATH                   MODE
system/core-os usr/bin/amd64/newtask  4555
system/core-os usr/bin/amd64/uptime   4555
system/core-os usr/bin/at             4755
system/core-os usr/bin/atq            4755
system/core-os usr/bin/atrm           4755
system/core-os usr/bin/crontab        4555
system/core-os usr/bin/mail           2511
system/core-os usr/bin/mailx          2511
system/core-os usr/bin/newgrp         4755
system/core-os usr/bin/pfedit         4755
system/core-os usr/bin/su             4555
system/core-os usr/bin/tip            4511
system/core-os usr/bin/write          2555
system/core-os usr/lib/utmp_update    4555
system/core-os usr/sbin/amd64/prtconf 2555
system/core-os usr/sbin/amd64/swap    2555
system/core-os usr/sbin/amd64/sysdef  2555
system/core-os usr/sbin/amd64/whodo   4555
system/core-os usr/sbin/prtdiag       2755
system/core-os usr/sbin/quota         4555
system/core-os usr/sbin/wall          2555


Tuesday Oct 30, 2012

New ZFS Encryption features in Solaris 11.1

Solaris 11.1 brings a few small but significant improvements to ZFS dataset encryption.  There is a new readonly property 'keychangedate' that shows that date and time of the last wrapping key change (basically the last time 'zfs key -c' was run on the dataset), this is similar to the 'rekeydate' property that shows the last time we added a new data encryption key.

$ zfs get creation,keychangedate,rekeydate rpool/export/home/bob
NAME                   PROPERTY       VALUE                  SOURCE
rpool/export/home/bob  creation       Mon Mar 21 11:05 2011  -
rpool/export/home/bob  keychangedate  Fri Oct 26 11:50 2012  local
rpool/export/home/bob  rekeydate      Tue Oct 30  9:53 2012  local

The above example shows that we have changed both the wrapping key and added new data encryption keys since the filesystem was initially created.  If we haven't changed a wrapping key then it will be the same as the creation date.  It should be obvious but for filesystems that were created prior to Solaris 11.1 we don't have this data so it will be displayed as '-' instead.

Another change that I made was to relax the restriction that the size of the wrapping key needed to match the size of the data encryption key (ie the size given in the encryption property).  In Solaris 11 Express and Solaris 11 if you set encryption=aes-256-ccm we required that the wrapping key be 256 bits in length.  This restriction was unnecessary and made it impossible to select encryption property values with key lengths 128 and 192 when the wrapping key was stored in the Oracle Key Manager.  This is because currently the Oracle Key Manager stores AES 256 bit keys only.  Now with Solaris 11.1 this restriciton has been removed.

There is still one case were the wrapping key size and data encryption key size will always match and that is where they keysource property sets the format to be 'passphrase', since this is a key generated internally to libzfs and to preseve compatibility on upgrade from older releases the code will always generate a wrapping key (using PKCS#5 PBKDF2 as before) that matches the key length size of the encryption property.

The pam_zfs_key module has been updated so that it allows you to specify encryption=off.

There were also some bugs fixed including not attempting to load keys for datasets that are delegated to zones and some other fixes to error paths to ensure that we could support Zones On Shared Storage where all the datasets in the ZFS pool were encrypted that I discussed in my previous blog entry.

If there are features you would like to see for ZFS encryption please let me know (direct email or comments on this blog are fine, or if you have a support contract having your support rep log an enhancement request).




  

  


Monday Oct 29, 2012

Solaris 11.1: Encrypted Immutable Zones on (ZFS) Shared Storage

Solaris 11 brought both ZFS encryption and the Immutable Zones feature and I've talked about the combination in the past.  Solaris 11.1 adds a fully supported method of storing zones in their own ZFS using shared storage so lets update things a little and put all three parts together.

When using an iSCSI (or other supported shared storage target) for a Zone we can either let the Zones framework setup the ZFS pool or we can do it manually before hand and tell the Zones framework to use the one we made earlier.  To enable encryption we have to take the second path so that we can setup the pool with encryption before we start to install the zones on it.

We start by configuring the zone and specifying an rootzpool resource:

# zonecfg -z eizoss
Use 'create' to begin configuring a new zone.
zonecfg:eizoss> create
create: Using system default template 'SYSdefault'
zonecfg:eizoss> set zonepath=/zones/eizoss
zonecfg:eizoss> set file-mac-profile=fixed-configuration
zonecfg:eizoss> add rootzpool
zonecfg:eizoss:rootzpool> add storage \
  iscsi://my7120.example.com/luname.naa.600144f09acaacd20000508e64a70001
zonecfg:eizoss:rootzpool> end
zonecfg:eizoss> verify
zonecfg:eizoss> commit
zonecfg:eizoss> 

Now lets create the pool and specify encryption:

# suriadm map \
   iscsi://my7120.example.com/luname.naa.600144f09acaacd20000508e64a70001
PROPERTY	VALUE
mapped-dev	/dev/dsk/c10t600144F09ACAACD20000508E64A70001d0
# echo "zfscrypto" > /zones/p
# zpool create -O encryption=on -O keysource=passphrase,file:///zones/p eizoss \
   /dev/dsk/c10t600144F09ACAACD20000508E64A70001d0
# zpool export eizoss

Note that the keysource example above is just for this example, realistically you should probably use an Oracle Key Manager or some other better keystorage, but that isn't the purpose of this example.  Note however that it does need to be one of file:// https:// pkcs11: and not prompt for the key location.  Also note that we exported the newly created pool.  The name we used here doesn't actually mater because it will get set properly on import anyway. So lets go ahead and do our install:

zoneadm -z eizoss install -x force-zpool-import
Configured zone storage resource(s) from:
    iscsi://my7120.example.com/luname.naa.600144f09acaacd20000508e64a70001
Imported zone zpool: eizoss_rpool
Progress being logged to /var/log/zones/zoneadm.20121029T115231Z.eizoss.install
    Image: Preparing at /zones/eizoss/root.

 AI Manifest: /tmp/manifest.xml.ujaq54
  SC Profile: /usr/share/auto_install/sc_profiles/enable_sci.xml
    Zonename: eizoss
Installation: Starting ...

              Creating IPS image
Startup linked: 1/1 done
              Installing packages from:
                  solaris
                      origin:  http://pkg.oracle.com/solaris/release/
              Please review the licenses for the following packages post-install:
                consolidation/osnet/osnet-incorporation  (automatically accepted,
                                                          not displayed)
              Package licenses may be viewed using the command:
                pkg info --license <pkg_fmri>
DOWNLOAD                                PKGS         FILES    XFER (MB)   SPEED
Completed                            187/187   33575/33575  227.0/227.0  384k/s

PHASE                                          ITEMS
Installing new actions                   47449/47449
Updating package state database                 Done 
Updating image state                            Done 
Creating fast lookup database                   Done 
Installation: Succeeded

         Note: Man pages can be obtained by installing pkg:/system/manual

 done.

        Done: Installation completed in 929.606 seconds.


  Next Steps: Boot the zone, then log into the zone console (zlogin -C)

              to complete the configuration process.

Log saved in non-global zone as /zones/eizoss/root/var/log/zones/zoneadm.20121029T115231Z.eizoss.install

That was really all we had to do, when the install is done boot it up as normal.

The zone administrator has no direct access to the ZFS wrapping keys used for the encrypted pool zone is stored on.  Due to how inheritance works in ZFS he can still create new encrypted datasets that use those wrapping keys (without them ever being inside a process in the zone) or he can create encrypted datasets inside the zone that use keys of his own choosing, the output below shows the two cases:

rpool is inheriting the key material from the global zone (note we can see the value of the keysource property but we don't use it inside the zone nor does that path need to be (or is) accessible inside the zone). Whereas rpool/export/home/bob has set keysource locally.

 
  

# zfs get encryption,keysource rpool rpool/export/home/bob NAME PROPERTY VALUE SOURCE rpool encryption on inherited from $globalzone rpool keysource passphrase,file:///zones/p inherited from $globalzone rpool/export/home/bob encryption on local rpool/export/home/bob keysource passphrase,prompt local

 

 

Thursday Sep 13, 2012

To encryption=on or encryption=off a simple ZFS Crypto demo

I've just been asked twice this week how I would demonstrate ZFS encryption really is encrypting the data on disk.  It needs to be really simple and the target isn't forensics or cryptanalysis just a quick demo to show the before and after.

I usually do this small demo using a pool based on files so I can run strings(1) on the "disks" that make up the pool. The demo will work with real disks too but it will take a lot longer (how much longer depends on the size of your disks).  The file hamlet.txt is this one from gutenberg.org

# mkfile 64m /tmp/pool1_file
# zpool create clear_pool /tmp/pool1_file
# cp hamlet.txt /clear_pool
# grep -i hamlet /clear_pool/hamlet.txt | wc -l

Note the number of times hamlet appears

# zpool export clear_pool
# strings /tmp/pool1_file | grep -i hamlet | wc -l

Note the number of times hamlet appears on disk - it is 2 more because the file is called hamlet.txt and file names are in the clear as well and we keep at least two copies of metadata.

Now lets encrypt the file systems in the pool.
Note you MUST use a new pool file don't reuse the one from above.

# mkfile 64m /tmp/pool2_file
# zpool create -O encryption=on enc_pool /tmp/pool2_file
Enter passphrase for 'enc_pool': 
Enter again: 
# cp hamlet.txt /enc_pool
# grep -i hamlet /enc_pool/hamlet.txt | wc -l

Note the number of times hamlet appears is the same as before

# zpool export enc_pool
# strings /tmp/pool2_file | grep -i hamlet | wc -l

Note the word hamlet doesn't appear at all!

As a said above this isn't indended as "proof" that ZFS does encryption properly just as a quick to do demo.

Wednesday Jul 04, 2012

Delegation of Solaris Zone Administration

In Solaris 11 'Zone Delegation' is a built in feature. The Zones system now uses fine grained RBAC authorisations to allow delegation of management of distinct zones, rather than all zones which is what the 'Zone Management' RBAC profile did in Solaris 10.

The data for this can be stored with the Zone or you could also create RBAC profiles (that can even be stored in NIS or LDAP) for granting access to specific lists of Zones to administrators.

For example lets say we have zones named zoneA through zoneF and we have three admins alice, bob, carl.  We want to grant a subset of the zone management to each of them.

We could do that either by adding the admin resource to the appropriate zones via zonecfg(1M) or we could do something like this with RBAC data directly:

First lets look at an example of storing the data with the zone.

# zonecfg -z zoneA
zonecfg:zoneA> add admin
zonecfg:zoneA> set user=alice
zonecfg:zoneA> set auths=manage
zonecfg:zoneA> end
zonecfg:zoneA> commit
zonecfg:zoneA> exit

Now lets look at the alternate method of storing this directly in the RBAC database, but we will show all our admins and zones for this example:

# usermod -P +'Zone Management' -A +solaris.zone.manage/zoneA alice

# usermod -A +solaris.zone.login/zoneB alice


# usermod -P +'Zone Management' -A +solaris.zone.manage/zoneB bob
# usermod -A +solaris.zone.manage/zoneC bob


# usermod -P +'Zone Management' -A +solaris.zone.manage/zoneC carl
# usermod -A +solaris.zone.manage/zoneD carl
# usermod -A +solaris.zone.manage/zoneE carl
# usermod -A +solaris.zone.manage/zoneF carl

In the above alice can only manage zoneA, bob can manage zoneB and zoneC and carl can manage zoneC through zoneF.  The user alice can also login on the console to zoneB but she can't do the operations that require the solaris.zone.manage authorisation on it.

Or if you have a large number of zones and/or admins or you just want to provide a layer of abstraction you can collect the authorisation lists into an RBAC profile and grant that to the admins, for example lets great an RBAC profile for the things that alice and carl can do.

# profiles -p 'Zone Group 1'
profiles:Zone Group 1> set desc="Zone Group 1"
profiles:Zone Group 1> add profile="Zone Management"
profiles:Zone Group 1> add auths=solaris.zone.manage/zoneA
profiles:Zone Group 1> add auths=solaris.zone.login/zoneB
profiles:Zone Group 1> commit
profiles:Zone Group 1> exit
# profiles -p 'Zone Group 3'
profiles:Zone Group 1> set desc="Zone Group 3"
profiles:Zone Group 1> add profile="Zone Management"
profiles:Zone Group 1> add auths=solaris.zone.manage/zoneD
profiles:Zone Group 1> add auths=solaris.zone.manage/zoneE
profiles:Zone Group 1> add auths=solaris.zone.manage/zoneF
profiles:Zone Group 1> commit
profiles:Zone Group 1> exit


Now instead of granting carl  and aliace the 'Zone Management' profile and the authorisations directly we can just give them the appropriate profile.

# usermod -P +'Zone Group 3' carl

# usermod -P +'Zone Group 1' alice


If we wanted to store the profile data and the profiles granted to the users in LDAP just add '-S ldap' to the profiles and usermod commands.

For a documentation overview see the description of the "admin" resource in zonecfg(1M), profiles(1) and usermod(1M)

Tuesday May 01, 2012

Podcast: Immutable Zones in Oracle Solaris 11

In this episode of the "Oracle Solaris: In a Class By Itself" podcast series, the focus is a bit more technical. I was interviewed by host Charlie Boyle, Senior Director of Solaris Product Marketing. We talked about a new feature in Oracle Solaris 11: immutable zones. Those are read-only root zones for highly secure deployment scenarios.

See also my previous blog post on Enctypted Immutable Zones.

Wednesday Feb 29, 2012

Solaris 11 has the security solution Linus wants for Desktop Linux

Recently Linus Torvalds was venting (his words!) about the frustrating requirement to keep giving his root password for common desktop tasks such as connecting to a wifi network or configuring printers.

Well I'm very pleased to say that the Solaris 11 desktop doesn't have this problem thanks to our RBAC system and how it is used including how it is tightly integrated into the desktop.

One of the new RBAC features in Solaris 11 is location context RBAC profiles, by default we grant the user on the system console (ie the one on the laptop or workstation locally at the physical keyboard/screen) the "Console User" profile.  Which on a default install has the necessary authorisations and execution profiles to do things like joining a wireless network, changing CPU power management, and using removal media.   The user created at initial install time also has the much more powerful "System Administrator" profile granted to them so they can do even more without being required to give a password for root (they also have access to the root role and the ability to use sudo).

Authorisations in Solaris RBAC (which dates back in main stream Solaris to Solaris 8 and even further 17+ years in Trusted Solaris) are checked by privileged programs and the whole point is so you don't have to reauthenticate.  SMF is a very heavy user of RBAC authorisations.  In the case of things like joining a wireless network it is privileged daemons that are checking the authorisations of the clients connecting to them (usually over a door)

In addition to that GNOME in Solaris 11 has been explicitly integrated with Solaris RBAC as well, any GNOME menu entry that needs to run with elevated privilege will be exectuted via Solaris RBAC mechanisms.  The panel works out the least intrusive way to get the program running for you.  For example if I select "Wireshark" from the GNOME panel menu it just starts - I don't get prompted for any root password - but it starts with the necessary privileges because GNOME on Solaris 11 knows that I have the "Network Management" RBAC profile which allows running /usr/sbin/wireshark with the net_rawaccess privilege.   If I didn't have "Network Management" directly but I had an RBAC role that had it then GNOME would use gksu to assume the role (which might be root) and in which case I would have been prompted for the role password.  If you are using roleauth=user that password is yours and if you are using pam_tty_tickets you won't keep getting prompted.

GNOME can even go further and not even present menu entries to users who don't have granted to them any RBAC profile that allows running those programs - this is useful in a large multi user system like a Sun Ray deployment.

If you want to do it the "old way" and use the CLI and/or give a root password for every "mundane" little thing, you can still do that too if you really want to.

So maybe Linus could try Solaris 11 desktop ;-)

Monday Feb 20, 2012

Solaris 11 Common Criteria Evaluation

Oracle Solaris 11 is now "In Evaluation" for Common Criteria at EAL4+.  The protection profile is OSPP with the following extended packages: AM - Advanced Management  EIA - Extended Identification and Authentication, LS - Label Security, VIRT - Virtualization.  For information on other Oracle products that are evaluated under Common Criteria or FIPS 140 please see the general Oracle Security Evalutions page.

Please email seceval_us@oracle.com for all inquiries regarding Oracle security evaluations, I can't answer questions about the content of the evaluation on this blog or directly by email to me.

Friday Feb 03, 2012

What Free/Open Source software is Solaris 11 still missing

Note this is not a commitment from Oracle to deliver anything as a result of your answers, nor is it an official survey of any kind.

Okay first my dirty little secret... my family home desktop machine runs Windows 7.  Earlier this week I had a need to check the MD5 or SHA256 checksum on an iso image I'd downloaded.  On Solaris I'd just run 'digest -a sha256' or sha256sum on Solaris or any Linux distro.  But on Windows 7 the best I could come up with was code it up in Java myself or install the GNU versions via Cygwin.

So that got me thinking, the Solaris 11 repository has a lot more "upstream" Free/Open Source tools and frameworks than any other release of Solaris ever had.  We have Python (which is really a core part of Solaris 11 now), Ruby loads of the GNU runtime and development toolchains and much much more.   However many common Linux distributions still have more than we do but some of that isn't target at server use cases.

So what Free/Open Source software is Solaris 11 still missing that you use to run your business on your Solaris servers?

Even if you don't have Solaris 11 installed you can quickly search for packags at http://pkg.oracle.com/solaris/release

Please add details in the comments.

Again note this is not a commitment from Oracle to deliver anything as a result of your answers, nor is it an official survey of any kind, just my curiosity.  I will of course log the relevant bugs for viable things if any come up.

Update 1: Thanks for all the submissions so far, some great suggestions in there - keep them coming and don't worry about looking for duplicates in others comments (in fact I'd rather things were listed my multiple people since it shows more interest in a given component).

Update 2: comments are moderated (site requirement), submitting multiple times unfortunately sometimes results in you being told your comment is spam but I still see it and will approve it. Thanks for your patience.


Tuesday Dec 20, 2011

How low can we go ? (Minimised install of Solaris 11)

I wondered how little we can actually install as a starting point for building a minimised system. The new IPS package system makes this much easier and makes it work in a supportable way without all the pit falls of patches and packages we had previously.

For Solaris 11 I believe the currently smallest configuration we recommend is the solaris-small-server group package.

Note the following is not intended to imply this is a supported configuration and is illustrative only of a short amount of investigative work.

First lets look at a zone (it is easier since there are no driver issues to deal with): I discovered it is possible to get a 'working' zone by choosing a single package to install in the zone AI manifest and that package is: pkg:/package/pkg

That resulted in 175M being reported as being transferred by pkg. Which results in 255M on disk of which about 55M is /var/pkg. I had 140 'real' packages (ie excluding the incorporations). We have 71 online SMF services (nothing in maintenance) with 96 known SMF services. Around 23 processes are running (excluding the ps I used to check this and the shell I was logged in on).

I have discovered some potential packaging that could result in this being a little bit smaller but not much unless a break up of pkg:/system/core-os was done.

Now onto the bare metal install case. This was a little harder and I've not gotten it to where I wanted yet.

Ignoring drivers the only thing I needed on an x86 system was: pkg:/package/pkg and pkg:/system/boot/grub

Which is good and not really different from the zones case. However that won't produce a bootable system - even though it produces one that will install!

To get it to boot I took the list of all the network and storage drivers from the solaris-small-server group package.  I removed all the wlan drivers and also any drivers I knew to be SPARC only.   My list of packages in the AI manifest had 113 driver packages in it. That got me a bootable system, though not one minimized with respect to drivers.

We have a few more processes in the global zone (again ignoring the ps and my shell) this time I counted 32.  This came from 89 online services. Again ignoring the incorporation packages  I had 161 packages installed of which 73 were in the pkg:/driver namespace.

The disk space footprint is much bigger at total of 730M - but remember I've likely installed drivers that I might not need. This time /var/pkg is 174M of that.


Tuesday Nov 22, 2011

HOWTO Turn off SPARC T4 or Intel AES-NI crypto acceleration.

Since we released hardware crypto acceleration for SPARC T4 and Intel AES-NI support we have had a common question come up: 'How do I test without the hardware crypto acceleration?'.

Initially this came up just for development use so developers can do unit testing on a machine that has hardware offload but still cover the code paths for a machine that doesn't (our integration and release testing would run on all supported types of hardware anyway).  I've also seen it asked in a customer context too so that we can show that there is a performance gain from the hardware crypto acceleration, (not just the fact that SPARC T4 much faster performing processor than T3) and measure what it is for their application.

With SPARC T2/T3 we could easily disable the hardware crypto offload by running 'cryptoadm disable provider=n2cp/0'.  We can't do that with SPARC T4 or with Intel AES-NI because in both of those classes of processor the encryption doesn't require a device driver instead it is unprivileged user land callable instructions.

Turns out there is away to do this by using features of the Solaris runtime loader (ld.so.1). First I need to expose a little bit of implementation detail about how the Solaris Cryptographic Framework is implemented in Solaris 11.  One of the new Solaris 11 features of the linker/loader is the ability to have a single ELF object that has multiple different implementations of the same functions that are selected at runtime based on the capabilities of the machine.  The alternate to this is having the application coded to call getisax() and make the choice itself.  We use this functionality of the linker/loader when we build the userland libraries for the Solaris Cryptographic Framework (specifically libmd.so, and the unfortunately misnamed due to historical reasons libsoftcrypto.so)

The Solaris linker/loader allows control of a lot of its functionality via environment variables, we can use that to control the version of the cryptographic functions we run.  To do this we simply export the LD_HWCAP environment variable with values that tell ld.so.1 to not select the HWCAP section matching certain features even if isainfo says they are present. 

For SPARC T4 that would be:

export LD_HWCAP="-aes -des -md5 -sha256 -sha512 -mont -mpmul" 

and for Intel systems with AES-NI support:

export LD_HWCAP="-aes"

This will work for consumers of the Solaris Cryptographic Framework that use the Solaris PKCS#11 libraries or use libmd.so interfaces directly.  It also works for the Oracle DB and Java JCE.  However does not work for the default enabled OpenSSL "t4" or "aes-ni" engines (unfortunately) because they do explicit calls to getisax() themselves rather than using multiple ELF cap sections.

However we can still use OpenSSL to demonstrate this by explicitly selecting "pkcs11" engine  using only a single process and thread. 

$ openssl speed -engine pkcs11 -evp aes-128-cbc
...
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
aes-128-cbc      54170.81k   187416.00k   489725.70k   805445.63k  1018880.00k

$ LD_HWCAP="-aes" openssl speed -engine pkcs11 -evp aes-128-cbc
...
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
aes-128-cbc      29376.37k    58328.13k    79031.55k    86738.26k    89191.77k

We can clearly see the difference this makes in the case where AES offload to the SPARC T4 was disabled. The "t4" engine is faster than the pkcs11 one because there is less overhead (again on a SPARC T4-1 using only a single process/thread - using -multi you will get even bigger numbers).

$ openssl speed -evp aes-128-cbc
...
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
aes-128-cbc      85526.61k    89298.84k    91970.30k    92662.78k    92842.67k

Yet another cool feature of the Solaris linker/loader, thanks Rod and Ali.

Note these above openssl speed output is not intended to show the actual performance of any particular benchmark just that there is a significant improvement from using hardware acceleration on SPARC T4. For cryptographic performance benchmarks see the http://blogs.oracle.com/BestPerf/ postings.

Thursday Nov 17, 2011

OpenSSL Versions in Solaris

Those of you have have installed Solaris 11 or have read some of the blogs by my colleagues will have noticed Solaris 11 includes OpenSSL 1.0.0, this is a different version to what we have in Solaris 10.  I hope the following explains why that is and how it fits with the expectations on binary compatibility between Solaris releases.

Solaris 10 was the first release where we included OpenSSL libraries and headers (part of it was actually statically linked into the SSH client/server in Solaris 9).  At time we were building and releasing Solaris 10 the current train of OpenSSL was 0.9.7. 

The OpenSSL libraries at that time were known to not always be completely API and ABI (binary) compatible between releases (some times even in the lettered patch releases) though mostly if you stuck with the documented high level APIs you would be fine.   For this reason OpenSSL was classified as a 'Volatile' interface and in Solaris 10 Volatile interfaces were not part of the default library search path which is why the OpenSSL libraries live in /usr/sfw/lib on Solaris 10.  Okay, but what does Volatile mean ?

Quoting from the attributes(5) man page description of Volatile (which was called External in older taxonomy):

         Volatile interfaces can change at any time and  for  any
         reason.

         The Volatile interface stability level allows  Sun  pro-
         ducts to quickly track a fluid, rapidly evolving specif-
         ication. In many cases, this is preferred  to  providing
         additional  stability to the interface, as it may better
         meet the expectations of the consumer.

         The most common application of this taxonomy level is to
         interfaces that are controlled by a body other than Sun,
         but unlike specifications controlled by standards bodies
         or Free or Open Source Software (FOSS) communities which
         value interface compatibility, it can  not  be  asserted
         that  an incompatible change to the interface specifica-
         tion would be exceedingly rare. It may also  be  applied
         to  FOSS  controlled  software  where  it is deemed more
         important to track the community  with  minimal  latency
         than to provide stability to our customers.

         It also common  to  apply  the  Volatile  classification
         level  to  interfaces in the process of being defined by
         trusted or  widely  accepted  organization.   These  are
         generically  referred  to  as draft standards.  An "IETF
         Internet draft"  is  a  well  understood  example  of  a
         specification under development.

         Volatile can also be applied to experimental interfaces.

         No assertion is made regarding either source  or  binary
         compatibility  of  Volatile  interfaces  between any two
         releases,  including  patches.  Applications  containing
         these  interfaces might fail to function properly in any
         future release.

Note that last paragraph!  OpenSSL is only one example of the many interfaces in Solaris that are classified as Volatile.  At the other end of the scale we have Committed (Stable in Solaris 10 terminology) interfaces, these include things like the POSIX APIs or Solaris specific APIs that we have no intention of changing in an incompatible way.  There are also Private interfaces and things we declare as Not-an-Interface (eg command output not intended for scripting against only to be read by humans).

Even if we had declared OpenSSL as a Committed/Stable interface in Solaris 10 there are allowed exceptions, again quoting from attributes(5):

         4.   An interface specification which  isn't  controlled
              by  Sun  has been changed incompatibly and the vast
              majority of interface consumers  expect  the  newer
              interface.

         5.   Not  making  the  incompatible  change   would   be
              incomprehensible  to our customers. 

In our opinion and that of our large and small customers keeping up with the OpenSSL community is important, and certainly both of the above cases apply.

Our policy for dealing with OpenSSL on Solaris 10 was to stay at 0.9.7 and add fixes for security vulnerabilities (the version string includes the CVE numbers of fixed vulnerabilities relevant to that release train).  The last release of OpenSSL 0.9.7 delivered by the upstream community was more than 4 years ago in Feb 2007.

Now lets roll forward to just before the release of Solaris 11 Express in 2010. By that point in time the current OpenSSL release was 0.9.8 with the 1.0.0 release known to be coming soon.  Two significant changes to OpenSSL were made between Solaris 10 and Solaris 11 Express.  First in Solaris 11 Express (and Solaris 11) we removed the requirement that Volatile libraries be placed in /usr/sfw/lib, that means OpenSSL is now in /usr/lib, secondly we upgraded it to the then current version stream of OpenSSL (0.9.8) as was expected by our customers.

In between Solaris 11 Express in 2010 and the release of Solaris 11 in 2011 the OpenSSL community released version 1.0.0.  This was a huge milestone for a long standing and highly respected open source project.  It would have been highly negligent of Solaris not to include OpenSSL 1.0.0e in the Solaris 11 release. It is the latest best supported and best performing version.  

 

In fact Solaris 11 isn't 'just' OpenSSL 1.0.0 but we have added our SPARC T4 engine and the AES-NI engine to support the on chip crypto acceleration. This gives us 4.3x better AES performance than OpenSSL 0.9.8 running on AIX on an IBM POWER7. We are now working with the OpenSSL community to determine how best to integrate the SPARC T4 changes into the mainline OpenSSL.  The OpenSSL 'pkcs11' engine we delivered in Solaris 10 to support the CA-6000 card and the SPARC T1/T2/T3 hardware is still included in Solaris 11.

When OpenSSL 1.0.1 and 1.1.0 come out we will asses what is best for Solaris customers. It might be upgrade or it might be parallel delivery of more than one version stream.  At this time Solaris 11 still classifies OpenSSL as a Volatile interface, it is our hope that we will be able at some point in a future release to give it a higher interface stability level.

Happy crypting! and thank-you OpenSSL community for all the work you have done that helps Solaris.

About

DarrenMoffat

Search

Categories
Archives
« March 2015
MonTueWedThuFriSatSun
      
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
     
Today