Sunday Jul 12, 2015

Solaris 11.3 beta: Changes to bundled software packages

With the release of Solaris 11.3 beta, I've gone back and made a new list of changes to the bundled software packages available in the Solaris IPS package repository, as I've done for the Solaris 11.1, Solaris 11.2 beta, and the Solaris 11.2 GA releases.

Oracle packages

Several bundled packages improve integration with other Oracle software. The Oracle Instant Client packages are now in the IPS repo for building software that connects to Oracle databases. MySQL 5.6 has also been added alongside the existing version 5.5 packages.

The Java runtime & developer kits for Java 7 & 8 were updated to new versions, while the Java 6 versions were removed as its support life winds down. The End of Feature Notices for Oracle Solaris 11 warns that Java 7 will be coming out as well in a later release.

Also updated was Oracle Hardware Management Pack (HMP), a set of tools that work with the ILOM, firmware, and other components in Sun/Oracle servers to configure low-level system options. HMP 2.2 was introduced in Solaris 11.2, and Solaris 11.3 now delivers HMP 2.3 packages.

Python packages

Solaris has long included and depended on Python 2. Solaris 11.3 adds Python 3 support for the first time, with the bundling of Python 3.4 and many module packages that work with it. Python 2.7 is still included, as is 2.6 for now, but Python 2 software in Solaris is almost completely switched over to 2.7 now, and Python 2.6 will be obsoleted soon.

A side effect of these changes was a revamping of the naming pattern for Python module packages in IPS - previously most modules delivered a set of packages following the pattern:

  • library/python-2/<module name>
  • library/python-2/<module name>-<for each Python version>
For example, there were three Mako packages, library/python-2/mako, library/python-2/mako-26, library/python-2/mako-27, where the latter two installed the modules built for the named versions of Python, and the first uses IPS conditional dependencies to install the modules for any Python versions that were installed on the system.

In extending this to provide Python 3 modules, it was decided to drop the python major version from the library/python-N prefix, leaving just the version at the end of the module name. Thus in Solaris 11.3, you'll see that the mako packages are now library/python/mako, library/python/mako-26, library/python/mako-27, and library/python/mako-34.

NVIDIA graphics driver packages

NVIDIA has been providing graphics driver packages for Solaris for almost a decade now. As new families and models of graphics cards are regularly introduced, they retire support for older generations from time to time in the new drivers. Support for these models is retained in a legacy driver, but that requires uninstalling the latest version and switching to a legacy branch. Previously that meant installing NVDIA's SVR4 package release instead of IPS, losing the ability to get updates with a simple “pkg update” command.

Now the legacy drivers are also available in IPS packages, which will continue to be updated as necessary for bug fixes and support for new Xorg releases during NVIDIA’s Support timeframes for Unix legacy GPU releases. To switch to the version 340 legacy driver on Solaris 11.3 or the later Solaris 11.2 SRU’s simply run:

  # pkg install --reject driver/graphics/nvidia driver/graphics/nvidiaR340 
and then reboot into the new BE created. For the previous version 304, change the above command to end in nvidiaR304 instead.

Other packages

There are far more changes than I've covered here - fortunately, the engineers who worked on many of these changes have written their own blog posts about them for you to check out:

One more thing... Solaris 11.2 packages

While all these are available now in the Solaris 11.3 beta, many are also available for testing and evaluation on existing Solaris 11.2 systems, when you're ready to upgrade a FOSS package, but not the rest of the OS. This is planned to be an ongoing program, so once Solaris 11.3 is officially released, the evaluation packages will keep moving forward to new versions of many packages. More details are available in a Solaris FOSS blog post and an article in the Solaris 11 OTN community space.

Not all packages are available in the evaluation program though, since some depend on OS changes not in Solaris 11.2. For instance, OpenSSH is not available for Solaris 11.2, since it depends on changes to the existing SunSSH packages that allow for the ssh package mediator to choose which ssh software to use on a given system.

Detailed list of changes

This table shows most of the changes to the bundled packages between the original Solaris 11.2.0 release, the latest Solaris 11.2 support repository update (SRU11, aka 11.2.11, released June 13, 2015), and the Solaris 11.3 beta released today. These show the versions they were released with, and not later versions that may now be available via the new FOSS Evaluation Packages for existing Solaris releases.

As with last time, some were excluded for clarity, or to reduce noise and duplication. All of the bundled packages which didn’t change the version number in their packaging info are not included, even if they had updates to fix bugs, security holes, or add support for new hardware or new features of Solaris.

PackageUpstream11. Beta
cloud/openstack OpenStack0.2013.2.30.2014.2.20.2014.2.2
cloud/openstack/cinder OpenStack0.2013.2.30.2014.2.20.2014.2.2
cloud/openstack/glance OpenStack0.2013.2.30.2014.2.20.2014.2.2
cloud/openstack/heat OpenStacknot included0.2014.2.20.2014.2.2
cloud/openstack/horizon OpenStack0.2013.2.30.2014.2.20.2014.2.2
cloud/openstack/keystone OpenStack0.2013.2.30.2014.2.20.2014.2.2
cloud/openstack/neutron OpenStack0.2013.2.30.2014.2.20.2014.2.2
cloud/openstack/nova OpenStack0.2013.2.30.2014.2.20.2014.2.2
cloud/openstack/swift OpenStack1.
communication/im/pidgin Pidgin2.
compress/pigz pigznot included2.
crypto/gnupg GnuPG2.
database/mysql-56 MySQLnot included
(MySQL 5.5 in database/mysql-56)
database/sqlite-3 SQLite3.
developer/build/ant Apache Ant1.
developer/documentation-tool/help2man GNU help2mannot includednot included1.46.1
developer/documentation-tool/xmlto xmltonot includednot included0.0.25
developer/java/jdk-6 Java1.6.0.75
(Java SE 6u75)
(Java SE 6u95)
not included
developer/java/jdk-7 Java1.7.0.65
(Java SE 7u65)
(Java SE 7u80)
(Java SE 7u80)
developer/java/jdk-8 Java1.8.0.11
(Java SE 8u11)
(Java SE 8u45)
(Java SE 8u45)
developer/test/check checknot includednot included0.9.14
developer/versioning/mercurial Mercurial SCM2.
developer/versioning/subversion Apache Subversion1.
diagnostic/nicstat nicstatnot includednot included1.95
diagnostic/tcpdump tcpdump4.
diagnostic/wireshark Wireshark1.
driver/graphics/nvidia NVIDIA0.331.38.00.346.35.00.346.35.0
driver/graphics/nvidiaR304 NVIDIAnot included0.304.125.00.304.125.0
driver/graphics/nvidiaR340 NVIDIAnot included0.340.65.00.340.65.0
file/mc GNU Midnight Commander4.
library/apr-15 Apache Portable Runtimenot includednot included1.5.1
library/c++/net6 Gobby1.
library/jansson Janssonnot includednot included2.7
library/json-c JSON-C0.90.90.12
library/libee libee0.
library/libestr libestr0.
library/libgsl GNU GSLnot includednot included1.16
library/liblogging LibLoggingnot includednot included1.0.4
library/libmicrohttpd GNU Libmicrohttpdnot includednot included0.9.37
library/libmilter Sendmail8.
library/libxml2 XML C parser2.
library/neon neon0.
library/perl-5/openscap-512 OpenSCAP1.
library/perl-5/xml-libxml CPAN: XML::LibXML2.142.142.121
was library/python-2/alembic
was library/python-2/amqp
library/python/barbicanclient OpenStacknot included3.
was library/python-2/boto
library/python/ceilometerclient OpenStack1.
library/python/cinderclient OpenStack1.
was library/python-2/cliff
library/python/django Django1.
library/python/django-pyscss django-pyscssnot included1.
was library/python-2/django_compressor
was library/python-2/django_openstack_auth
was library/python-2/eventlet
library/python/futures pythonfuturesnot included2.
library/python/glance_store OpenStacknot included0.
library/python/glanceclient OpenStack0.
was library/python-2/greenlet
library/python/heatclient OpenStack0.
library/python/iniparse iniparsenot included0.40.4
library/python/ipaddr ipaddr-pynot included2.
library/python/jinja2 Jinja2.
library/python/keystoneclient OpenStack0.
library/python/keystonemiddleware OpenStack not included1.
was library/python-2/kombu
library/python/ldappool ldappoolnot included1.01.0
was library/python-2/netaddr
was library/python-2/netifaces
library/python/networkx NetworkXnot included1.
library/python/neutronclient OpenStack2.
library/python/novaclient OpenStack2.
library/python/oauthlib OAuthLibnot included0.
library/python/openscap OpenSCAP1.
library/python/oslo.config OpenStack1.
library/python/oslo.context OpenStacknot included0.
library/python/oslo.db OpenStacknot included1.
library/python/oslo.i18n OpenStacknot included1.
library/python/oslo.messaging OpenStacknot included1.
library/python/oslo.middleware OpenStacknot included0.
library/python/oslo.serialization OpenStacknot included1.
library/python/oslo.utils OpenStacknot included1.
library/python/oslo.vmware OpenStacknot included0.
library/python/osprofiler OpenStacknot included0.
was library/python-2/pep8
PyPI: pep81.
was library/python-2/pip
library/python/posix_ipc POSIX IPC for Pythonnot included0.
was library/python-2/py
library/python/pycadf OpenStacknot included0.
was library/python-2/pyflakes
library/python/pyscss pyScssnot included1.
library/python/pysendfile pysendfilenot included2.
was library/python-2/pytest
was library/python-2/python-mysql
was library/python-2/pytz
was library/python-2/requests
library/python/retrying Retryingnot included1.
library/python/rfc3986 rfc3986not included0.
library/python/saharaclient OpenStacknot included0.
was library/python-2/setuptools
PyPI: setuptools0.
library/python/simplegeneric PyPI: simplegenericnot included0.
was library/python-2/simplejson
library/python/six PyPI: six1.
was library/python-2/sqlalchemy
was library/python-2/sqlalchemy-migrate
was library/python-2/stevedore
library/python/swiftclient OpenStack2.
library/python/taskflow OpenStacknot included0.
was library/python-2/tox
library/python/troveclient OpenStack0.
was library/python-2/virtualenv
library/python/websockify Websockify0.
library/python/wsme wsmenot included0.
library/ruby/hiera Puppetnot included1.
library/security/libassuan GnuPG2.
library/security/libksba GnuPG1.
library/security/openssl OpenSSL1.0.1.8 (1.0.1h) (1.0.1m) (1.0.1o)
library/unixodbc unixODBC2.
library/zlib zlib1.
mail/mailman GNU Mailmannot includednot included2.1.18.1
network/dns/bind ISC BIND9.
network/firewall OpenBSD PFnot includednot included5.5
network/mtr MTRnot includednot included0.86
network/openssh OpenSSHnot includednot included6.5.0.1
network/rsync rsync3.
print/filter/hplip HPLIP3.
runtime/erlang erlang15.2.317.517.5
runtime/java/jre-6 Java1.6.0.75
(Java SE 6u75)
(Java SE 6u95)
not included
runtime/java/jre-7 Java1.7.0.65
(Java SE 7u65)
(Java SE 7u80)
(Java SE 7u80)
runtime/java/jre-8 Java1.8.0.11
(Java SE 8u11)
(Java SE 8u45)
(Java SE 8u45)
runtime/python-27 Python2.
runtime/python-34 Pythonnot includednot included3.4.3
runtime/ruby-21 Rubynot included
(Ruby 1.9.3 in runtime/ruby-19)
security/compliance/openscap OpenSCAP1.
security/sudo Sudo1.
service/network/dns/bind ISC BIND9.
service/network/ftp ProFTPD1. (1.3.4c)
service/network/ntp NTP4.2.7.381 (4.2.7p381) (4.2.8p2) (4.2.8p2)
service/network/samba Samba3.
service/network/smtp/postfix Postfixnot includednot included2.11.3
service/network/smtp/sendmail Sendmail8.
shell/bash GNU bash4.
shell/watch procps-ngnot includednot included3.3.10
shell/zsh Zsh5.
system/data/hardware-registry pci.ids
system/data/timezone IANA Time Zone Data0.5.11 (2014c)0.5.11 (2015d)2015.4 (2015d)
system/font/truetype/google-droid Droid Fonts0.2010.2.240.2010.2.240.2013.6.7
system/library/freetype-2 FreeType2.
system/library/hmp-libs Oracle HMP2.
system/library/i18n/libthai libthai0.
system/library/libdatrie datrie0.
system/management/biosconfig Oracle HMP2.
system/management/facter Puppet1.
system/management/fwupdate Oracle HMP2.
system/management/fwupdate/qlogic Oracle HMP1.
system/management/hmp-snmp Oracle HMP2.
system/management/hwmgmtcli Oracle HMP2.
system/management/hwmgmtd Oracle HMP2.
system/management/ocm Oracle Configuration Manager12.
system/management/puppet Puppet3.
system/management/raidconfig Oracle HMP2.
system/management/ubiosconfig Oracle HMP2.
system/rsyslog rsyslog6.
system/test/sunvts Oracle VTS7.
terminal/tmux tmux1.
text/gnu-patch GNU Patch2.
text/groff GNU troff1.
text/less Less436436458
text/text-utilities util-linuxnot includednot included2.24.2
web/browser/firefox Mozilla Firefox17.0.1131.
web/browser/links Links1.
web/curl cURL7.
web/java-servlet/tomcat Apache Tomcat6.0.416.0.436.0.43
web/java-servlet/tomcat-8 Apache Tomcatnot includednot included8.0.21
web/novnc noVNCnot included0.50.5
web/php-53 PHP5.
web/php-56 PHPnot includednot included5.6.8
web/php-56/extension/php-suhosin-extension Suhosinnot includednot included0.9.37.1
web/php-56/extension/php-xdebug Xdebugnot includednot included2.3.2
web/server/apache-22 Apache HTTPD2.
web/server/apache-22/module/apache-jk Apache Tomcat1.
web/server/apache-22/module/apache-security ModSecurity2.
web/server/apache-22/module/apache-wsgi mod_wsgi3.
web/server/apache-24 Apache HTTPDnot includednot included2.4.12
web/server/apache-24/module/apache-dtrace Apache DTrace modulenot includednot included0.3.1
web/server/apache-24/module/apache-fcgid Apache mod_fcgidnot includednot included2.3.9
web/server/apache-24/module/apache-jk Apache Tomcatnot includednot included1.2.40
web/server/apache-24/module/apache-security ModSecuritynot includednot included2.8.0
mod_wsginot includednot included4.3.0
web/wget GNU wget1.141.161.16
x11/server/xorg/driver/xorg-input-keyboard X.Org1.
x11/server/xorg/driver/xorg-input-mouse X.Org1.
x11/server/xorg/driver/xorg-input-synaptics X.Org1.
x11/server/xorg/driver/xorg-video-ast X.Org0.
x11/server/xorg/driver/xorg-video-dummy X.Org0.
x11/server/xorg/driver/xorg-video-mga X.Org1.
x11/server/xorg/driver/xorg-video-vesa X.Org2.

Tuesday Apr 29, 2014

Solaris 11.2: Changes to bundled software packages

When Solaris 11.1 came out in October 2012, I posted about the changes to the included FOSS packages. With the publication today of Solaris 11.2 beta, I thought it would be nice to revisit this and see what’s changed in the past year and a half. This time around, I’m including some bundled packages that aren’t necessarily covered by a free software or open source license, but are of interest to Solaris users.

Removing software in updates

Last time I discussed how IPS allowed us to make a variety of changes in update releases much more easily than in the Solaris 10 package system. One of these changes is obsoleting packages, and we’ve done that in a couple rare cases in both Solaris 11.1 and 11.2 where the software is abandoned by the upstream, and we’ve decided it would be worse to keep it around, potentially broken, than to remove it on upgrade.

When we do this, notices will be posted to the End of Features for Solaris 11 web page, alongside the list of features that have been declared deprecated and may be removed in future releases. As you can see there, in Solaris 11.1 the Adobe Flash Player and tavor HCA driver packages were removed.

In Solaris 11.2, three more packages have been removed. slocate was a “secure” version of the locate utility, which wouldn’t show a user any files that they didn’t have permission to access. Unfortunately, this utility was broken by changes in the AST library, and since there is no longer an upstream for it, we decided to follow the lead of several Linux distros and moved to mlocate instead, which is added in this release.

The other two removed packages are both Xorg video drivers - the nv driver for NVIDIA graphics, and the trident driver for old Trident graphics chipsets. Most users will not notice these removals, but if you had manually created an xorg.conf file specifying one of these drivers, you may need to edit it to use the vesa driver instead.

NVIDIA had previously supported the nv open source driver and contributed updates to X.Org to support new chipsets in it, but in 2010, they announced they would no longer do so, and considered nv deprecated, recommending the use of the VESA driver for those who had no better driver to use. While we had continued to ship the nv driver in Solaris, it led to an increasing number of crashes, hangs, and other bugs for which the resolution was to remove the nv driver and use vesa instead, so we are removing it to end those issues. For systems with graphics devices new enough to be supported by the bundled nvidia closed-source driver, this will have no effect. For those with older devices, this will cause Xorg autoconfiguration to load the vesa driver instead, until and unless the user downloads & installs an appropriate NVIDIA legacy driver.

The trident driver was still in Solaris even after we dropped 32-bit support on x86, and years after Trident Microsystems exited the graphics business and sold its graphics product line to XGI, as the Sun Fire V20z server included a Trident chipset for the console video device. Unfortunately, the upstream driver has been basically unmaintained since then, and Oracle has had to apply patches to port to new Xorg releases. Meanwhile, in order to resolve bugs that caused system hangs, the trident driver was modified to not load on V20z systems, which left us shipping an unmaintained driver solely for a system that could not use it, but uses the vesa driver instead, so we decided to remove it as well.

If you had either of these Xorg driver packages installed, then when you update to 11.2, then pkg update will inform you there are release notes for these drivers, to warn you of the possibility you may need to edit your xorg.conf.

System Management Stack

The popular Puppet system for automating configuration changes across machines has been included in Solaris, and updated to support several Solaris features in both the framework and in individual configuration provideers. For instance, configuration changes made via Puppet will be recorded in the Solaris audit logs as part of a puppet session, and Puppet’s configuration file is generated from SMF properties using the new SMF stencil facilities. Providers are included that can configure IPS publishers, SMF properties, ZFS datasets, Solaris boot environments, and a variety of Solaris NIC, VNIC, and VLAN settings.

Another addition is the Oracle Hardware Management Pack (HMP), a set of tools that work with the ILOM, firmware, and other components in Sun/Oracle servers to configure low-level system options. Previously these needed to be downloaded and installed separately, now they are a simple pkg install away, and kept up to date with the rest of the OS.

A collaboration with Intel led to the integration of a Solaris port of Intel’s numatop tool for observing memory access locality across CPUs.

From the open source world, we’ve integrated several tools to allow admins and users to do multiple things at once, including the tmux terminal multiplexer, cssh tool for cluster administration via ssh, and GNU Parallel for running commands in parallel.

Developer Stack

For developers, GNU Compiler Collection (gcc) versions 4.7 & 4.8 are added alongside the previous 3.4 & 4.5 packages, and the gcc packages have been refactored to better allow installing different subsets of compilers. Other updated developer tools include Mercurial 2.8.2, GNU emacs 24.3, pylint 0.25.2, and version 7.6 of the GNU debugger, gdb. Newly added tools for developers include GNU indent, JavaScript Lint, and Python’s pep8.

The Java 8 development kit & runtime environment are both available as well. The default installation clusters will only install Java 7, but you can install the Java 8 runtime with “pkg install jre-8” or get both the runtime & development kits with “pkg install jdk-8”. The /usr/java mediated link, through which all the links in /usr/bin for the java, jar, javac, etc. commands flow will be set by default to the most recent version installed, so installing Java 8 will make that version default. You can see this via “ls -l /usr/java” reporting:

lrwxrwxrwx   1 root   root     15 Apr 23 14:01 /usr/java -> jdk/jdk1.8.0_05
or via “pkg mediator java” reporting:
java         system    1.8     system
If you want to choose a different version to be default, you can manually set the mediator to that version with “pkg set-mediator -V 1.7 java”. Of course, for many operations, you can directly access any installed java version via the full path, such as /usr/jdk/instances/jdk1.8.0/bin/java instead of relying on the /usr/bin symlinks.

One caveat to be aware of is that Java 8 for Solaris is only provided as 64-bit binaries, as all Solaris 11 and later machines are running 64-bit now. This means that any JNI modules you rely on will need to be compiled as 64-bit and any programs that try to load Java must be 64-bit. There is also no 64-bit version provided of either the Java plugin for web browsers, or the Java Webstart program for starting Java client applications from web pages.

Desktop Stack

Most of the changes in the desktop stack in this release were updates needed to fix security issues, and are mostly covered on the Oracle Third Party Vulnerability Resolution Blog.

There were some feature updates in the X Window System layers of the desktop stack though – most notably the Xorg server was upgraded from 1.12 to 1.14, and the accompanying Mesa library was upgraded to version 9.0.3, which includes support for OpenGL 3.1 and GLSL 1.40 on Intel graphics. The bundled version of NVIDIA’s graphics driver was also updated, to NVIDIA’s latest “long lived branch” - 331. For users with older graphics cards which are no longer supported in this branch, legacy branches are available from NVIDIA’s Unix driver download site.


And last, but certainly not least, especially in the number of packages added to the repository, is the addition of OpenStack support in Solaris. The Cinder Block Storage Service, Glance Image Service, Horizon Dashboard, Keystone Identity Service, Neutron Networking Service, and Nova Compute Service from the OpenStack Grizzly (2013.1) release are all provided, in versions tested and integrated with Solaris features. Between the Open Stack packages themselves and all the python modules required for them, there’s over 100 new FOSS packages in this release.

Detailed list of changes

This table shows most of the changes to the bundled packages between the original Solaris 11.1 release, the latest Solaris 11.1 support repository update (SRU18, released April 14, 2014), and the Solaris 11.2 beta released today.

As with last time, some were excluded for clarity, or to reduce noise and duplication. All of the bundled packages which didn’t change the version number in their packaging info are not included, even if they had updates to fix bugs, security holes, or add support for new hardware or new features of Solaris.

PackageUpstream11.111.1 SRU1811.2 Beta
archiver/gnu-tarGNU tar1.
cloud/openstack/cinderOpenStacknot includednot included0.2013.1.4
cloud/openstack/glanceOpenStacknot includednot included0.2013.1.4
cloud/openstack/horizonOpenStacknot includednot included0.2013.1.4
cloud/openstack/keystoneOpenStacknot includednot included0.2013.1.4
cloud/openstack/neutronOpenStacknot includednot included0.2013.1.4
cloud/openstack/novaOpenStacknot includednot included0.2013.1.4
compress/gzipGNU gzip1.41.51.5
compress/pbzip2Parallel bzip2not includednot included1.1.6
compress/pixzpixznot includednot included1.0
database/berkeleydb-5Oracle Berkeley DB5.
database/mysql-55MySQLnot includednot included5.5.31
developer/build/antApache Ant1.
developer/debug/gdbGNU GDB6.86.87.6
developer/gcc-47GNU Compiler Collectionnot includednot included4.7.3
developer/gcc-48GNU Compiler Collectionnot includednot included4.8.2
developer/gnu-indentGNU indentnot includednot included2.2.9
developer/java/jdk-8Javanot includednot included1.
developer/javascript/jslJavaScript Lintnot includednot included0.3.0
developer/versioning/mercurialMercurial SCM2.
diagnostic/numatopnumatopnot includednot included1.0
editor/gnu-emacsGNU Emacs23.423.424.3
file/gnu-coreutilsGNU Coreutils8.58.58.16
file/mcGNU Midnight Commander4.
file/mlocatemlocatenot includednot included0.25
file/slocate3.13.1not included
library/cacaoCommon Agent Container2.
library/libarchivelibarchivenot includednot included3.0.4
library/libxml2XML C parser2.
library/perl-5/perl-x11-protocolCPAN: X11-Protocolnot includednot included0.56
library/perl-5/xml-libxmlCPAN: XML::LibXMLnot included2.142.14
library/perl-5/xml-namespacesupportCPAN: XML::NamespaceSupportnot included1.111.11
library/perl-5/xml-parser-threaded-512CPAN: XML::Parsernot included2.362.36
library/perl-5/xml-saxCPAN: XML::SAXnot included0.990.99
library/perl-5/xml-sax-baseCPAN: XML::SAX::Basenot included1.081.08
library/perl5/perl-tkCPAN: Tknot includednot included804.31
library/python-2/alembicalembicnot includednot included0.6.0
library/python-2/amqpamqpnot includednot included1.0.12
library/python-2/anyjsonanyjsonnot includednot included0.3.3
library/python-2/argparseargparsenot included1.
library/python-2/babelbabelnot includednot included1.3
library/python-2/beautifulsoup4beautifulsoup4not includednot included4.2.1
library/python-2/botobotonot includednot included2.9.9
library/python-2/cheetahcheetahnot includednot included2.4.4
library/python-2/cliffcliffnot includednot included1.4.5
library/python-2/cmd2cmd2not includednot included0.6.7
library/python-2/cov-corecov-corenot includednot included1.7
library/python-2/cssutilscssutilsnot includednot included0.9.6
library/python-2/d2to1d2to1not includednot included0.2.10
library/python-2/decoratordecoratornot includednot included3.4.0
library/python-2/djangodjangonot includednot included1.4.10
library/python-2/django-appconfdjango-appconfnot includednot included0.6
library/python-2/django_compressordjango_compressornot includednot included1.3
library/python-2/django_openstack_authOpenStacknot includednot included1.1.3
library/python-2/eventleteventletnot includednot included0.13.0
library/python-2/filechunkiofilechunkionot includednot included1.5
library/python-2/formencodeformencodenot includednot included1.2.6
library/python-2/greenletgreenletnot includednot included0.4.1
library/python-2/httplib2httplib2not includednot included0.8
library/python-2/importlibimportlibnot includednot included1.0.2
library/python-2/ipythonipythonnot includednot included0.10
library/python-2/iso8601iso8601not includednot included0.1.4
library/python-2/jsonpatchjsonpatchnot includednot included1.1
library/python-2/jsonpointerjsonpointernot includednot included1.0
library/python-2/jsonschemajsonschemanot includednot included2.0.0
library/python-2/kombukombunot includednot included2.5.12
library/python-2/lesscpylesscpynot includednot included0.9.10
library/python-2/librabbitmqlibrabbitmqnot includednot included1.0.1
library/python-2/lockfilelockfilenot includednot included0.9.1
library/python-2/markdownmarkdownnot includednot included2.3.1
library/python-2/markupsafemarkupsafenot includednot included0.18
library/python-2/mockmocknot includednot included1.0.1
library/python-2/netaddrnetaddrnot includednot included0.7.10
library/python-2/netifacesnetifacesnot includednot included0.8
library/python-2/nose-cover3nose-cover3not includednot included0.0.4
library/python-2/ordereddictordereddictnot includednot included1.1
library/python-2/oslo.configoslo.confignot includednot included1.2.1
library/python-2/passlibpasslibnot includednot included1.6.1
library/python-2/pastepastenot includednot included1.7.5.1
library/python-2/paste.deploypaste.deploynot includednot included1.5.0
library/python-2/pbrpbrnot includednot included0.5.21
library/python-2/pep8pep8not includednot included1.4.4
library/python-2/pippipnot includednot included1.4.1
library/python-2/prettytableprettytablenot includednot included0.7.2
library/python-2/pypynot includednot included1.4.15
library/python-2/pyasn1pyasn1not includednot included0.1.7
library/python-2/pyasn1-modulespyasn1-modulesnot includednot included0.0.5
library/python-2/pycountrypycountrynot includednot included0.17
library/python-2/pydnspydnsnot includednot included2.3.6
library/python-2/pyflakespyflakesnot includednot included0.7.2
library/python-2/pygmentspygmentsnot includednot included1.6
library/python-2/pyparsingpyparsingnot includednot included2.0.1
library/python-2/pyrabbitpyrabbitnot includednot included1.0.1
library/python-2/pytestpytestnot includednot included2.3.5
library/python-2/pytest-capturelogpytest-capturelognot includednot included0.7
library/python-2/pytest-codecheckerspytest-codecheckersnot includednot included0.2
library/python-2/pytest-covpytest-covnot includednot included1.6
library/python-2/python-imagingpython-imagingnot includednot included1.1.7
library/python-2/python-ldappython-ldapnot includednot included2.4.10
library/python-2/python-mysqlpython-mysqlnot includednot included1.2.2
library/python-2/python-zope-interfaceZopenot includednot included3.3.0
library/python-2/pytzpytznot includednot included2013.4
library/python-2/repoze.lrurepoze.lrunot includednot included0.6
library/python-2/requestsrequestsnot includednot included1.2.3
library/python-2/routesroutesnot includednot included1.13
library/python-2/setuptools-gitsetuptools-gitnot includednot included1.0
library/python-2/simplejsonsimplejsonnot includednot included2.1.2
library/python-2/sixsixnot includednot included1.4.1
library/python-2/sqlalchemysqlalchemynot includednot included0.7.9
library/python-2/sqlalchemy-migratesqlalchemy-migratenot includednot included0.7.2
library/python-2/stevedorestevedorenot includednot included0.10
library/python-2/sudssudsnot includednot included0.4
library/python-2/tempitatempitanot includednot included0.5.1
library/python-2/toxtoxnot includednot included1.4.3
library/python-2/unittest2unittest2not includednot included0.5.1
library/python-2/virtualenvvirtualenvnot includednot included1.9.1
library/python-2/waitresswaitressnot includednot included0.8.5
library/python-2/warlockwarlocknot includednot included1.0.1
library/python-2/webobwebobnot includednot included1.2.3
library/python-2/websockifywebsockifynot includednot included0.3.0
library/python-2/webtestWebTestnot includednot included2.0.6
library/python/cinderclientOpenStacknot includednot included1.0.7
library/python/glanceclientOpenStacknot includednot included0.12.0
library/python/keystoneclientOpenStacknot includednot included0.4.1
library/python/neutronclientOpenStacknot includednot included2.3.1
library/python/novaclientOpenStacknot includednot included2.15.0
library/python/quantumclientOpenStacknot includednot included2.2.4.3
library/python/swiftclientOpenStacknot includednot included2.0.2
library/security/opensslOpenSSL1.0.0.10 (1.0.0j) (1.0.0k) (1.0.1g)
mail/thunderbirdMozilla Thunderbird10.0.61717.0.6
mail/thunderbird/plugin/thunderbird-lightningMozilla Lightning10.0.61717.0.6
network/amqp/rabbitmqRabbitMQnot includednot included3.1.3
network/dns/bindISC BIND9.
runtime/clispGNU CLISP2.472.472.49
runtime/java/jre-8Javanot includednot included1.
runtime/perl-threaded-512Perlnot included5.
runtime/ruby-19Rubynot includednot included1.9.3.484
runtime/ruby-19/ruby-tkRubynot includednot included1.9.3.484
service/network/dhcp/isc-dhcpISC DHCP4.
service/network/dns/bindISC BIND9. (9.6-ESV-R7-P2) (9.6-ESV-R10-P2) (9.6-ESV-R10-P2)
service/network/dnsmasqDnsmasqnot includednot included2.68
service/network/ftpProFTPD1. (1.3.3g) (1.3.4c) (1.3.4c)
service/network/ntpNTP4.2.5.200 (4.2.5p200) (4.2.7p381) (4.2.7p381)
service/network/ptpPTPdnot includednot included2.2.0
shell/gnu-getoptGNU getoptnot includednot included1.1.5
shell/parallelGNU parallelnot includednot included0.2012.11.22
system/library/hmp-libsHMPnot includednot included2.2.8
system/library/security/libgcryptGNU libgcrypt1.
system/management/biosconfigHMPnot includednot included2.2.8
system/management/facterPuppetnot includednot included1.6.18
system/management/fwupdateHMPnot includednot included2.2.8
system/management/fwupdate/emulexHMPnot includednot included6.3.12.2
system/management/fwupdate/qlogicHMPnot includednot included1.7.3
system/management/hmp-snmpHMPnot includednot included2.2.8
system/management/hwmgmtcliHMPnot includednot included2.2.8
system/management/hwmgmtdHMPnot includednot included2.2.8
system/management/puppetPuppetnot includednot included3.4.1
system/management/raidconfigHMPnot includednot included2.2.8
system/management/ubiosconfigHMPnot includednot included2.2.8
terminal/csshCluster SSHnot includednot included4.2.1
terminal/tmuxtmuxnot includednot included1.8
text/gnu-grepGNU grep2.102.142.14
text/texinfoGNU texinfo4.74.134.13
web/browser/firefoxMozilla Firefox10.0.61717.0.6
web/java-servlet/tomcatApache Tomcat6.0.356.0.376.0.39
web/php-53/extension/php-zendopcacheZend OPcachenot includednot included7.0.2
web/server/apache-22Apache HTTPD2.
web/server/apache-22/module/apache-fcgidApache FastCGI2.
web/server/apache-22/module/apache-sedApache HTTPD2.
web/wgetGNU wget1.121.121.14
x11/demo/mesa-demosMesa 3-D8.
x11/diagnostic/intel-gpu-toolsX.Orgnot includednot included1.3
x11/library/mesaMesa 3-D7.
x11/server/xorg/driver/xorg-video-nvX.Org2. included
x11/server/xorg/driver/xorg-video-tridentX.Org1. included

Thursday Jan 02, 2014

Book Review: “Oracle Solaris 11: First Look”

I saw a tweet this weekend from Packt Publishing (@packtpub) offering their entire catalog of ebooks for $5 apiece until Friday, January 3, 2014, so I decided to check out their selection.

One of the books I found there was Oracle Solaris 11: First Look by long-time Solaris admin Phil Brown, whose website was a staple of the Solaris x86 mailing list a decade ago, and whose advocacy of Solaris x86 helped save it from permanent cancellation. I first met Phil when we were undergrads at U. C. Berkeley, where we served as sysadmin staff on the student-run cluster of Unix systems together, and we’ve kept in touch via the Solaris & OpenSolaris communities we both found ourselves in later.

Since it was only $5, I picked it up to see if it was worth recommending to others. When not on sale, the ebook is only $15, since it’s not a large book - the PDF is 168 pages, the epub on my iPad was 199 pages - and in both forms that includes an index of about 25 pages. That also made it a quick enough read that I could get through in an afternoon, skimming over the examples and reference materials.

This book is intended to get existing Solaris admins up to speed quickly on Solaris 11 - it’s not going to introduce the basics of system administration, but will tell you what commands to run now. If you don’t know what routers, subnets, or tunnels are, you probably want a more introductory book - if you know what they are, and need to know what to run instead of ifconfig or editing /etc/hostname.e1000g0 to configure them on Solaris 11, this book will help.

Phil’s biases as a long time server admin are obvious in some sections, such as the introduction to NWAM, the Network AutoMagic feature, which he suggests “from a server sysadmin perspective, it might perhaps be better named "Never Wake A Monster"” though he admits on a laptop it can be useful to adjust to networks in different locations. There’s also a few areas where you can tell the book was written before Solaris 11.1 was out and didn’t get updated for the latest changes.

As the author of the classic pkg-get tool for installing Solaris SVR4 packages from network repositories, he has plenty to say about the new IPS packaging system in Solaris 11 as well, providing some useful tips on finding packages and setting up local repositories, though he does discourage use of many of the more advanced pkg subcommands that can help admins take more control over exactly what gets installed and updated on their systems.

Overall it’s a decent aide, and something I may refer to in the future, as I don’t do system administration that often these days, and often need a refresher, especially when old habits no longer work. It’s more detailed in many sections that the official Transitioning From Oracle Solaris 10 to Oracle Solaris 11.1 manual, which mainly points off to the other Solaris 11 manuals for details; but concentrated only on the areas a typical system administrator will be configuring, not developer or end-user visible changes. I’d recommend it to experienced admins looking for a hands on guide for dealing with Solaris 11 systems, but it’s not the right level for those trying to plan a migration or learn Solaris administration from scratch.

As always, the above is solely my personal opinion, not an official Oracle corporate position or endorsement.

Sunday Nov 11, 2012

Solaris 11.1 changes building of code past the point of __NORETURN

While Solaris 11.1 was under development, we started seeing some errors in the builds of the upstream X.Org git master sources, such as:

"Display.c", line 65: Function has no return statement : x_io_error_handler
"hostx.c", line 341: Function has no return statement : x_io_error_handler
from functions that were defined to match a specific callback definition that declared them as returning an int if they did return, but these were calling exit() instead of returning so hadn't listed a return value.

These had been generating warnings for years which we'd been ignoring, but X.Org has made enough progress in cleaning up code for compiler warnings and static analysis issues lately, that the community turned up the default error levels, including the gcc flag -Werror=return-type and the equivalent Solaris Studio cc flags -v -errwarn=E_FUNC_HAS_NO_RETURN_STMT, so now these became errors that stopped the build. Yet on Solaris, gcc built this code fine, while Studio errored out. Investigation showed this was due to the Solaris headers, which during Solaris 10 development added a number of annotations to the headers when gcc was being used for the amd64 kernel bringup before the Studio amd64 port was ready. Since Studio did not support the inline form of these annotations at the time, but instead used #pragma for them, the definitions were only present for gcc.

To resolve this, I fixed both sides of the problem, so that it would work for building new X.Org sources on older Solaris releases or with older Studio compilers, as well as fixing the general problem before it broke more software building on Solaris.

To the X.Org sources, I added the traditional Studio #pragma does_not_return to recognize that functions like exit() don't ever return, in patches such as this Xserver patch. Adding a dummy return statement was ruled out as that introduced unreachable code errors from compilers and analyzers that correctly realized you couldn't reach that code after a return statement.

And on the Solaris 11.1 side, I updated the annotation definitions in <sys/ccompile.h> to enable for Studio 12.0 and later compilers the annotations already existing in a number of system headers for functions like exit() and abort(). If you look in that file you'll see the annotations we currently use, though the forms there haven't gone through review to become a Committed interface, so may change in the future.

Actually getting this integrated into Solaris though took a bit more work than just editing one header file. Our ELF binary build comparison tool, wsdiff, actually showed a large number of differences in the resulting binaries due to the compiler using this information for branch prediction, code path analysis, and other possible optimizations, so after comparing enough of the disassembly output to be comfortable with the changes, we also made sure to get this in early enough in the release cycle so that it would get plenty of test exposure before the release.

It also required updating quite a bit of code to avoid introducing new lint or compiler warnings or errors, and people building applications on top of Solaris 11.1 and later may need to make similar changes if they want to keep their build logs similarly clean.

Previously, if you had a function that was declared with a non-void return type, lint and cc would warn if you didn't return a value, even if you called a function like exit() or panic() that ended execution. For instance:

#include <stdlib.h>

callback(int status)
    if (status == 0)
        return status;
would previously require a never executed return 0; after the exit() to avoid lint warning "function falls off bottom without returning value".

Now the compiler & lint will both issue "statement not reached" warnings for a return 0; after the final exit(), allowing (or in some cases, requiring) it to be removed. However, if there is no return statement anywhere in the function, lint will warn that you've declared a function returning a value that never does so, suggesting you can declare it as void. Unfortunately, if your function signature is required to match a certain form, such as in a callback, you not be able to do so, and will need to add a /* LINTED */ to the end of the function.

If you need your code to build on both a newer and an older release, then you will either need to #ifdef these unreachable statements, or, to keep your sources common across releases, add to your sources the corresponding #pragma recognized by both current and older compiler versions, such as:

#pragma does_not_return(exit)
#pragma does_not_return(panic) 
Hopefully this little extra work is paid for by the compilers & code analyzers being able to better understand your code paths, giving you better optimizations and more accurate errors & warning messages.

Sunday Oct 28, 2012

Solaris 11.1: Changes to included FOSS packages

Besides the documentation changes I mentioned last time, another place you can see Solaris 11.1 changes before upgrading is in the online package repository, now that the 11.1 packages have been published to, as the “” branch. (Oracle Solaris Package Versioning explains what each field in that version string means.)

When you’re ready to upgrade to the packages from either this repo, or the support repository, you’ll want to first read How to Update to Oracle Solaris 11.1 Using the Image Packaging System by Pete Dennis, as there are a couple issues you will need to be aware of to do that upgrade, several of which are due to changes in the Free and Open Source Software (FOSS) packages included with Solaris, as I’ll explain in a bit.

Solaris 11 can update more readily than Solaris 10

In the Solaris 10 and older update models, the way the updates were built constrained what changes we could make in those releases. To change an existing SVR4 package in those releases, we created a Solaris Patch, which applied to a given version of the SVR4 package and replaced, added or deleted files in it. These patches were released via the support websites (originally SunSolve, now My Oracle Support) for applying to existing Solaris 10 installations, and were also merged into the install images for the next Solaris 10 update release. (This Solaris Patches blog post from Gerry Haskins dives deeper into that subject.)

Some of the restrictions of this model were that package refactoring, changes to package dependencies, and even just changing the package version number, were difficult to do in this hybrid patch/OS update model. For instance, when Solaris 10 first shipped, it had the Xorg server from X11R6.8. Over the first couple years of update releases we were able to keep it up to date by replacing, adding, & removing files as necessary, taking it all the way up to Xorg server release 1.3 (new version numbering begun after the X11R7 split of the X11 tree into separate modules gave each module its own version). But if you run pkginfo on the SUNWxorg-server package, you’ll see it still displayed a version number of 6.8, confusing users as to which version was actually included.

We stopped upgrading the Xorg server releases in Solaris 10 after 1.3, as later versions added new dependencies, such as HAL, D-Bus, and libpciaccess, which were very difficult to manage in this patching model. (We later got libpciaccess to work, but HAL & D-Bus would have been much harder due to the greater dependency tree underneath those.) Similarly, every time the GNOME team looked into upgrading Solaris 10 past GNOME 2.6, they found these constraints made it so difficult it wasn’t worthwhile, and eventually GNOME’s dependencies had changed enough it was completely infeasible. Fortunately, this worked out for both the X11 & GNOME teams, with our management making the business decision to concentrate on the “Nevada” branch for desktop users - first as Solaris Express Desktop Edition, and later as OpenSolaris, so we didn’t have to fight to try to make the package updates fit into these tight constraints.

Meanwhile, the team designing the new packaging system for Solaris 11 was seeing us struggle with these problems, and making this much easier to manage for both the development teams and our users was one of their big goals for the IPS design they were working on. Now that we’ve reached the first update release to Solaris 11, we can start to see the fruits of their labors, with more FOSS updates in 11.1 than we had in many Solaris 10 update releases, keeping software more up to date with the upstream communities.

Of course, just because we can more easily update now, doesn’t always mean we should or will do so, it just removes the package system limitations from forcing the decision for us. So while we’ve upgraded the X Window System in the 11.1 release from X11R7.6 to 7.7, the Solaris GNOME team decided it was not the right time to try to make the jump from GNOME 2 to GNOME 3, though they did update some individual components of the desktop, especially those with security fixes like Firefox. In other parts of the system, decisions as to what to update were prioritized based on how they affected other projects, or what customer requests we’d gotten for them.

So with all that background in place, what packages did we actually update or add between Solaris 11.0 and 11.1?

Core OS Functionality

One of the FOSS changes with the biggest impact in this release is the upgrade from Grub Legacy (0.97) to Grub 2 (1.99) for the x64 platform boot loader. This is the cause of one of the upgrade quirks, since to go from Solaris 11.0 to 11.1 on x64 systems, you first need to update the Boot Environment tools (such as beadm) to a new version that can handle boot environments that use the Grub2 boot loader. System administrators can find the details they need to know about the new Grub in the Administering the GRand Unified Bootloader chapter of the Booting and Shutting Down Oracle Solaris 11.1 Systems guide. This change was necessary to be able to support new hardware coming into the x64 marketplace, including systems using UEFI firmware or booting off disk drives larger than 2 terabytes.

For both platforms, Solaris 11.1 adds rsyslog as an optional alternative to the traditional syslogd, and OpenSCAP for checking security configuration settings are compliant with site policies.

Note that the support repo actually has newer versions of BIND & fetchmail than the 11.1 release, as some late breaking critical fixes came through from the community upstream releases after the Solaris 11.1 release was frozen, and made their way to the support repository. These are responsible for the other big upgrade quirk in this release, in which to upgrade a system which already installed those versions from the support repo, you need to either wait for those packages to make their way to the 11.1 branch of the support repo, or follow the steps in the aforementioned upgrade walkthrough to let the package system know it's okay to temporarily downgrade those.

Developer Stack

While Solaris 11.0 included Python 2.7, many of the bundled python modules weren’t packaged for it yet, limiting its usability. For 11.1, many more of the python modules include 2.7 versions (enough that I filtered them out of the below table, but you can always search on the package repository server for them.

For other language runtimes and development tools, 11.1 expands the use of IPS mediated links to choose which version of a package is the default when the packages are designed to allow multiple versions to install side by side.

For instance, in Solaris 11.0, GNU automake 1.9 and 1.10 were provided, and developers had to run them as either automake-1.9 or automake-1.10. In Solaris 11.1, when automake 1.11 was added, also added was a /usr/bin/automake mediated link, which points to the automake-1.11 program by default, but can be changed to another version by running the pkg set-mediator command.

Mediated links were also used for the Java runtime & development kits in 11.1, changing the default versions to the Java 7 releases (the 1.7.0.x package versions), while allowing admins to switch links such as /usr/bin/javac back to Java 6 if they need to for their site, to deal with Java 7 compatibility or other issues, without having to update each usage to use the full versioned /usr/jdk/jdk1.6.0_35/bin/javac paths for every invocation.

Desktop Stack

As I mentioned before, we upgraded from X11R7.6 to X11R7.7, since a pleasant coincidence made the X.Org release dates line up nicely with our feature & code freeze dates for this release. (Or perhaps it wasn’t so coincidental, after all, one of the benefits of being the person making the release is being able to decide what schedule is most convenient for you, and this one worked well for me.) For the table below, I’ve skipped listing the packages in which we use the X11 “katamari” version for the Solaris package version (mainly packages combining elements of multiple upstream modules with independent version numbers), since they just all changed from 7.6 to 7.7.

In the graphics drivers, we worked with Intel to update the Intel Integrated Graphics Processor support to support 3D graphics and kernel mode setting on the Ivy Bridge chipsets, and updated Nvidia’s non-FOSS graphics driver from 280.13 to 295.20.

Higher up in the desktop stack, PulseAudio was added for audio support, and liblouis for Braille support, and the GNOME applications were built to use them.

The Mozilla applications, Firefox & Thunderbird moved to the current Extended Support Release (ESR) versions, 10.x for each, to bring up-to-date security fixes without having to be on Mozilla’s agressive 6 week feature cycle release train.

Detailed list of changes

This table shows most of the changes to the FOSS packages between Solaris 11.0 and 11.1. As noted above, some were excluded for clarity, or to reduce noise and duplication. All the FOSS packages which didn't change the version number in their packaging info are not included, even if they had updates to fix bugs, security holes, or add support for new hardware or new features of Solaris.

archiver/unrar 3.8.5 4.1.4
audio/sox 14.3.0 14.3.2
backup/rdiff-backup 1.2.1 1.3.3
communication/im/pidgin 2.10.0 2.10.5
compress/gzip 1.3.5 1.4
compress/xz not included 5.0.1
database/sqlite-3 3.7.11
desktop/remote-desktop/tigervnc 1.0.90 1.1.0
desktop/window-manager/xcompmgr 1.1.5 1.1.6
desktop/xscreensaver 5.12 5.15
developer/build/autoconf 2.63 2.68
developer/build/autoconf/xorg-macros 1.15.0 1.17
developer/build/automake-111 not included 1.11.2
developer/build/cmake 2.6.2 2.8.6
developer/build/gnu-make 3.81 3.82
developer/build/imake 1.0.4 1.0.5
developer/build/libtool 1.5.22 2.4.2
developer/build/makedepend 1.0.3 1.0.4
developer/gnu-binutils 2.19 2.21.1
developer/java/jdepend not included 2.9
developer/java/jpackage-utils not included 1.7.5
developer/java/junit 4.5 4.10
developer/lexer/jflex not included 1.4.1
developer/parser/byaccj not included 1.14
developer/parser/java_cup not included 0.10
developer/quilt 0.47 0.60
developer/versioning/mercurial 1.8.4 2.2.1
developer/versioning/subversion 1.6.16 1.7.5
diagnostic/constype 1.0.3 1.0.4
diagnostic/nmap 5.21 5.51
diagnostic/scanpci 0.12.1 0.13.1
diagnostic/wireshark 1.4.8 1.8.2
diagnostic/xload 1.1.0 1.1.1
editor/gnu-emacs 23.1 23.4
editor/vim 7.3.254 7.3.600
file/lndir 1.0.2 1.0.3
image/editor/bitmap 1.0.5 1.0.6
image/gnuplot 4.4.0 4.6.0
image/library/libexif 0.6.19 0.6.21
image/library/libpng 1.4.8 1.4.11
image/library/librsvg 2.26.3 2.34.1
image/xcursorgen 1.0.4 1.0.5
library/audio/pulseaudio not included 1.1
library/expat 2.0.1 2.1.0
library/gc 7.1 7.2
library/graphics/pixman 0.22.0 0.24.4
library/guile 1.8.4 1.8.6
library/java/subversion 1.6.16 1.7.5
library/json-c not included 0.9
library/libedit not included 3.0
library/libee not included 0.3.2
library/libestr not included 0.1.2
library/libevent 1.3.5
library/liblouis not included 2.1.1
library/liblouisxml not included 2.1.0
library/libtecla 1.6.0 1.6.1
library/libtool/libltdl 1.5.22 2.4.2
library/nspr 4.8.8 4.8.9
library/openldap 2.4.25 2.4.30
library/pcre 7.8 8.21
library/perl-5/subversion 1.6.16 1.7.5
library/python-2/jsonrpclib not included 0.1.3
library/python-2/lxml 2.1.2 2.3.3
library/python-2/nose not included 1.1.2
library/python-2/pyopenssl not included 0.11
library/python-2/subversion 1.6.16 1.7.5
library/python-2/tkinter-26 2.6.4 2.6.8
library/python-2/tkinter-27 2.7.1 2.7.3
library/security/nss 4.12.10 4.13.1
library/security/openssl (1.0.0e) (1.0.0j)
mail/thunderbird 6.0 10.0.6
package/pkgbuild not included 1.3.104
print/filter/enscript not included 1.6.4
print/filter/gutenprint 5.2.4 5.2.7
print/lp/filter/foomatic-rip 3.0.2 4.0.15
runtime/perl-512 5.12.3 5.12.4
runtime/python-26 2.6.4 2.6.8
runtime/python-27 2.7.1 2.7.3
runtime/tcl-8/tcl-sqlite-3 3.7.11
security/compliance/openscap not included 0.8.1
security/nss-utilities 4.12.10 4.13.1
service/network/dhcp/isc-dhcp 4.1
service/network/ftp (ProFTPD)
service/network/samba 3.5.10 3.6.6
shell/conflict 0.2004.9.1 0.2010.6.27
shell/pipe-viewer 1.1.4 1.2.0
shell/zsh 4.3.12 4.3.17
system/boot/grub 0.97 1.99
system/font/truetype/liberation 1.4 1.7.2
system/library/freetype-2 2.4.6 2.4.9
system/library/libnet 1.1.5
system/management/cim/pegasus 2.9.1 2.11.0
system/management/ipmitool 1.8.10 1.8.11
system/management/wbem/wbemcli 1.3.7
system/network/routing/quagga 0.99.8 0.99.19
system/rsyslog not included 6.2.0
terminal/luit 1.1.0 1.1.1
text/convmv 1.14 1.15
text/gawk 3.1.5 3.1.8
text/gnu-grep 2.5.4 2.10
web/browser/firefox 6.0.2 10.0.6
web/browser/links 1.0 1.0.3
web/java-servlet/tomcat 6.0.33 6.0.35
web/php-53 not included 5.3.14
web/php-53/extension/php-apc not included 3.1.9
web/php-53/extension/php-idn not included 0.2.0
web/php-53/extension/php-memcache not included 3.0.6
web/php-53/extension/php-mysql not included 5.3.14
web/php-53/extension/php-pear not included 5.3.14
web/php-53/extension/php-suhosin not included 0.9.33
web/php-53/extension/php-tcpwrap not included 1.1.3
web/php-53/extension/php-xdebug not included 2.2.0
web/php-common not included 11.1
web/proxy/squid 3.1.8 3.1.18
web/server/apache-22 2.2.20 2.2.22
web/server/apache-22/module/apache-sed 2.2.20 2.2.22
web/server/apache-22/module/apache-wsgi not included 3.3
x11/diagnostic/xev 1.1.0 1.2.0
x11/diagnostic/xscope 1.3 1.3.1
x11/documentation/xorg-docs 1.6 1.7
x11/keyboard/xkbcomp 1.2.3 1.2.4
x11/library/libdmx 1.1.1 1.1.2
x11/library/libdrm 2.4.25 2.4.32
x11/library/libfontenc 1.1.0 1.1.1
x11/library/libfs 1.0.3 1.0.4
x11/library/libice 1.0.7 1.0.8
x11/library/libsm 1.2.0 1.2.1
x11/library/libx11 1.4.4 1.5.0
x11/library/libxau 1.0.6 1.0.7
x11/library/libxcb 1.7 1.8.1
x11/library/libxcursor 1.1.12 1.1.13
x11/library/libxdmcp 1.1.0 1.1.1
x11/library/libxext 1.3.0 1.3.1
x11/library/libxfixes 4.0.5 5.0
x11/library/libxfont 1.4.4 1.4.5
x11/library/libxft 2.2.0 2.3.1
x11/library/libxi 1.4.3 1.6.1
x11/library/libxinerama 1.1.1 1.1.2
x11/library/libxkbfile 1.0.7 1.0.8
x11/library/libxmu 1.1.0 1.1.1
x11/library/libxmuu 1.1.0 1.1.1
x11/library/libxpm 3.5.9 3.5.10
x11/library/libxrender 0.9.6 0.9.7
x11/library/libxres 1.0.5 1.0.6
x11/library/libxscrnsaver 1.2.1 1.2.2
x11/library/libxtst 1.2.0 1.2.1
x11/library/libxv 1.0.6 1.0.7
x11/library/libxvmc 1.0.6 1.0.7
x11/library/libxxf86vm 1.1.1 1.1.2
x11/library/mesa 7.10.2 7.11.2
x11/library/toolkit/libxaw7 1.0.9 1.0.11
x11/library/toolkit/libxt 1.0.9 1.1.3
x11/library/xtrans 1.2.6 1.2.7
x11/oclock 1.0.2 1.0.3
x11/server/xdmx 1.10.3 1.12.2
x11/server/xephyr 1.10.3 1.12.2
x11/server/xorg 1.10.3 1.12.2
x11/server/xorg/driver/xorg-input-keyboard 1.6.0 1.6.1
x11/server/xorg/driver/xorg-input-mouse 1.7.1 1.7.2
x11/server/xorg/driver/xorg-input-synaptics 1.4.1 1.6.2
x11/server/xorg/driver/xorg-input-vmmouse 12.7.0 12.8.0
x11/server/xorg/driver/xorg-video-ast 0.91.10 0.93.10
x11/server/xorg/driver/xorg-video-ati 6.14.1 6.14.4
x11/server/xorg/driver/xorg-video-cirrus 1.3.2 1.4.0
x11/server/xorg/driver/xorg-video-dummy 0.3.4 0.3.5
x11/server/xorg/driver/xorg-video-intel 2.10.0 2.18.0
x11/server/xorg/driver/xorg-video-mach64 6.9.0 6.9.1
x11/server/xorg/driver/xorg-video-mga 1.4.13 1.5.0
x11/server/xorg/driver/xorg-video-openchrome 0.2.904 0.2.905
x11/server/xorg/driver/xorg-video-r128 6.8.1 6.8.2
x11/server/xorg/driver/xorg-video-trident 1.3.4 1.3.5
x11/server/xorg/driver/xorg-video-vesa 2.3.0 2.3.1
x11/server/xorg/driver/xorg-video-vmware 11.0.3 12.0.2
x11/server/xserver-common 1.10.3 1.12.2
x11/server/xvfb 1.10.3 1.12.2
x11/server/xvnc 1.0.90 1.1.0
x11/session/sessreg 1.0.6 1.0.7
x11/session/xauth 1.0.6 1.0.7
x11/session/xinit 1.3.1 1.3.2
x11/transset 0.9.1 1.0.0
x11/trusted/trusted-xorg 1.10.3 1.12.2
x11/x11-window-dump 1.0.4 1.0.5
x11/xclipboard 1.1.1 1.1.2
x11/xclock 1.0.5 1.0.6
x11/xfd 1.1.0 1.1.1
x11/xfontsel 1.0.3 1.0.4
x11/xfs 1.1.1 1.1.2

P.S. To get the version numbers for this table, I ran a quick perl script over the output from:

% pkg contents -H -r -t depend -a type=incorporate -o fmri \
  `pkg contents -H -r -t depend -a type=incorporate -o fmri entire@0.5.11,5.11-` \
  | sort >> /tmp/11.1
% pkg contents -H -r -t depend -a type=incorporate -o fmri \
  `pkg contents -H -r -t depend -a type=incorporate -o fmri entire@0.5.11,5.11-` \
  | sort >> /tmp/11.0

Thursday Oct 25, 2012

Documentation Changes in Solaris 11.1

One of the first places you can see Solaris 11.1 changes are in the docs, which have now been posted in the Solaris 11.1 Library on I spent a good deal of time reviewing documentation for this release, and thought some would be interesting to blog about, but didn't review all the changes (not by a long shot), and am not going to cover all the changes here, so there's plenty left for you to discover on your own.

Just comparing the Solaris 11.1 Library list of docs against the Solaris 11 list will show a lot of reorganization and refactoring of the doc set, especially in the system administration guides. Hopefully the new break down will make it easier to get straight to the sections you need when a task is at hand.

Packaging System

Unfortunately, the excellent in-depth guide for how to build packages for the new Image Packaging System (IPS) in Solaris 11 wasn't done in time to make the initial Solaris 11 doc set. An interim version was published shortly after release, in PDF form on the OTN IPS page. For Solaris 11.1 it was included in the doc set, as Packaging and Delivering Software With the Image Packaging System in Oracle Solaris 11.1, so should be easier to find, and easier to share links to specific pages the HTML version.

Beyond just how to build a package, it includes details on how Solaris is packaged, and how package updates work, which may be useful to all system administrators who deal with Solaris 11 upgrades & installations. The Adding and Updating Oracle Solaris 11.1 Software Packages was also extended, including new sections on Relaxing Version Constraints Specified by Incorporations and Locking Packages to a Specified Version that may be of interest to those who want to keep the Solaris 11 versions of certain packages when they upgrade, such as the couple of packages that had functionality removed by an (unusual for an update release) End of Feature process in the 11.1 release.

Also added in this release is a document containing the lists of all the packages in each of the major package groups in Solaris 11.1 (solaris-desktop, solaris-large-server, and solaris-small-server). While you can simply get the contents of those groups from the package repository, either via the web interface or the pkg command line, the documentation puts them in handy tables for easier side-by-side comparison, or viewing the lists before you've installed the system to pick which one you want to initially install.

X Window System

We've not had good X11 coverage in the online Solaris docs in a while, mostly relying on the man pages, and upstream X.Org docs. In this release, we've integrated some X coverage into the Solaris 11.1 Desktop Adminstrator's Guide, including sections on installing fonts for fontconfig or legacy X11 clients, X server configuration, and setting up remote access via X11 or VNC. Of course we continue to work on improving the docs, including a lot of contributions to the upstream docs all OS'es share (more about that another time).


One of the things Oracle likes to do for its products is to publish security guides for administrators & developers to know how to build systems that meet their security needs. For Solaris, we started this with Solaris 11, providing a guide for sysadmins to find where the security relevant configuration options were documented. The Solaris 11.1 Security Guidelines extend this to cover new security features, such as Address Space Layout Randomization (ASLR) and Read-Only Zones, as well as adding additional guidelines for existing features, such as how to limit the size of tmpfs filesystems, to avoid users driving the system into swap thrashing situations.

For developers, the corresponding document is the Developer's Guide to Oracle Solaris 11 Security, which has been the source for years for documentation of security-relevant Solaris API's such as PAM, GSS-API, and the Solaris Cryptographic Framework. For Solaris 11.1, a new appendix was added to start providing Secure Coding Guidelines for Developers, leveraging the CERT Secure Coding Standards and OWASP guidelines to provide the base recommendations for common programming languages and their standard API's. Solaris specific secure programming guidance was added via links to other documentation in the product doc set.

In parallel, we updated the Solaris C Libary Functions security considerations list with details of Solaris 11 enhancements such as FD_CLOEXEC flags, additional *at() functions, and new stdio functions such as asprintf() and getline(). A number of code examples throughout the Solaris 11.1 doc set were updated to follow these recommendations, changing unbounded strcpy() calls to strlcpy(), sprintf() to snprintf(), etc. so that developers following our examples start out with safer code. The Writing Device Drivers guide even had the appendix updated to list which of these utility functions, like snprintf() and strlcpy(), are now available via the Kernel DDI.

Little Things

Of course all the big new features got documented, and some major efforts were put into refactoring and renovation, but there were also a lot of smaller things that got fixed as well in the nearly a year between the Solaris 11 and 11.1 doc releases - again too many to list here, but a random sampling of the ones I know about & found interesting or useful:

Wednesday Nov 09, 2011

S11 X11: ye olde window system in today's new operating system

Today's the big release for Oracle Solaris 11, after 7 years of development. For me, the Solaris 11 release comes a little more than 11 years after I joined the X11 engineering team at what was then Sun, and finishes off some projects that were started all the way back then.

For instance, when I joined the X team, Sun was finishing off the removal of the old OpenWindows desktop, and we kept getting questions asking about the rest of the stuff being shipped in /usr/openwin, the directory that held both the OpenLook applications and the X Window System software. I wrote up an ARC case at the time to move the X software to /usr/X11, but there were various issues and higher priority work, so we didn't end up starting that move until near the end of the Solaris 10 development cycle several years later. Solaris 10 thus had a mix of the recently added Xorg server and related code delivered in /usr/X11, while most of the existing bits from Sun's proprietary fork of X11R6 were still in /usr/openwin.

During Solaris 11 development, we finished that move, and then jumped again, moving the programs directly into /usr/bin, following the general Solaris 11 strategy of using /usr/bin for most of the programs shipped with the OS, and using other directories, such as /usr/gnu/bin, /usr/xpg4/bin, /usr/sunos/bin, and /usr/ucb for conflicting alternate implementations of the programs shipped in /usr/bin, no longer as a way to segregate out various subsystems to allow the OS to better fit onto the 105Mb hard disks that shipped with Sun workstations back when /usr/openwin was created. However, if for some reason you wanted to build your own set of X binaries, you could put them in /usr/X11R7 (as I do for testing builds of the upstream git master repos), and then put that first in your $PATH, so nothing is really lost here.

The other major project that was started during Solaris 10 development and finished for Solaris 11 was replacing that old proprietary fork of X11R6, including the Xsun server, with the modernized, modularized, open source X11R7.* code base from the new X.Org, including the Xorg server. The final result, included in this Solaris 11 release, is based mostly on the X11R7.6 release, including recent additions such as the XCB API I blogged about last year, though we did include newer versions of modules that had upstream releases since the X11R7.6 katamari, such as Xorg server version 1.10.3.

That said, we do still apply some local patches, configuration options, and other changes, for things from just fitting into the Solaris man page style or adding support for Trusted Extensions labeled desktops. You can see all of those changes in our source repository, which is searchable and browsable via OpenGrok on (or via hgweb on community mirrors such as and available for anonymous hg cloning as well. That xnv-clone tree is now frozen, a permanent snapshot of the Solaris 11 sources, while we've created a new x-s11-update-clone tree for the Solaris 11 update releases now being developed to follow on from here.

Naturally, when your OS has 7 years between major release cycles, the hardware environment you run on greatly changes in the meantime as well, and as the layer that handles the graphics hardware, there have been changes due to that. Most of the SPARC graphics devices that were supported in Solaris 10 aren't any more, because the platforms they ran in are no longer supported - we still ship a couple SPARC drivers that are supported, the efb driver for the Sun XVR-50, XVR-100, and XVR-300 cards based on the ATI Radeon chipsets, and the astfb driver for the AST2100 remote Keyboard/Video/Mouse/Storage (rKVMS) chipset in the server ILOM devices. On the x86 side, the EOL of 32-bit platforms let us clear out a lot of the older x86 video device drivers for chipsets and cards you wouldn't find in x64 systems - of course, there's still many supported there, due to the wider variety of graphics hardware found in the x64 world, and even some recent updates, such as the addition of Kernel Mode Setting (KMS) support for Intel graphics up through the Sandy Bridge generation.

For those who followed the development as it happened, either via watching our open source code releases or using one of the many development builds and interim releases such as the various Solaris Express trains, much of this is old news to you. For those who didn't, or who want a refresher on the details, you can see last year's summary in my X11 changes in the 2010.11 release blog post. Once again, the detailed change logs for the X11 packages are available, though unfortunately, all the links in them to the bug reports are now broken, so browsing the hg history log is probably more informative.

Since that update, which covered up to the build 151 released as 2010.11, we've continued development and polishing to get this Solaris 11 release finished up. We added a couple more components, including the previously mentioned xcb libraries, the FreeGLUT library, and the Xdmx Distributed Multihead X server. We cleaned up documentation, including the addition of some docs for the Xserver DTrace provider in /usr/share/doc/Xserver/. The packaging was improved, clearing up errors and optimizing the builds to reduce unnecessary updates. A few old and rarely used components were dropped, including the rstart program for starting up X clients remotely (ssh X forwarding replaces this in a more secure fashion) and the xrx plugin for embedding X applications in a web browser page (which hasn't been kept up to date with the rapidly evolving browser environment). Because Solaris 11 only supports 64-bit systems, and most of the upstream X code was already 64-bit clean, the X servers and most of the X applications are now shipped as 64-bit builds, though the libraries of course are delivered in both 32-bit and 64-bit versions for binary compatibility with applications of each flavor. The Solaris auditing system can now record each attempt by a client to connect to the Xorg server and whether or not it succeeded, for sites which need that level of detail.

In total, we recorded 1512 change request id's during Solaris 11 development, from the time we forked the “Nevada” gate from the Solaris 10 release until the final code freeze for todays release - some were one line bug fixes, some were man page updates, some were minor RFE's and some were major projects, but in the end, the result is both very different (and hopefully much better) than what we started with, and yet, still contains the core X11 code base with 24 years of backwards compatibility in the core protocols and APIs.

Monday Nov 15, 2010

X11 changes in the 2010.11 release

Another OS release came out today, 2010.11, and as usual, it has a number of X11 changes. The biggest change in X is probably... Hmm, I can see by the look on your face, you're not buying the casual use of “as usual” there. Okay, you caught me, this OS release isn't quite following our previous pattern, so I guess we better get that out of the way first. Please remember I am not an Oracle spokesman, and can't speak on behalf of Oracle, so don't even think of quoting this as “Oracle says...”

In many ways, this release is simply the continuation of the OpenSolaris distro releases of the last few years. It's built the same way, using the IPS packaging system and repositories, and Caiman installers, as the OpenSolaris 2009.06 and prior releases were. Where OpenSolaris 2009.06 (the last full release) was the biweeekly build numbered 111b, and the release we'd planned to put out as OpenSolaris 2010.03 earlier this year (and which made it to the package repository, but was not put up as downloadable ISO's) would have been biweekly build 134b, this release is 151a. You should be able to upgrade to it from OpenSolaris 2009.06 or OpenSolaris /dev builds via the package repository following the instructions in the 2010.11 release notes.

So what's different about this OS release? Well, it's not named OpenSolaris anymore for starters - it's Oracle Solaris 11 Express. We'd always said that OpenSolaris releases were leading up to Solaris 11 eventually, and this name emphasizes we're getting closer to that (though still not there yet). It also recognizes that this release is built by Oracle, not Sun nor the OpenSolaris community. While it's built on the work done by the OpenSolaris community, and many portions of it are still developed as open projects on, the kernel and core utilities are once again being developed behind closed doors, and the final assembly and testing are similarly done in house. The license terms for the free downloads have changed as well (though it's still offered under support contract for commercial production use as well), and the OS images include some of the encumbered packages we'd had to keep out of OpenSolaris in order to allow OpenSolaris to be freely redistributable. (Not all of them, since some were simply EOL'ed as they were for hardware well past the end of its supported lifetime, like many of the old SPARC frame buffers.)

So with that out of the way, back to the topic at hand - what's new in the X Window System in this release? Well that depends on how far back you're coming from. You can browse the complete changelogs for X going back to the point we branched the Nevada branches from the Solaris 10 release, so I'll try to stick to the highlights.

Changes since the last OpenSolaris X11 source release

None, since the X sources on are still updated automatically from our internal master gate on each commit. (In fact, since the source gates currently reflect a point between biweekly builds 153 & 154, they have changes newer than this release, such as the integrations of libxcb and FreeGLUT.)

Changes since the last OpenSolaris developer build release (b134)

There were 17 biweekly builds between the last one published to in March and this release. The biggest change in the X packages in this period was their packaging. Previously we built our packages using the old SVR4 package format that was used since Solaris 2.0, and in many cases following the breakdown used in the old Solaris 2 releases (SUNWxwinc for most headers, SUNWxwplt for most libraries, SUNWxwman for most man pages), and then the release team converted those to the IPS format used in the OpenSolaris releases. Like several of the other consolidations, X has now converted to building IPS packages directly, and in the process refactored the X packages to better follow the way the upstream X.Org sources were split into modules at X11R7, which also happens to be more similar to the way most Linux distros break them up. This should allow easier creation of minimized environments with the subset of X packages you need.

As for headers and man pages, they are now included in the packages they are used with - for instance all the libX11 headers and API man pages are directly in the x11/library/libx11 package. System admins can still decide to include or exclude them in their installs though, since they are tagged with the devel and docfacets”, which are the IPS mechanism for controlling optional package components. To read more about how to use these with X or the other changes in the refactoring, see the heads up messages I posted when this work integrated.

Of course, there were also the usual updates to new upstream releases - Xorg 1.7.7, freetype 2.4.2, fontconfig 2.8.0, among many others. The X server packages now also include the mdb modules and scripts for getting client and grab information from the server that I blogged about back in April.

Changes since the last OpenSolaris stable release (2009.06 / b111b)

This period saw the completion of our multiyear project to completely replace the old Solaris X code base with the X11R7 open source code base from X.Org. Solaris 10 and earlier shipped with Sun's proprietary fork of X11R6, with bits of X11R5, X11R6.4, X11R6.6, & X11R6.8 mixed in. We're now set up to much more easily track upstream and are deviated from upstream in much fewer places than before (partially due to pushing a number of our previous fixes back upstream, in other cases, we determined the upstream code was better and went with it).

We also had a very large user-visible change in build 130: all the files moved from /usr/X11 directly into /usr/bin & /usr/lib, following the work done in other parts of Solaris to move files from locations like /usr/ccs/bin and /usr/sfw to the common /usr directories. We still have symlinks in /usr/openwin and /usr/X11 for backwards compatibility, so we shouldn't break your .xinitrc calls to /usr/openwin/bin/xrdb or /usr/X11/bin/xmodmap.

Since 2009.06, we moved from Xorg 1.5 to 1.7.4. Of course, with this upgrade, we got the HAL support for input device configuration working just as X.Org started moving off HAL upstream, something we still need to deal with for Solaris - for this release, input devices are still configured in HAL .fdi files. The xorgcfg and xorgconfig programs did go away as part of this move though - fortunately more and more systems are working without any xorg.conf at all, and when one is needed, only the sections being changed have to be included, lessening the utility of programs to generate full configuration files. The new Xorg also includes support for virtual consoles on systems with the necessary kernel driver support (all x86 systems and SPARCs with frame buffers supporting “coherent console”).

We also added the synaptics touchpad driver, synergy software for sharing input devices with multiple systems, the simple xcompmgr composite manager, the xinput client for input device configuration, and finally provided IPS packaged versions of the classic xdm display manager and xfs legacy font server. The Xprint server and several related commands did go away, but the libXp library was kept for binary compatibility.

Our VNC implementation was converted from RealVNC 4.1.3 to TigerVNC 1.0.1, which is being kept up-to-date with new Xorg releases, unlike RealVNC, which hasn't really been updating it's open source release in the last few years. xscreensaver was finally updated from 5.01 to 5.11, and was actually moved out of the X gate in OpenSolaris to building as a RPM-style pkgbuild spec file with the other higher-level desktop software - hopefully in the process we fixed some long-standing bugs in our forked code.

Graphics updates included Nvidia's driver support for various new devices and OpenGL 4.0, and Intel's DRI updates, including GEM support in their DRM module. Mesa was added on SPARC to provide a matching OpenGL implementation, but with only the software renderer, no hardware acceleration.

What else has changed?

Besides the official Solaris 11 Express release information, you can find more details on changes in this release on a bunch of other blogs, such as:

But here's some changes in other parts of the OS you may not see listed on those:

Of course, that's just a small sample, the full changelogs are a few thousand items long (and unfortunately, some of the consolidations haven't published theirs outside the firewall).


Engineer working on Oracle Solaris and with the X.Org open source community.


The views expressed on this blog are my own and do not necessarily reflect the views of Oracle, the X.Org Foundation, or anyone else.

See Also
Follow me on twitter


« July 2015