News, tips, partners, and perspectives for the Oracle Solaris operating system

Future of Python on Solaris

Darren Moffat
Senior Software Architect

If you are an administrator of Oracle Solaris 11 systems you will very likely be aware that several of the critical installation and management tools are implemented in Python rather than the more traditional C and/or shell script.  Python is often a good choice as a systems development language and Oracle Solaris is not alone in choosing to use it more extensively.

If you have done any programming with Python or have installed other Python tools you are probably aware that there are compatibility issues between the Python 2.x and 3.x language runtimes.  In Oracle Solaris 11.4.7 we deliver multiple releases of Python: 2.7, 3.4, 3.5.  Future updates to the Oracle Solaris support repository will change the exact version mix.  The Python community will no longer be supporting and providing security or other fixes for the 2.7 environment from the year 2020.  That is currently just about 9 months away.

Python Versions

A default installation of Oracle Solaris 11.4 will currently have at least Python 2.7 and a version of Python 3, /usr/bin/python by default will run Python 2.7.  There are also paths that allow explicitly picking a version, this is very useful for the #! line of a script: eg /usr/bin/python2, /usr/bin/python3 or even more specific /usr/bin/python3.5. The packaging system allows you to choose now which version of Python /usr/bin/python represents; there are two package mediators used to control this.  The 'python' mediator allows selecting which /usr/bin/python represents and the 'python3' mediator allows selecting which minor version of the 3.x release train the /usr/bin/python3 link points to. In a future support repository update we will change the default to be a version of Python 3. 

Python Modules & Solaris Tools

The Oracle Solaris 11 package tools (IPS: Image Packaging System) is one of the critical OS tools implemented in Python, in current releases of 11.4 this is still using Python 2.7, but we have also delivered a parallel version for testing that uses Python 3.4. We don't expect to have a single support repository update where we switch over all of the OS Python consumers to Python 3 in one release. When we implement operating system tools using Python the #! line should always be precise as to which major.minor version it is the script is expected to run with. Our package dependencies then ensure the correct versions of pkg:/runtime/python are installed.

As a result of using Python to implement parts of the OS we deliver a fairly large number of Python modules from open source communities.  When we deliver a Python module via IPS it is always installed below /usr/lib/pythonX.Y/vendor-packages/.  The intent of the vendor-packages area is to be like the core Python site-packages area but to make it clear these modules came as part of the OS and should be updated with the OS tools.  This provides some level of isolation between OS installed modules and administrator/developer use of tools such as 'pip' package installer for Python.

Sharing the Python Installation

We have seen some recent cases where use of 'pip' to add additional modules, or different versions of modules delivered to vendor-packages has broken critical system administration tools such as /usr/bin/pkg and /usr/bin/beadm.  The reason for this is because vendor-packages is really just an extension of the core Python site-packages concept.  While it is possible to instruct the Python interpreter to ignore the user and site packages area for isolation reasons that doesn't help when the OS core tools are implemented in Python and need access to content installed in the vendor-packages area.  Oracle Solaris is not the only operating system impacted by this problem, other OS vendors are developing their own solutions to this, some of the concepts and recommendations you will find are similar.

Firstly, like others do we strongly recommend the use of Python Virtual Environments when using pip or other Python install tools to add modules not delivered by Oracle Solaris packages.  If you wish to leverage the fact that Oracle Solaris has already included some of the modules you need, you can use '--system-site-packages' when you create the virtual environment. Using that allows pip/python to reference the system site-packages and thus find the vendor-package are but it won't install content into /usr/lib/pythonX.y/site-packages.  For example:

$ python3 -m venv --system-site-packages /usr/local/mytool
$ source /usr/local/mytool/bin/activate
(mytool) $ python
Python 3.5.6 (default, Mar  7 2019, 08:31:32) [C] on sunos5
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', '/usr/lib/python35.zip', '/usr/lib/python3.5', '/usr/lib/python3.5/plat-sunos5', '/usr/lib/python3.5/lib-dynload', '/usr/local/mytool/lib/python3.5/site-packages', '/usr/lib/python3.5/site-packages', '/usr/lib/python3.5/vendor-packages']

We can see from the above that the Python module search path includes the system and the 'mytool' application area as well as the Solaris IPS delivered content in vendor-packages.  If we now run 'pip install' it will add any modules it needs to the site-packages directory below 'mytool' rather than the one under /usr/lib/python3.5.

Hardening the shared Python Installation

So that we can keep providing open source Python modules for administrator/developer use and continue to use Python for the OS itself we need a solution that protects the Python modules in vendor-packages from being overridden by administrator installed ones in site-packages.  We don't wish to duplicate whole Python environments, since as I mentioned above we currently use multiple Python versions for the OS itself.  The solution we have come up was based on the following requirements:

  1. pip and other install tools should not be able to add incompatible versions of modules to site-packages that confuse core OS tools
  2. pip uninstall must not be able to remove packages from vendor-packages instead of site-packages. This happens because 'vendor-packages' is higher than 'site-packages' in the Python module search path.
  3. pip and other Python installers should work in a virtualenv and remain working for the system 'site-packages' area
  4. the solution should not require patching the Python runtime or pip to enforce the above.

We will be delivering parts of the solution to this over time when they are ready and tested to our satisfaction, there should be no impact to any of your tooling that uses Python we deliver.

The first change we will deliver is to further harden the core Python tools that we author against content in the 'site-packages' area that is potentially toxic to our tools.  Ideally we should have been able to just pass -S as part of the #! line of the script, our tools already use -s to ignore the per user 'site-packages' but that would mean the vendor-packages area is never setup since it depends on /usr/lib/pythonX.Y/site-packages/vendor-packages.pth to get added.  So instead we deliver a new module 'solaris.no_site_packages' that removes the system site-packages leaving vendor-packages.  This module only makes sense for Oracle Solaris core components, in particular any 3rd party FOSS tools that are in Python do not include this.

We are still investigating the best  the solution to ensure that tools such as pip can not uninstall IPS delivered content in vendor-packages. Patching the version of pip we deliver would only be  a partial solution, since it would only work until an admin runs 'pip install --upgrade pip', since that would effectively remove any patch we made to ignore the vendor-packages area. Even with that restriction we may still apply a patch to pip. 

We could potentially deliver an /etc/pip.conf file that forces the use of virtual environments, as described in this Python document virtual-env. However doing that could potentially have unexpected side effects if you are managing to safely use 'pip install' to place content into the system 'site-packages' area.

Instead we intend to use a feature of Oracle Solaris that allows us to tag files as not being able to be removed even by the root user (even when running with all privileges), a feature I described in this blog post 11 years ago! IPS can deliver system attributes (sysattr) for files already.  Currently we only use that in a single package (pkg://solaris/release/name) to ensure that the version files that contain the value used for 'uname -v' are not easy to "accidentally" change, for that we add sysattr=readonly to the package manifest for those files. We are exploring the possibility of using the 'nounlink' or maybe even the 'immutable' attribute to protect all of the Python modules delivered by IPS to vendor-packages directory. In theory, we could do this for all content delivered via IPS that is not explicitly marked with the preserve attribute but we probably won't go that far in the initial use and instead focus just on hardening the Python environment.

In summary Python on Oracle Solaris is very much here to stay as /usr/bin/python and we are working to ensure that you can continue to use the Python modules we deliver while providing an environment where our core tools implemented in Python are safe from any additional Python modules added to the system.

Join the discussion

Comments ( 4 )
  • David Lange Wednesday, July 10, 2019
    Thanks for your efforts, but the support in the compiler team is making python quite difficult out in the field both on solaris10x and 11x. I went through an exercise just to get the python3.6 to build on sol10 with solaris compiler. Now the addon libraries like pandas is even more fun aka https://github.com/pandas-dev/pandas/issues/8780

    I don't believe management is taking opensource seriously on solaris. Our developers want this tools and it's putting old coders to task to get work around these issues.

    How come you don't have a separate build just for the OS and another pkg for app development?
  • Darren J Moffat Friday, July 12, 2019
    I feel I need to point out that "Management" alone is not responsible for the technical decisions about what goes into Oracle Solaris and how it is built. Senior engineering staff do that with the support of management and all the other engineering staff, but it is primarily technical engineering staff that makes the choices about what FOSS we deliver and how we deliver it.

    All of our engineering staff take FOSS very seriously for Oracle Solaris, particularly since more than 80% of the deliverables for Oracle Solaris 11.4 is FOSS. We have many team members that are dedicated to the delivery of FOSS on Oracle Solaris.

    The Python (and similarly Perl, Ruby as well) environment we provide is primarily for use by the OS. If it helps develop applications then great please do use it. If you really need to isolate the OS Python from the one used by your app then you can do that using virtualenv and a separate Python install that you deliver and support.

    We made a conscious and informed decision to only ship one installation of each Python (or Perl or Ruby) version rather than multiple installs that need to be maintained. If that that doesn't meet your needs then I'm afraid you need to do something else yourself but you maybe better off trying to embrace what we do ship.

    Your complaints about the "compiler team" are directed to the wrong place. The issue you hit with Pandas appears to be about how Python was built for Oracle Solaris and the fact that Python records which compilers were used and may attempt to use those same compilers again when building modules with native C code. The Oracle Solaris side of the bug mentioned in the Pandas github issue was fixed in 11.3.30. You appear to be attempting to build Python 3.6 which was released in December 2016 on Solaris 10 which had its last update release in 2013. If you can update your application environment to new versions of Python you really should look at updating the underlying OS as well, that way you will get Python 3.7 shipped with the OS and fully supported by Oracle. The Python 3.7 we ship is built using the gcc we ship, while older versions of Python were built using the Oracle Solaris Studio compiler suite.

    If you have specific issues with how we build Python or any other part of the operating system please do log a Service Request with Oracle support. We will be happy to investigate and work with you on a resolution to any bugs we have introduced due to how we build and deliver any FOSS component.
  • Guido van Rossum Thursday, October 29, 2020
  • Darren J Moffat Monday, November 30, 2020
    Thanks for asking about Python support on Solaris and referencing the recent discussion on the python-dev mailing list.

    The issue has now been resolved and support for Solaris, and the related Illumos based distributions, will be staying in the main Python codebase.

    See this message in the thread for the resolution:

Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.