Friday Nov 04, 2016

Non-reboot immutable zone (as of 11.3 SRU 12)

Immutable Zones

In Solaris 11 we introduced Immutable Zones.  In Solaris 11.2 we added the Immutable Global Zone.  Please read my earlier blog if you are unfamiliar with Immutable Zones.

What is new: Why was a reboot required?

When a Solaris system reboots after the system was just installed or upgraded, self assembly is performed.  This phase requires the system to be read-write, even when it is an immutable (global) zone.  All services which require self-assembly specify that the self-assembly-complete milestone is dependent on them. When the self-assembly-complete milestone is reached, the system reboots immutable.

As the self-assembly-complete milestone isn't reached until man-index and several other service have run their course, the second boot may come quite a bit later.  Use: the following command to list all the services:

# svcs -d self-assembly-complete

The self-assembly-service itself reboots the system; for non-global zones this is pretty quick but for global zones this can be pretty slow.  We have always found this a bug rather than a feature so we decided to fix that for the next Solaris release.  Of course, if self-assembly needs to wait for man-index and other services to complete, not requiring a reboot was only a small part of the puzzle we needed to solve.

In Solaris 11.2 with the design of the immutable global zone, we created the Trusted Path.  Processes which run under the Trusted Path can modify files which are normally read-only in immutable zones.  While this feature was invented to allow modification and updates of an Immutable Global Zone as well as modifications to non-global Immutable Zones using the zlogin -T or -U options, we realized that this could be used to do certain self-assembly operations asynchronously. Asynchronous updates would be interrupted if we rebooted so we needed to fix both problems at the same time.

We implemented this in the next Solaris release and one of our customers who got earlier access to that release filed a service request for a back port of that feature and the rest is history.

What is new in Solaris 11.3 SRU 12?

We introduce a new milestone. immutable-setup, which is reached early during boot and sets the zone's file-mac-profile.  Services which need to know whether the system is immutable need to wait until that milestone is reached.  For native zones this is already done by zoneadmd before starting the zone.

When the self-assembly-complete milestone is reached, the zone will become immutable at that time.  In earlier releases of Solaris a reboot would happen:

[NOTICE: This read-only system transiently booted read/write] [NOTICE: Now that self assembly has been completed, the system is rebooting]

as of Solaris 11.3 SRU 12 you will see:

[NOTICE: switching to read-only mode]

and the system will continue while man-index asynchronously continues to format manual pages.

But that is not all!  It is now much easier to configure an immutable global zone and, again, without requiring a reboot:

# zonecfg -z global set file-mac-profile=<profile>

# zoneadm -z global apply

You will see the same message on /dev/console

Thursday May 01, 2014

Solaris 11.2: Immutable Global Zone

This is blog is a bit more substantial; it requires some knowledge about Solaris Zones, Immutable Zones and Solaris administration in general. It is high-level; in future I'm hoping to get down to the nuts and bolts.

Immutable Zones

In Solaris 11 we added the Read-Only Root Non-Global Zones, marketed as Immutable Zones; this is a feature that makes a zone tamper-proof.

In an Immutable Zone is configured simply by setting the "file-mac-profile" to one of "strict" (not much writeable), "fixed-configuration" and "flexible-configuration" (configuration is writeable but binaries and such or not). This is all implemented in the kernel based on pathnames and depending on the context; the super-user in the global zone can still update the zone or even modify protected files as long as that is not done from within the zone.

We have made some changes to Immutable Non-Global Zones (IMZ, for short) that came out of developing the Immutable Global Zones (IMGZ); we have added a new feature, the "Trusted Path (TP)"; when logged in through the Trusted Path using the "-T" option to zlogin(1m), you can now modify protected files from within the zone. This is much safer as you no longer need to give root access in the global zone nor do you need to boot the IMZ in writeable mode. In the following example, we log in to the zone "fixed" which has been configured with the fixed-configuration file-mac-profile. A normal root login doesn't allow us to modify "/etc/passwd"; I'm using touch(1) under privilege debugging to illustrate the error and was caused the error. When we login with the "-T" option we suddenly can modify "/etc/passwd" because we're now in the Trusted Path. Notice also that the output from privilege debugging has been clarified; it points to the MWAC(5) manual page and it now also lists the system call name and not the number as it did before.

# zlogin fixed
[Connected to zone 'fixed' pts/3]
Oracle Corporation      SunOS 5.11      11.2    April 2014
root@fixed:~# ppriv -De touch /etc/passwd
touch[117063]: MWAC(5) policy violation (euid = 0, syscall = "utimensat") for "/etc/passwd" at fop_setattr+0x10b
touch: cannot change times on /etc/passwd: Read-only file system
root@fixed:~# logout
[Connection to zone 'fixed' pts/3 closed]

# zlogin -T fixed
[Connected to zone 'fixed' pts/3]
Oracle Corporation      SunOS 5.11      11.2    April 2014
root@fixed:~# ppriv -De touch /etc/passwd

Additionally, we have restricted the use of mount(1m) in an IMZ, while we allowed random loopback mounts before we now only allow loopback mounts on empty directories unless the file or directory isn't protected by MWAC(5).

Immutable Global Zone

In order to prevent tampering of the file system, we have extended Immutable Zones in Solaris 11.2 to the global zone; using the same mechanism you can now configure the global zone as an IMGZ. As there is no "super-global" zone, a different mechanism has been designed to enter the Trusted Path. A kernel-zone still has a bare-metal zone controlling it, so this doesn't apply to kernel zones. Some additional steps need to be taken and they are listed here.

Preparing the global zone for immutable global zone.

As maintenance of the global zone is only possible using the Trusted Path access; Trusted Path is only available on the console, so make sure the console is accessible through the ILOM, a serial connection or through the graphical console.

Once a system is configured as an immutable global zone, the break sequence, F1-A on a graphical console, <break> or the alternate break sequence (CR-tilde-<ctl-b>) on a serial console, will instead start the Trusted Path login. (A immediate second break sequence will work as a standard break-sequence: start the kernel debugger (if it is loaded), drop to the OBP, etc)

Configuring the Global Immutable Zone

The configuration of the global zone is done through zonecfg(1m) by picking the appropriate file-mac-profile for your situation; they allowed values are the same for non-global immutable zones: "strict", "fixed-configuration", "flexible-configuration". See zonecfg(1m).

Note that if the system uses DHCP to set network interfaces, the "flexible-configuration" must be selected.

        # zonecfg -z global
        zonecfg:global> set file-mac-profile=flexible-configuration
The "rpool" dataset will be restricted but sub dataset can be unrestricted using "add dataset"
        zonecfg:global> add dataset
        zonecfg:global:dataset> set name=rpool/export
        zonecfg:global:dataset> end

        zonecfg:global> add dataset
        zonecfg:global:dataset> set name=rpool/zones
        zonecfg:global:dataset> end
In this example we add "rpool/export" and "rpool/zones"; writable data sets for users and for zones. An immutable global zone can only run zones in unrestricted datasets. All the children of an unrestricted dataset are also unrestricted.

Note that all datasets on other zpools are unrestricted and there is no needed to add them with "add dataset".

After committing the zonecfg boot information is written and the boot archive is updated:

        zonecfg:global> commit
        updating /platform/sun4u/boot_archive
When the system is configured, it should be rebooted the system will boot with an immutable global zone.

Maintenance of the immutable global zone

An immutable zone cannot be updated other then through the Trusted Path login or when the system is booted in writeable mode by using the "-w" flag when booting. Note that if you try to reboot the immutable zone with "reboot -- -w", the argument is ignored when not performed through the Trusted Path login.

After using the break-sequence on the console, you should be greeted with:

        trusted path console login:
Login and assume the root role; at that point ordinary commands used to update the system are available; this includes "pkg update", "beadm activate" or also "zonecfg" if the need arises to change the global zone's configuration.

A separate pam stack can be configured for tpdlogin(1).

When "pkg update" is performed, the first boot of the immutable global zone is read write; this is needed by the system to perform the needed self-assembly steps. When the self-assembly steps have been performed, the system will reboot and in this second boot the system will be immutable again.




« April 2017