Monday Oct 22, 2012

Ops Center 12c - Update - Provisioning Solaris on x86 Using a Card-Based NIC

Last week, I posted a blog describing how to use Ops Center to provision Solaris over the network via a NIC on a card rather than the built-in NIC.  Really, that was all about how to install Solaris on a SPARC system.  This week, we'll look at how to do the same thing for an x86-based server.

Really, the overall process is exactly the same, at least for Solaris 11, with only minor updates. We will focus on Solaris 11 for this blog.  Once I verify that the same approach works for Solaris 10, I will provide another update.

Booting Solaris 11 on x86

Just as before, in order to configure the server for network boot across a card-based NIC, it is necessary to declare the asset to associate the additional MACs with the server.  You likely will need to access the server console via the ILOM to figure out the MAC and to get a good idea of the network instance number. 

The simplest way to find both of these is to start a network boot using the desired NIC and see where it appears in the list of network interfaces and what MAC is used when it tries to boot.  Go to the ILOM for the server.  Reset the server and start the console.  When the BIOS loads, select the boot menu, usually with Ctrl-P.  This will give you a menu of devices to boot from, including all of the NICs.  Select the NIC you want to boot from.  Its position in the list is a good indication of what network number Solaris will give the device.

In this case, we want to boot from the 5th interface (GB_4, net4).  Pick it and start the boot processes.  When it starts to boot, you will see the MAC address for the interface

Once you have the network instance and the MAC, go through the same process of declaring the asset as in the SPARC case.  This associates the additional network interface with the server..

Creating an OS Provisioning Plan

The simplest way to do the boot via an alternate interface on an x86 system is to do a manual boot.  Update the OS provisioning profile as in the SPARC case to reflect the fact that we are booting from a different interface.  Update, in this case, the network boot device to be GB_4/net4, or the device corresponding to your network instance number.  Configure the profile to support manual network boot by checking the box for manual boot in the OS Provisioning profile.

Booting the System

Once you have created a profile and plan to support booting from the additional NIC, we are ready to install the server. Again, from the ILOM, reset the system and start the console.  When the BIOS loads, select boot from the Boot Menu as above.  Select the network interface from the list as before and start the boot process.  When the grub bootloader loads, the default boot image is the Solaris Text Installer.  On the grub menu, select Automated Installer and Ops Center takes over from there.


The key lesson from all of this is that Ops Center is a valuable tool for provisioning servers whether they are connected via built-in network interfaces or via high-speed NICs on cards.  This is great news for modern datacenters using converged network infrastructures.  The process works for both SPARC and x86 Solaris installations.  And it's easy and repeatable.

Tuesday Oct 16, 2012

Ops Center 12c - Provisioning Solaris Using a Card-Based NIC

It's been a long time since last I added something here, but having some conversations this last week, I got inspired to update things.

I've been spending a lot of time with Ops Center for managing and installing systems these days.  So, I suspect a number of my upcoming posts will be in that area.

Today, I want to look at how to provision Solaris using Ops Center when your network is not connected to one of the built-in NICs.  We'll talk about how this can work for both Solaris 10 and Solaris 11, since they are pretty similar.  In both cases, WANboot is a key piece of the story.

Here's what I want to do:  I have a Sun Fire T2000 server with a Quad-GbE nxge card installed.  The only network is connected to port 2 on that card rather than the built-in network interfaces.  I want to install Solaris on it across the network, either Solaris 10 or Solaris 11.  I have met with a lot of customers lately who have a similar architecture.  Usually, they have T4-4 servers with the network connected via 10GbE connections.

Add to this mix the fact that I use Ops Center to manage the systems in my lab, so I really would like to add this to Ops Center.  If possible, I would like this to be completely hands free.  I can't quite do that yet. Close, but not quite.

WANBoot or Old-Style NetBoot?

When a system is installed from the network, it needs some help getting the process rolling.  It has to figure out what its network configuration (IP address, gateway, etc.) ought to be.  It needs to figure out what server is going to help it boot and install, and it needs the instructions for the installation.  There are two different ways to bootstrap an installation of Solaris on SPARC across the network.   The old way uses a broadcast of RARP or more recently DHCP to obtain the IP configuration and the rest of the information needed.  The second is to explicitly configure this information in the OBP and use WANBoot for installation

WANBoot has a number of benefits over broadcast-based installation: it is not restricted to a single subnet; it does not require special DHCP configuration or DHCP helpers; it uses standard HTTP and HTTPS protocols which traverse firewalls much more easily than NFS-based package installation.  But, WANBoot is not available on really old hardware and WANBoot requires the use o Flash Archives in Solaris 10.  Still, for many people, this is a great approach.

As it turns out, WANBoot is necessary if you plan to install using a NIC on a card rather than a built-in NIC.

Identifying Which Network Interface to Use

One of the trickiest aspects to this process, and the one that actually requires manual intervention to set up, is identifying how the OBP and Solaris refer to the NIC that we want to use to boot.  The OBP already has device aliases configured for the built-in NICs called net, net0, net1, net2, net3.  The device alias net typically points to net0 so that when you issue the command  "boot net -v install", it uses net0 for the boot.  Our task is to figure out the network instance for the NIC we want to use. 

We will need to get to the OBP console of the system we want to install in order to figure out what the network should be called.  I will presume you know how to get to the ok prompt.  Once there, we have to see what networks the OBP sees and identify which one is associated with our NIC using the OBP command show-nets.

SunOS Release 5.11 Version 11.0 64-bit
Copyright (c) 1983, 2011, Oracle and/or its affiliates. All rights reserved.

{4} ok banner

Sun Fire T200, No Keyboard
Copyright (c) 1998, 2010, Oracle and/or its affiliates. All rights reserved.
OpenBoot 4.30.4.b, 32640 MB memory available, Serial #69057548.
Ethernet address 0:14:4f:1d:bc:c, Host ID: 841dbc0c.

{4} ok show-nets
a) /pci@7c0/pci@0/pci@2/network@0,1
b) /pci@7c0/pci@0/pci@2/network@0
c) /pci@780/pci@0/pci@8/network@0,3
d) /pci@780/pci@0/pci@8/network@0,2
e) /pci@780/pci@0/pci@8/network@0,1
f) /pci@780/pci@0/pci@8/network@0
g) /pci@780/pci@0/pci@1/network@0,1
h) /pci@780/pci@0/pci@1/network@0
Enter Selection, q to quit: d
/pci@780/pci@0/pci@8/network@0,2 has been selected.
Type ^Y ( Control-Y ) to insert it in the command line. 
e.g. ok nvalias mydev ^Y 
         for creating devalias mydev for /pci@780/pci@0/pci@8/network@0,2
{4} ok devalias
net3                     /pci@7c0/pci@0/pci@2/network@0,1
net2                     /pci@7c0/pci@0/pci@2/network@0
net1                     /pci@780/pci@0/pci@1/network@0,1
net0                     /pci@780/pci@0/pci@1/network@0
net                      /pci@780/pci@0/pci@1/network@0
name                     aliases

By looking at the devalias and the show-nets output, we can see that our Quad-GbE card must be the device nodes starting with  /pci@780/pci@0/pci@8/network@0.  The cable for our network is plugged into the 3rd slot, so the device address for our network must be /pci@780/pci@0/pci@8/network@0,2.

With that, we can create a device alias for our network interface.  Naming the device alias may take a little bit of trial and error, especially in Solaris 11 where the device alias seems to matter more with the new virtualized network stack. So far in my testing, since this is the "next" network interface to be used, I have found success in naming it net4, even though it's a NIC in the middle of a card that might, by rights, be called net6 (assuming the 0th interface on the card is the next interface identified by Solaris and this is the 3rd interface on the card).  So, we will call it net4.  We need to assign a device alias to it:

{4} ok nvalias net4 /pci@780/pci@0/pci@8/network@0,2
{4} ok devalias
net4                     /pci@780/pci@0/pci@8/network@0,2

We also may need to have the MAC for this particular interface, so let's get it, too.  To do this, we go to the device and interrogate its properties.

{4} ok cd /pci@780/pci@0/pci@8/network@0,2
{4} ok .properties
assigned-addresses       82060210 00000000 03000000 00000000 01000000 
                         82060218 00000000 00320000 00000000 00008000 
                         82060220 00000000 00328000 00000000 00008000 
                         82060230 00000000 00600000 00000000 00100000 
local-mac-address        00 21 28 20 42 92 
phy-type                 mif

From this, we can see that the MAC for this interface is  00:21:28:20:42:92.  We will need this later.

This is all we need to do at the OBP.  Now, we can configure Ops Center to use this interface.

Network Boot in Solaris 10

Solaris 10 turns out to be a little simpler than Solaris 11 for this sort of a network boot.  Since WANBoot in Solaris 10 fetches a specified

In order to install the system using Ops Center, it is necessary to create a OS Provisioning profile and its corresponding plan.  I am going to presume that you already know how to do this within Ops Center 12c and I will just cover the differences between a regular profile and a profile that can use an alternate interface.

Create a OS Provisioning profile for Solaris 10 as usual.  However, when you specify the network resources for the primary network, click on the name of the NIC, probably GB_0, and rename it to GB_N/netN, where N is the instance number you used previously in creating the device alias.  This is where the trial and error may come into play.  You may need to try a few instance numbers before you, the OBP, and Solaris all agree on the instance number.  Mark this as the boot network.

For Solaris 10, you ought to be able to then apply the OS Provisioning profile to the server and it should install using that interface.  And if you put your cards in the same slots and plug the networks into the same NICs, this profile is reusable across multiple servers.

Why This Works

If you watch the console as Solaris boots during the OSP process, Ops Center is going to look for the device alias netN.  Since WANBoot requires a device alias called just net, Ops Center uses the value of your netN device alias and assigns that device to the net alias.  That means that boot net will automatically use this device.  Very cool!  Here's a trace from the console as Ops Center provisions a server:

Sun Sun Fire T200, No Keyboard
Copyright (c) 1998, 2010, Oracle and/or its affiliates. All rights reserved.
OpenBoot 4.30.4.b, 32640 MB memory available, Serial #69057548.
Ethernet address 0:14:4f:1d:bc:c, Host ID: 841dbc0c.

auto-boot? =            false
{0} ok  
{0} ok printenv network-boot-arguments
network-boot-arguments =  host-ip=,router-ip=,subnet-mask=,hostname=atl-sewr-52,client-id=0100144F1DBC0C,file=
{0} ok
{0} ok devalias net
net                      /pci@780/pci@0/pci@1/network@0
{0} ok devalias net4
net4                     /pci@780/pci@0/pci@8/network@0,2
{0} ok devalias net /pci@780/pci@0/pci@8/network@0,2
{0} ok setenv network-boot-arguments host-ip=,router-ip=,subnet-mask=,hostname=atl-sewr-52,client-id=0100144F1DBC0C,file=
network-boot-arguments =  host-ip=,router-ip=,subnet-mask=,hostname=atl-sewr-52,client-id=0100144F1DBC0C,file=
{0} ok
{0} ok boot net - install
Boot device: /pci@780/pci@0/pci@8/network@0,2  File and args: - install
/pci@780/pci@0/pci@8/network@0,2: 1000 Mbps link up
<time unavailable> wanboot info: WAN boot messages->console
<time unavailable> wanboot info: configuring /pci@780/pci@0/pci@8/network@0,2

See what happened?  Ops Center looked for the network device alias called net4 that we specified in the profile, took the value from it, and made it the net device alias for the boot.  Pretty cool!

WANBoot and Solaris 11

Solaris 11 requires an additional step since the Automated Installer in Solaris 11 uses the MAC address of the network to figure out which manifest to use for system installation.  In order to make sure this is available, we have to take an extra step to associate the MAC of the NIC on the card with the host.  So, in addition to creating the device alias like we did above, we also have to declare to Ops Center that the host has this new MAC.

Declaring the NIC

Start out by discovering the hardware as usual.  Once you have discovered it, take a look under the Connectivity tab to see what networks it has discovered.  In the case of this system, it shows the 4 built-in networks, but not the networks on the additional cards.  These are not directly visible to the system controller.

 In order to add the additional network interface to the hardware asset, it is necessary to Declare it.  We will declare that we have a server with this additional NIC, but we will also  specify the existing GB_0 network so that Ops Center can associate the right resources together.  The GB_0 acts as sort of a key to tie our new declaration to the old system already discovered.  Go to the Assets tab, select All Assets, and then in the Actions tab, select Add Asset.  Rather than going through a discovery this time, we will manually declare a new asset.

When we declare it, we will give the hostname, IP address, system model that match those that have already been discovered.  Then, we will declare both GB_0 with its existing MAC and the new GB_4 with its MAC.  Remember that we collected the MAC for GB_4 when we created its device alias.

After you declare the asset, you will see the new NIC in the connectivity tab for the asset.  You will notice that only the NICs you listed when you declared it are seen now.  If you want Ops Center to see all of the existing NICs as well as the additional one, declare them as well.  Add the other GB_1, GB_2, GB_3 links and their MACs just as you did GB_0 and GB_4.

 Installing the OS

 Once you have declared the asset, you can create an OS Provisioning profile for Solaris 11 in the same way that you did for Solaris 10.  The only difference from any other provisioning profile you might have created already is the network to use for installation.  Again, use GB_N/netN where N is the interface number you used for your device alias and in your declaration.

 And away you go.  When the system boots from the network, the automated installer (AI) is able to see which system manifest to use, based on the new MAC that was associated, and the system gets installed.

{0} ok
{0} ok printenv network-boot-arguments
network-boot-arguments =  host-ip=,router-ip=,subnet-mask=,hostname=atl-sewr-52,client-id=01002128204292,file=
{0} ok
{0} ok devalias net
net                      /pci@780/pci@0/pci@1/network@0
{0} ok devalias net4
net4                     /pci@780/pci@0/pci@8/network@0,2
{0} ok devalias net /pci@780/pci@0/pci@8/network@0,2
{0} ok setenv network-boot-arguments host-ip=,router-ip=,subnet-mask=,hostname=atl-sewr-52,client-id=01002128204292,file=
network-boot-arguments =  host-ip=,router-ip=,subnet-mask=,hostname=atl-sewr-52,client-id=01002128204292,file=
{0} ok
{0} ok boot net - install
Boot device: /pci@780/pci@0/pci@8/network@0,2  File and args: - install
/pci@780/pci@0/pci@8/network@0,2: 1000 Mbps link up
<time unavailable> wanboot info: WAN boot messages->console
<time unavailable> wanboot info: configuring /pci@780/pci@0/pci@8/network@0,2
SunOS Release 5.11 Version 11.0 64-bit
Copyright (c) 1983, 2011, Oracle and/or its affiliates. All rights reserved.
Remounting root read/write
Probing for device nodes ...
Preparing network image for use
Downloading solaris.zlib
--2012-02-17 15:10:17--
Connecting to connected.
HTTP request sent, awaiting response... 200 OK
Length: 126752256 (121M) [text/plain]
Saving to: `/tmp/solaris.zlib'

100%[======================================>] 126,752,256 28.6M/s   in 4.4s   

2012-02-17 15:10:21 (27.3 MB/s) - `/tmp/solaris.zlib' saved [126752256/126752256]


So, why go to all of this trouble?  More and more, I find that customers are wiring their data center to only use higher speed networks - 10GbE only to the hosts.  Some customers are moving aggressively toward consolidated networks combining storage and network on CNA NICs.  All of this means that network-based provisioning cannot rely exclusively on the built-in network interfaces.  So, it's important to be able to provision a system using other than the built-in networks.  Turns out, that this is pretty straight-forward for both Solaris 10 and Solaris 11 and fits into the Ops Center deployment process quite nicely.

Hopefully, you will be able to use this as you build out your own private cloud solutions with Ops Center.

Monday Dec 20, 2010

Finish Scripts and First-Boot Services with Bootable AI

In my last couple of blogs, I have talked about using the Automated Installer, specifically using Bootable AI.  We talked about creating a manifest to guide the installation and we talked about creating a system configuration manifest to configure the system identity.  The piece that has not yet been addressed, from the perspective of a long-time Jumpstart user, is how to do the final customization via finish scripts.

In general, there is no notion of an install script that is bundled into a package when using the Image Package System in Solaris 11 Express as there is with SVR4 packages from Solaris 10.  However, a package can install a service that can carry out the same function.  Its operation is a bit more controlled since it has to have its dependencies satisfied and can run in a reduced privilege environment if needed.  Additionally, many of the actions typically scripted into installation scripts, such as creation of users, groups, links, directories, are all built-in actions of the packaging system.

So, the question arises of how to use the IPS packaging system to add our own packages to a system, whether at installation time or later, and how to perform the necessary first-boot customizations to a system we are installing.  The requirement to create our own packages comes from the fact that there is no other way to deliver content to the system being installed during the installation process except through the AI manifest - and that means IPS packages.  In Jumpstart, there was a variable set at installation that pointed to a particular NFS-mounted directory where install scripts could reside.  This was all well and good so long as you could mount that directory.  When it was not available, you were left again with the notion of creating and delivering SVR4 packages via the Jumpstart profile.  So, the situation is not far different than in Solaris 10 and earlier.  There's just a little different syntax and a different network protocol in use to deliver the payload.

Why Use a Local Repository

There are two main reasons to create a local repository for use by AI and IPS.  First, you might choose to replicate the Solaris repository rather than make a home-run through your corporate network firewall to Oracle for every package installation on every server.  Performance and control are clearly going to be better and more deterministic by locating the data closer to where you plan to use it.  The second reason to create a local repository would be to host your own locally provided packages - whether developed locally or provided by an ISV.

The question then arises whether to combine both of these into the same repository.  My personal opinion is that it is better to keep them separate.  Just as it is a good practice to keep the binaries and files of the OS separate from those locally created or provided by applications on the disk, it seems a good idea to keep the repositories separate.  That does not mean that multiple systems are needed to host the data, however.  Multiple repository services can be hosted on the same system on different network ports, pointing to different directories on the disk.  Think of it like hosting multiple web sites with different ports and different htdocs directories.

Rather than go through all the details of creating a local mirror of the Solaris repository, I will refer you to Brian Leonard's blog on that topic.  Here, he talks about creating a local mirror from the repository ISO.  He also shows how you can create a file-based repository for local use.

For much of the rest of this exercise, I am relying on an article in the OpenSolaris Migration Hub that talks about creating a First Boot Service, which in turn references a page on creating a local package repository.  It is well worth it to read through these two pages.

Setting Up A Local Repository

So far, we have avoided the use of any sort of install server.  However, to have a local repository available during installation, this becomes a necessity.  So, pick a server to be used as a package repository.

Rather than clone the Solaris repository, we will create a new, empty repository to fill with our own packages.  We will configure the necessary SMF properties to enable the repository.  And then we will fill it with the packages that we need to deploy.

On the host that will host the repository, select a location in its filesystem and set it aside for this function.  A good practice would be to create a ZFS filesystem for the repository.  In this case, you can enable compression on the repository and easily control its size through quotas and reservations.  In this example, we will just create a ZFS filesystem within the root pool.  Often you will have a separate pool for this sort of function.

# zfs create -p -o mountpoint=/var/repos/ -o compression=on rpool/repos/

Next, we will need to set the SMF properties to support the repository.  The service application/pkg/server is responsible for managing the actual package depot process.  As such, it refers to properties to locate the repository on disk, establish what port to use, etc.

The property pkg/inst_root specifies where on the repository server's disk the repo resides.  

# svccfg -s application/pkg/server setprop pkg/inst_root=/rpool/repo0906/repo

pkg/readonly specifies whether or not the repository can be updated.  Typically, for a cloned Solaris repository, this will be set to true.  It also is a good practice to set it to true when it should not be updated.

# svccfg -s application/pkg/server setprop pkg/readonly=true

pkg/prefix specifies the name that the repository will take when specified as a publisher by a client system.  pkg/port specifies the port where the repository will answer.

# svccfg -s application/pkg/server setprop pkg/prefix=local-pkgs
# svccfg -s application/pkg/server setprop pkg/port=9000

Once the properties are set, refresh and enable the service.

# svcadm refresh application/pkg/server
# svcadm enable application/pkg/server

Creating a First-Boot Package

Now that the repository has been created, we need to create a package to go into that repository.  Since there are no post-install scripts with the Image Packaging System, we will create an SMF service that will be automatically enabled so that it will run when the system boots.  One technique used with Jumpstart was to install a script into /etc/rc3.d that would run late in the boot sequence and would then remove itself so that it would only run on the first boot.  We will take a similar path with our first-boot service.  We will have it disable itself so that it doesn't continue to run on each boot.  There are two parts to creating this simple package.  First, we have to create the manifest for the service, and second, we have to create the script that will be used as the start method within the service.  This area is covered in more depth in the OpenSolaris Migration Hub paper on Creating a First Boot Service.  In fact, we will use the manifest from that paper.

Creating the Manifest

We will use the manifest from Creating a First Boot Service directly and call it finish-script-manifest.xml.  The main points to see here are
  • the service is enabled automatically when we import the manifest
  • the service is dependent on svc:/milestone/multi-user so that it won't run until the system is in the Solaris 11 Express equivalent of run level 3.
  • the script /usr/bin/, which we will provide, is going to be run as the start method when the service begins.


<?xml version="1.0"?>
<!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1">

<service_bundle type='manifest' name='Finish:finish-script'>


   <create_default_instance enabled='true' />

    <single_instance />

    <dependency name='autofs' grouping='require_all' restart_on='none' type='service'>
        <service_fmri value='svc:/system/filesystem/autofs:default' />

<dependency name='multi-user' grouping='require_all' restart_on='none' type='service'>
      <service_fmri value='svc:/milestone/multi-user:default' />



        <property_group name='startd' type='framework'>
                <propval name='duration' type='astring' value='transient' />


Creating the Script

In this example, we will create a trivial finish script.  All it will do is log that it has run and then disable itself.  You could go so far as to have the finish script uninstall itself.  However, rather than do that we will just disable the service.  Certainly, you could have a much more expansive finish script, with multiple files and multiple functions.  Our script is short and simple:

svcadm disable finish-script
#pkg uninstall pkg:/finish
echo "Completed Finish Script" > /var/tmp/finish_log.$$
exit 0

Adding Packages to Repository

Now that we have created the finish script and the manifest for the first-boot service, we have to insert these into the package repository that we created earlier.  Take a look at the pkgsend man page for a lot more details about how all of this works.  It's possible with pkgsend to add SVR4 package bundles into the repository, as well as tar-balls and directories full of files.  Since our package is simple, we will just insert each package.

When we open the package in the repository, we have to specify the version number.  Take a good look at the pkg(5) man page to understand the version numbering and the various actions that could be part of the package.  Since I have been working on this script, I have decided that it is version 0.3.  We start by opening the package for insertion.  Then, we add each file to the package in the repository, specifying file ownership, permissions, and path.  Once all the pieces have been added, we close the package and the FMRI for the package is returned.

# eval `pkgsend -s http://localhost:9000/ open finish@0.3`
# pkgsend -s http://localhost:9000/ add file finish-script-manifest.xml mode=0555 owner=root group=bin path=/var/svc/manifest/system/finish-script-manifest.xml restart_fmri=svc:/system/manifest-import:default
# pkgsend -s http://localhost:9000/ add file mode=0555 owner=root group=bin path=/usr/bin/
# pkgsend -s http://localhost:9000/ close

Updating the AI Manifest

Now that the repository is created, we can add the new repository as a publisher to verify that it has the contents we expect.  On the package server, itself, we can add this publisher.  Remember that we set the prefix for the publisher to local-pkgs and that we specified it should run on port 9000.  This name could be anything that makes sense for your enterprise - perhaps the domain for the company or something that will identify it as local rather than part of Solaris is a good choice.

# pkg set-publisher -p http://localhost:9000 local-pkgs
pkg set-publisher:
  Added publisher(s): local-pkgs
# pkg list -n "pkg://local-pkgs/\*"
NAME (PUBLISHER)                                    VERSION         STATE      UFOXI
finish (local-pkgs)                                 0.3             known      -----

Now that we know the repository is on-line and contains the packages that we want to deploy, we have to update the AI manifest to reflect both the new publisher and the new packages to install.  First, to update the publisher, beneath the section that specifies the default publisher, add a second source for this new publisher:

        <publisher name="solaris">
          <origin name=""/>
         <publisher name="local-pkgs">
           <origin name="http://my-local-repository-server:9000/"/>

Then, add the packages to the list of packages to install.  Depending on how you named your package, you may not need to specify the repository for installation.  IPS will search the path of repositories to find each package.  However, it's not a bad idea to be specific.

      <software_data action="install" type="IPS">

Now, when you install the system using this manifest, in addition to the regular Solaris bits being installed, and the system configuration services executing, the finish script included in your package will be run on the first boot.  Since the script called by the service turns itself off, it will not continue to run on subsequent boots.  You can do whatever sort of additional configuration that you might need to do.  But, before you spend a long time converting all of your old Jumpstart scripts into a first-boot service, take a look at the built-in capabilities of AI and of Solaris 11 Express in general.  It may be that much of the work you had to code yourself before is no longer required.  For example, IP interface configuration is simple and persistent with ipadm.  Other new functions in Solaris 11 Express remove the need to write custom code, too.  But for the cases where you need custom code, this sort of first-boot service gives you a hook so that you can do what you need.

Booting and Installing with Bootable AI

My last couple of blogs have been about creating a manifest to be used when installing a system with the Solaris 11 Express Automated Installer.  Now that we have this basic manifest constructed, let's install a system.

To review, the Automated Installer is the facility in Solaris 11 Express that supports network-based installation.  The manifest used with AI determines what is installed and how the system is customized during installation.  

Typically, one would set up a network install service and fetch the manifest from there.  However, sometimes this is not desired or practical.  An option in these cases is to use bootable AI.  In this case, you boot the system to be installed from the AI ISO image.  During the boot process, you are prompted for a URL that points to a valid AI manifest.  This manifest is just fetched using HTTP (wget is actually used).  So, so long as you can get to the manifest, you are good to go.  Once fetched, the manifest is validated and acted upon to complete the installation.

In this installment, we will go through the boot process.  In particular, I will show what this looks like on an x86 host.  For details on SPARC, see my previous blog.

So, to start, boot your system from the Automated Install (AI) ISO.  When presented with the Grub menu, the default selection is to use a custom manifest.  This selection is the one we want and will prompt us for the URL of the manifest.  The other options allow you to use the default manifest built into the ISO and to perform the installation across a serial connection.

After the system has booted the small Solaris image on the AI ISO, you will be prompted for the URL of the manifest.

AI fetches the manifest and begins the installation process.  The installation goes on in the background rather than on the console.  Like the LiveCD installation, Solaris is running at this point.  If you want to monitor the progress of the installation, log in as the default admin user included in the ISO image.  The default login name is jack, with the password jack.  Once you have logged in, you can monitor the progress of the installation by tailing the installation log, found in /tmp/install_log.

Since Solaris is up and running, it is possible to enable network logins to the system while it is still installing.  To do this, su to root and enable the ssh service  (svcadm enable ssh).  Once ssh is enabled, you can ssh into the system as jack.  I have sometimes found this to be a useful tool when installing a virtual machine over a slow and unreliable network, where VNC is unable to sustain its required bandwidth.

Once the installation completes, you can reboot the system.  Of course, the regular first-boot SMF import will happen.  And the services that we configured in the last section will be activated to configure the networks and system identity. Once all of this complete, the system is ready for use.  The piece that you might have noticed is missing is any sort of finish-script customization.  Stay tuned for future installments to cover this.

Using the bootable AI, it is simple to provide a manifest via a simple URL and perform a near-hands-free, customized installation of the system.  It is important to note that DHCP is still used to fetch an address for the system, along with routing and DNS information.  For a truly hands-free installation, a network-based install server and install service would have to be created.

Using System Configuration Manifests with Bootable AI

In my last blog, I talked about how to configure a manifest for a bootable AI installation.  The main thing there was how to select which packages to install.  This time we are going to talk about how to handle AI's version of sysidcfg and configuring system identity at install-time.

In a Jumpstart world, many of the things that make up a system's identity - hostname, network configuration, timezone, naming services, etc. - can be configured at installation time by providing a sysidcfg file.  Alternately, an interactive dialog starts and prompts the installer for this sort of information.

The System Configuration manifest provides this same sort of information in an Automated Installer world.  The documentation for AI shows how to create either a separate or an embedded SC manifest to be served by an AI server.  When using Bootable AI, the SC manifest needs to be embedded within the AI manifest.  The SC manifest, whether embedded or not, is basically an XML document that is providing a bunch of properties for SMF services that are going to run on the first system boot to help complete the system configuration.  Some of the main tasks that can be completed in the SC manifest are:

  • Identify and configure the administrative "first user" created at install time.
  • Specify a root password and whether root is a role or a standard user
  • Configure timezone, hostname, keyboard maps, terminal type
  • Specify whether the network should be configured automatically or manually.
  • Configure network settings, including DNS, for manually configured networks

But, in the end, all of this is just setting SMF properties, so it's pretty straightforward.  It appears as a large service_bundle with properties for multiple SMF services.

As far as including the SC manifest information in the bootable AI manifest, the SC manifest is essentially embedded into the AI manifest as a large comment.  Don't be put off by the comment notation.  This whole section is passed on to SMF to assign the necessary service properties.

In order to explain the various sections of properties, I will just annotate an updated SC manifest.  In this manifest, I will specify some of the more common configuration settings you might use. 

The whole SC embedded manifest is identified within the AI manifest with the tag sc_embedded_manifest.  See the Automated Installer Guide for more details on the rest of the options to this tag.  The two lines following the sc_embedded_manifest tag are just the top part of the stand-alone SC manifest XML document.  Look in the default AI manifest for exact placement of this section.

    <sc_embedded_manifest name="AI">
      <!-- <?xml version='1.0'?>
      <!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1">    

The rest of the SC manifest sets up service properties for a service bundled named "system configuration."

      <service_bundle type="profile" name="system configuration">

The service system/install/config is responsible for doing some of the basic configuration actions at install-time, such as setting up the first, or admin, user, setting a root password, and giving the system a name.  The property group "user_account" specifies how the first user, used for administration, should be configured.  You can specify here the username (name="login"), an encrypted password, the GECOS field information (name="description"), as well as the UID and GID for the account.  Note that the default password supplied for the first user (by default, named jack) in the default manifest is "jack".

Special note should be made of the property "roles".  Recall that in Solaris 11 Express, root is no longer a regular login user, but becomes a role.  Therefore, in order to be able to assume the root role for administrative functions, this first user needs to be given the root role.  Other roles can also be specified here as needed.  Also notice that the profile "Primary Administrator" is no longer assigned to this first user, as was done in OpenSolaris.  Additional properties around roles, profiles, authorizations, etc. may be assigned.  See the Automated Installer Guide for details.

        <service name="system/install/config" version="1" type="service">
          <instance name="default" enabled="true">
            <property_group name="user_account" type="application">
              <propval name="login" type="astring" value="sewr"/>
              <propval name="password" type="astring" value="9Nd/cwBcNWFZg"/>
              <propval name="description" type="astring" value="default_user"/>
              <propval name="shell" type="astring" value="/usr/bin/bash"/>
              <propval name="uid" type='count' value='27589'/>
              <propval name="gid" type='count' value='10'/>
              <propval name="type" type="astring" value="normal"/>
              <propval name="roles" type="astring" value="root"/>

As with Jumpstart, it is possible to specify a root password at install time.  The encrypted string for the root password is given here as the password property.  If no new password is supplied, the default root password at install-time is "solaris".  Also note here that root is created as a role rather than a regular login user.

            <property_group name="root_account" type="application">
                <propval name="password" type="astring" value="$5$dnRfcZs$Hx4aBQ161Uvn9ZxJFKMdRiy8tCf4gMT2s2rtkFba2y4"/>
                <propval name="type" type="astring" value="role"/>

A few other housekeeping properties can also be set here for the system/install/config service.  These include the local timezone and the hostname (/etc/nodename) for the system.

            <property_group name="other_sc_params" type="application">
              <propval name="timezone" type="astring" value="US/Eastern"/>
              <propval name="hostname" type="astring" value="myfavoritehostname"/>

The system/console-login service establishes the login service for the console. Here you can specify the terminal type to be used for the console.

        <service name="system/console-login" version="1" type="service">
          <property_group name="ttymon" type="application">
            <propval name="terminal_type" type="astring" value="xterms"/>

The service system/keymap establishes what sort of keyboard input is to be expected on the system.

        <service name='system/keymap' version='1' type='service'>
          <instance name='default' enabled='true'>
            <property_group name='keymap' type='system'>
              <propval name='layout' type='astring' value='US-English'/>

By default, Solaris 11 Express enabled NWAM (NetWork Automagic) to automatically configure the primary network interface.  NWAM, by default, activates a primary network for the system, whether wired or wireless, monitors its availability and tries to restore network connectivity if it should go away.  Most people would say that its behavior is best suited for mobile or desktop systems, and it functions well in that space.  It includes the ability to have profiles that guide its behavior in a variety of networked environments.  NWAM relies on DHCP to get an available IP address and other data needed to configure the network.

In the default AI profile, the network/physical:nwam service instance is enabled and the network/physical:default service instance is disabled.  In most server configurations, static addressing and configuration might be more desirable.  In that case, you can do as we have below and switch with service instance is enabled and which is disabled by default.

        <service name="network/physical" version="1" type="service">
          <instance name="nwam" enabled="false"/>
          <instance name="default" enabled="true"/>

In the case where we are doing static network configuration, we will rely on the network/install service to set up our networks.  The properties and values used here correspond to arguments to the ipadm command, new in Solaris 11 Express.  ipadm is used to configure and tune IP interfaces.  See its man page for details on syntax.

In this case, we are setting up a single IPv4 network interface (xnf0), giving it a static IP address and netmask, and specifying a default route.

        <service name="network/install" version="1" type="service">
          <instance name="default" enabled="true">
            <property_group name="install_ipv4_interface" type="application">
              <propval name="name" type="astring" value="xnf0/v4"/>
              <propval name="address_type" type="astring" value="static"/>
              <propval name="static_address" type="net_address_v4" value=""/>
              <propval name="default_route" type="net_address_v4" value=""/>

As with Jumpstart using sysidcfg, it is possible to set up DNS information at install-time.  Note that only DNS and not NIS or LDAP naming services can be set up this way.  The System Administration Guide: Naming and Directory Services manual discusses how to configure these naming services.  NIS+ is no longer supported in Solaris 11 Express.

The network/dns/install service is used to set up DNS at install-time.  For this, we specify the regular sorts of data that will populate the /etc/resolv.conf file: nameservers, domain, and a domain name search path.  Some of these data items take multiple values, so lists of values are used, as shown below.

        <service name="network/dns/install" version="1" type="service">
          <instance name="default" enabled="true">
            <property_group name="install_props" type="application">
              <property name="nameserver" type="net_address">
                  <value_node value=""/>
                  <value_node value=""/>
                  <value_node value=""/>
              <propval name="domain" type="astring" value=""/>
              <property name="search" type="astring">
                  <value_node value=""/>
                  <value_node value=""/>
                  <value_node value=""/>

And we close out the service bundle and the embedded SC manifest.


So, by building a custom AI manifest with its embedded SC manifest, you can accomplish the same sorts of install-time configuration of a system as you could with Jumpstart and sysidcfg, without having to build any sort of complex finish scripts or any kind of extra coding.  This approach makes is possible to have a repeatable methodology for creating the administrative user, with known, standard credentials, and for configuring the base system networks and naming services.

Sunday Dec 19, 2010

Configuring Bootable AI in Solaris 11 Express

This is the first of several blogs around bootable AI, the ability with the Solaris 11 Express automated installer to boot directly from the AI ISO, fetch an installation manifest, and act on it, without having to set up an AI install server. Most of this focuses on the manifest, so it applies to AI booted from the network and from the ISO. However, I do not plan to go into creating and configuring the AI network services, at least not right now.  I think that other folks have talked about this already.

Solaris 11 Express includes the Automated Installer as the tool for performing automated network system installations. Using AI, it is possible to install a customized load of Solaris 11 Express, across the network, without manual intervention. AI allows you to specify which repositories to use, which packages to install from the repositories, and how to handle the initial system configuration (like sysidcfg) for the system.

Generally speaking, AI relies on a network automated install service to answer requests from a client trying to install it self. This is sort of like the jumpstart approach. The client starts to boot, looks around to see who on the network can help out, fetches what it needs from that network server, and goes ahead with the installation.

But, getting the base install information from the network isn't always feasible or even the most expeditious path. So, AI has a feature called "Bootable AI". I've blogged about this before, when it first came out in OpenSolaris.

The idea of bootable AI is that rather than rely on a network service to fetch the needed information to boot, you boot from the AI media instead. During the boot process, you can be prompted for a URL where an installation manifest can be found. The client fetches this manifest and carries on, just as it would from the AI service. The upside to this is that no AI service has to be installed on the network. The downside is that it does require at least one interaction if you want to specify a non-default manifest.

The manifest is the XML specification of how and what to install on the system. There is quite a lot that can be done in terms of selecting which disks to use, how to partition them, etc. Check the Oracle Solaris 11 Express Automated Installer Guide for detailed information on this.

In this note, I am going to show you how to put together some of the key parts of the manifest. My main goal is to then use this with bootable AI, but the same manifest can be installed with an install service in an AI server.

Locating a default manifest

How to get started? The simplest way to make a manifest is to start with the default manifest, delivered in the AI iso, and modify it to suit your needs. A default manifest is located in auto_install/default.xml in the AI ISO. Copy this and modify it as needed.

Selecting "Server Install" packages

When Solaris 11 Express is installed via the LiveCD or via the default manifest in AI, a full, desktop version of Solaris is installed. Often, however, when you use AI, you would prefer to have the smaller server installation provided by the text installer. Since the manifest specifies which top-level packages to install, this is easily accomplished.

In the default manifest, look for the software_data section with the install action. This section specifies what packages are to be installed. The two packages listed here are group packages, sort of like package clusters in Solaris 10. entire and babel_install are the packages that, when installed, provide the environment installed from the LiveCD. In order to get a reduced installation like that from the text installer, replace babel_install with server_install. If there are other packages that you want to add to the installation (for example the iSCSI packages referenced in the comments), you can add them here.

Change this section:

<software_data action="install" type="IPS">

to this section:

<software_data action="install" type="IPS">

Uninstalling the appropriate packages

The server_install package bundle has dependencies of the packages that make up the reduced server installation. By installing it, we get all of the other packages that come with it. That's part of the coolness of IPS. However, we also want to preserve the ability to uninstall or modify individual components of that overall bundle. So, we finish out our installation by uninstalling the server_install wrapper. This does not affect the dependent packages; it just unwraps them so we can modify them directly. So, to do this, update the uninstall section as below.

Additionally, even with the reduced server installation, there may still be packages that we want to remove. For example, there are still over 700MB of system locales installed that you may not need and might choose to remove. You can add any other packages that you want to remove in this section as well. Note that this really does first install the package and then remove it. Seems sort of redundant, but I have not yet found a way to cause IPS to build a plan that would note the uninstalled packages and just mark them to be skipped during installation.

Change this:

        babel_install and slim_install are group packages used to
        define the default installation.  They are removed here so
        that they do not inhibit removal of other packages on the
        installed system.
      <software_data action="uninstall" type="IPS">

to this:

<software_data action="uninstall" type="IPS">
        <!-- ... -->

Any other packages that you want to uninstall can be listed here, too.

So, you see how easy it is to build a manifest for AI that specifies which packages you want to include or exclude and how to create a smaller, server installation for Solaris 11 Express.


Interesting bits about Solaris, Virtualization, and Ops Center


« July 2016