Installing a Debian Zone with BrandZ

Background

One of the reasons we released BrandZ to the community before integrating with Solaris, was the hope that people would use the framework to create their own brands.

Less complex than creating a completely new brand is extending the lx brand to support other Linux distros. The simplest case is supporting a distro based on the same kernel/glibc version as our original RHEL/CentOS brand. To support such a distro, in theory we just need to create an installer for the new distro.

There are two distros that I have been asked about multiple times: Debian and Ubuntu. Both are based on the Debian package format, instead of RPM like the distros we currently support. I wanted to install Ubuntu, since its emphasis on getting wide distribution means it has a much smaller learning curve than Debian. Unfortunately (in this case), Ubuntu is also much more agressive about upgrading their core components. Debian still has an actively used version ('sarge') based on the kernel/glibc revs currently supported by BrandZ, while Ubuntu does not. So, Debian it was.

What follows is a description of the steps I went through to get Debian 'sarge' installed and running in a zone on Solaris. This isn't meant as a tutorial on how to perform a similar feat on your own system, since (as will become clear), it's really a pain in the neck. Rather, this is offered as a starting point (or possibly a cautionary tale) for anybody interested in writing a new install script for Debian or any other distro.

Zone configuration and layout

The first step, as always, is to configure the new Linux zone:

        crunchyfrog# zonecfg -z debian
        debian: No such zone configured
        Use 'create' to begin configuring a new zone.
        zonecfg:debian> create -B lx
        zonecfg:debian> set zonepath=/export/zones/debian
        zonecfg:debian> add net
        zonecfg:debian:net> set physical=bge0
        zonecfg:debian:net> set address=10.8.28.84
        zonecfg:debian:net> end
        zonecfg:debian> commit
        zonecfg:debian> exit

Next we do the install. Normally this process actually installs the Red Hat or CentOS bits into your zone. In this case, I wanted a blank slate on which to install the Debian software, so I used the 'tarball' install option to unpack an essentially empty tarball:

        crunchyfrog# cd tmp/
        crunchyfrog# mkdir usr
        crunchyfrog# tar cf dummy.tar usr
        crunchyfrog# zoneadm -z debian install -d /tmp/dummy.tar

With this done, I now had a Linux zone that was nominally in the 'installed' state, but it didn't actually have any software in it.

Installing Debian

When installing Red Hat or CentOS, we use the RPM software in the SFWrpm package. Debian uses a different packaging format, which we don't have any tool for manipulating. Fortunately, there is another OpenSolaris project which is working on creating a new Solaris distro built around this packaging format: Nexenta. I grabbed the version of dpkg they had built for Solaris, and I was ready to start installing.

Since I already had a sarge 'netinstall' CD handy, that's the medium I decided to install from. It can downloaded from the debian site here: debian-31r1a-i386-netinst.iso.

Getting to single-user mode

Through a little trial and error, I settled on these as the minimal set of packages needed to boot a zone that is capable of further self-installation:

        libc6, libacl1, libattr1, dpkg, dselect, tar, grep, perl, sed,
        find, mawk, gzip, zlib1g, libuuid1, debianutils, coreutils, procps,
        passwd, login, sysvinit, initscripts, sysv, util-linux, bsdutils,
        mount, base-passwd, base-files, bash, hostname, libncurses5,
        libpam0g, libpam-modules, libpam-runtime

It turns out that dpkg is much more insistent on running pre/post-install scripts than RPM. So much so that even an 'unpack' operation requires running a pre-install script. So, I decided to break the process into two parts: 'dpkg-deb --extract' and then do a real install once the zone was booted.

        crunchyfrog# for i in `cat pkgs`; do dpkg-deb --extract $i /export/zones/debian/root/; done

Since this method ensures that the install scripts aren't run, we have to do some of their work by hand before the zone will boot. Again, this was done largely by trial and error. The RHEL / CentOS installer we ship as part of lx does this work in the setup_lx_env script invoked by the installer.

        1. crunchyfrog# cd /export/zones/debian/root
           crunchyfrog# cp usr/share/sysvinit/inittab etc/
           crunchyfrog# cp usr/share/base-passwd/passwd.master etc/passwd
           crunchyfrog# cp usr/share/base-passwd/group.master etc/group
           crunchyfrog# cp usr/share/passwd/shells etc/shells

           crunchyfrog# cat > etc/rcS.d/S20proc
           #!/bin/bash
           echo "Mounting /proc"
           mount -n -t proc /proc /proc
           \^C
           crunchyfrog# chmod +x etc/rcS.d/S20proc
           crunchyfrog# cat /etc/fstab
           none /       ufs     defaults        1 1
           none /proc   proc    defaults        0 0

2. I also modified inittab to eliminate the gettys running on the virtual consoles (which Solaris doesn't support) and add one on the zone console.

           [...]
           1:2345:respawn:/sbin/getty 38400 console
           #2:23:respawn:/sbin/getty 38400 tty2
           #3:23:respawn:/sbin/getty 38400 tty3
           #4:23:respawn:/sbin/getty 38400 tty4
           #5:23:respawn:/sbin/getty 38400 tty5
           #6:23:respawn:/sbin/getty 38400 tty6
           [...]

3. Finally, there are a few links that need to be created because the zones framework expects the files to be in one place while Linux puts them somewhere else:

           ln -s /bin/sh sbin/sh
           ln -s /bin/su usr/bin/su
           ln -s /bin/login usr/bin/login

We now have a zone that can boot to single user mode:

        crunchyfrog# zoneadm -z debian boot -s
On the debian console we see:
        [NOTICE: Zone booting up]
        INIT: version 2.86 booting
        /etc/init.d/rcS: line 27: /etc/default/rcS: No such file or directory
        Press enter for maintenance
        (or type Control-D to continue):
        root@debian:~# uname -a
        Linux debian 2.4.21 BrandX fake linux i686 GNU/Linux

Housekeeping

Now to install the rest of the software. I started by copying the list of already-installed packages into the zone, and by making the mounted CD image visible as well:

        crunchyfrog# cp /tmp/pkgs /export/zones/debian/root/tmp/
        crunchyfrog# mount -F lofs /mnt/ /export/zones/debian/root/mnt/

The first thing I did was ensure sanity in the dpkg database by reinstalling all of the packages I had extracted before. Most of the following dpkg operations spit out a number of warnings, basically reflecting the fact that the environment is still insane.

To make dpkg happy I had to create a few files.

        debian# touch /var/lib/dpkg/status
        debian# touch /var/lib/dpkg/available

I then installed two packages 'by hand':

        debian# dpkg -i --force-depends /mnt/pool/main/d/dpkg/dpkg_1.10.28_i386.deb
        debian# dpkg -i --force-depends /mnt/pool/main/g/glibc/libc6_2.3.2.ds1-22_i386.deb

As part of its install process, libc interactively sets the timezone. Thus the need to install the package this way. Presumably this could be automated, but that's a problem for another day.

Next I did a bulk (re)install of all the other packages we added to the zone prior to booting.

        debian# for i in `cat /tmp/pkgs`; do dpkg -i --force-depends $i; done

We now have a working debian zone with a consistent dpkg database. dpkg -l should list all of the installed software and dpkg --audit shouldn't report any problems.

One final bit of cleanup is needed. In the process of properly installing all those packages, /etc/rcS.d was populated with a bunch of startup scripts that we don't want to run. The easiest way to deal with this is:

        debian# cd /etc/rcS.d
        debian# mv S20proc /tmp
        debian# rm \*
        debian# mv /tmp/S20proc .

This is a pretty heavy-handed operation, but it works because we don't have any necessary services installed yet. All we're wiping out are the low-level, hardware-config kind of scripts that we don't need inside a zone.

Preparing to apt-get the universe

The next step was to start loading the bulk of the software into the zone. My goal was to be able to use apt-get to get as much as possible off of the net, but obviously I needed to install at least enough stuff to get networking working first.

        libgcc1
        gcc-3.3-base
        libstdc++5
        apt
        net-tools
        debconf [1]
        debconf-english
        ifupdown
        netkit-inetd
        netkit-ping
        libwrap
        tcpd
        netbase

With these packages installed and my /etc/resolv.conf configured properly, I could now run apt-get to install any additional software into the zone.

First up: satisfy any lingering dependencies for the packages I already manhandled into place:

        root@debian:~# apt-get -f install
        Reading Package Lists... Done
        Building Dependency Tree... Done
        Correcting dependencies... Done
        The following extra packages will be installed:
          e2fslibs e2fsprogs initscripts libblkid1 libcap1 libcomerr2
          libdb1-compat libdb3 libdb4.2 libss2 slang1a-utf8
        Suggested packages:
          gpart parted e2fsck-static
        The following NEW packages will be installed:
          e2fslibs e2fsprogs initscripts libblkid1 libcap1 libcomerr2
           libdb1-compat libdb3 libdb4.2 libss2 slang1a-utf8
        0 upgraded, 11 newly installed, 0 to remove and 0 not upgraded.
        1 not fully installed or removed.
        Need to get 1685kB of archives.
        After unpacking 4497kB of additional disk space will be used.
        Do you want to continue? [Y/n]

Now we can install whatever applications we intend to run, and rely on apt-get to resolve any dependencies for us. For example, I want to run the IRC client 'xchat', so I do:

        root@debian:/# apt-get install xchat

The tool identifies the 41 packages I need to install before this will work, downloads all 25MB of .deb files, and installs them for me.

Lather, rinse, repeat until you have a zone that does everything you need it to.

Conclusion and Caveats

After all the work I went through to get the zone installed, I haven't actually used it very much. Our focus is primarily on Red Hat, since that's the distro we expect to formally support, so the Debian zone isn't getting a lot of use.

The only problems I found with the zone which are not resolved in our Build 31 release revolve around threading. Due to the completely different approaches Linux and Solaris take to threading, this has been a particularly tricky area for us to get right. It appears that Debian is still using the old linuxthreads model, while Red Hat has moved ahead to NPTL. In the unlikely event that I have some free time, I might investigate the linuxthreads issues. Otherwise, if you're trying to run a Debian zone and run into problems, this is probably the first place you should look.

As I said at the top, this really isn't a process that most people are going to want to go though manually. Bootstrapping any operating system is ugly, and trying to do it on top of a different operating system is doubly so.

My original goal was just to work through the install process to help light the way for other people, but I ended up uncovering a handful of bugs as well. The libraries and apps available on Debian are different enough from the CentOS code we've been using that they poke our emulation support in new and different ways. Since it revealed bugs that would have bitten us at some point, this has turned out to be a more generally useful exercise than I had originally expected.


[1] Something funny was going on with perl's flock(). When trying to configure debconf, it kept complaining that the configuration database was locked by another user. At first glance it appears that this is a bug in either perl or debconf. The file is being opened read-only, but we are trying to get a write-lock on it. That's not allowed in Solaris, but if it is allowed in Linux, then this is a bug in our emulation. Temporary workaround: comment out the flock() calls in

/usr/share/perl5/Debconf/DbDriver/File.pm
.

Technorati Tags:

Comments:

This post is a life saver. I'm operating an ASP, and I need a Solaris kernel so that I can use Doors in my app, as Linux has never accepted the Doors patch -- but I can't leave Debian userspace behind, as the robustness and high support level of Debian is pre-eminent. If this tests cleanly, I'll be able to get Doors and DTrace too, while remaining secure with my comfortable, stable, milk-safe standard-conforming Sarge world, and keeping my cutting-edge web platform from dotdeb.org, RoR, &c. I can't imagine why you're interested in RHEL/CentOS. Perhaps it's a partnering thing. Corporations like to make deals with other corporations. But RH is much in disfavor among the commercial users I know, who migrated without exception to Debian as RH8/9 become obsolete and Sarge stabilized a couple of years ago.

Posted by This is a lifesaver, if it works out on January 12, 2006 at 06:47 PM GMT-05:00 #

Is there a reason you dont use debootstrap for this?

Posted by Martin on January 13, 2006 at 01:40 AM GMT-05:00 #

Post a Comment:
Comments are closed for this entry.
About

nilsn

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today