Distribution Constructor

One of the features that shipped with OpenSolaris 2008.11 was the ability to create your own custom distributions of the product. This is pretty powerful if you want to deploy OpenSolaris throughout your organization tailored to your users' needs. 

The task of creating your own distribution is handled by the Distribution Constructor. As an example of how to use the Distribution Constructor, I'll resurrect a product some of you may remember: Solaris Express Developer Edition or SXDE (not to be confused with SXCE - rather an SXCE superset). 

SXDE managed four releases before ultimately meeting its demise when OpenSolaris was first released in May of 2008. SXDE was a distribution targeted toward developers, containing popular software such as Apache, MySQL, PHP, NetBeans, Sun Studio and OpenOffice.org. So my objective will be to create "OpenSolaris Developer Edition". Of course it won't fit on a CD, but I will end up with images from which I can create both a live DVD and USB.

Before we begin, it's worth noting that the process of building a distribution can be quite time consuming. All the packages need to be downloaded from the repository, customized and repackaged.  I highly recommend setting up a local repository to at least minimize the impact of the package installation phase of the process.

Step 1: Install the Distribution Constructor Files

The first step toward using the Distribution Constructor is to install the SUNWdistro-const package:

bleonard@opensolaris:~$ pfexec pkg install SUNWdistro-const
DOWNLOAD                                    PKGS       FILES     XFER (MB)
Completed                                    1/1       75/75     0.19/0.19 

PHASE                                        ACTIONS
Install Phase                                104/104 

Step 2: Copy and Customize the Files

The Distribution Constructor comes with everything you need to create an exact replica of OpenSolaris 2009.06. The configuration files and scripts are located in /usr/share/distro_const/slim_cd. So you could simply run distro_const build /usr/share/distro_constr/slim_cd/all_lang_slim_cd_x86.xml, wait several hours, and you'd end up with the same thing you can download from opensolaris.com.

To make our custom distribution, we're going to start by copying the all_lang_slim_cd_x86.xml to our home directory. I'm also changing the file name to match our use case:

bleonard@opensolaris:~$ cp /usr/share/distro_const/slim_cd/all_lang_slim_cd_x86.xml all_lang_osde_x86.xml
bleonard@opensolaris:~$ chmod +w all_lang_osde_x86.xml 

Step 3: Edit the Manifest

Editing the manifest files is very well documented (the docs can also be found in /usr/share/doc/distro_const). For our distribution, we're going to edit the following sections:

The name of our distribution:

<distribution name="OpenSolaris Developer Edition">

As I've stated earlier, to improve download performance, I'm using a local package repository:

<pkg_repo_default_authority>
        <main
                url="http://localhost:81"
                authname="opensolaris.org"/>

I'm adding 4 additional packages (amp-dev will bring in the bulk of the software). Tailor however you like:

<packages>
       <pkg name="entire"/>
       <pkg name="SUNWcsd"/>
       <pkg name="SUNWcs"/>
       <pkg name="babel_install"/>
       <pkg name="SUNWslim-utils"/>
       <pkg name="amp-dev"/>
       <pkg name="glassfishv2"/>
       <pkg name="sunstudio"/>
       <pkg name="openoffice"/>
</packages>

 Finally, I'll customize the grub menu title:

<title>OpenSolaris 2009.06 Developer Edition</title>

Step 4: Build the Custom Distribution

To build the distribution, we're going to use the distro_const command along with our customized manifest:

bleonard@opensolaris:~$ pfexec distro_const build all_lang_osde_x86.xml 
/usr/share/distro_const/DC-manifest.defval.xml validates
/tmp/all_lang_osde_x86_temp_5613.xml validates
Simple Log: /rpool/dc/logs/simple-log-2010-08-05-14-47-53
Detail Log: /rpool/dc/logs/detail-log-2010-08-05-14-47-53
Build started Thu Aug  5 14:47:53 2010
Distribution name: OpenSolaris Developer Edition
Build Area dataset: rpool/dc
Build Area mount point: /rpool/dc
==== im-pop: Populate the image with packages
Initializing the IPS package image area: /rpool/dc/build_data/pkg_image
Setting preferred authority: opensolaris.org
	Origin repository: http://localhost:81
Verifying the contents of the IPS repository
Installing the designated packages

The Distribution Constructor will remain on the "Installing the designated packages" step for a while. I recommend tailing the Detail Log highlighted above if you want more information on what exactly is going on.

Once package installation completes, it will continue, producing output like the following as it progresses through it's various stages (checkpoints):

Uninstalling the designated packages
Setting post-install preferred authority: opensolaris.org
	Origin repository: http://pkg.opensolaris.org/release
==== im-mod: Image area modifications 
==== slim-im-mod: Slim CD Image area Modifications 
0 blocks
WARNING: failed to parse type name `str'
WARNING: invalid or missing type for schema (/schemas/apps/packagemanager/preferences/remote_search_error)
WARNING: Failed to parse default value `கருப்பொருளை மாற்று ;gtk-theme-selector.desktop,முன்னிருப்பு பயன்பாடுகள் அமை;default-applications.desktop,அச்சுப்பொறி சேர்;gnome-cups-manager.desktop]' for schema (/schemas/apps/control-center/cc_actions_list)
gtk-update-icon-cache: Cache file created successfully.
gtk-update-icon-cache: Cache file created successfully.
gtk-update-icon-cache: Cache file created successfully.
gtk-update-icon-cache: Cache file created successfully.
gtk-update-icon-cache: Cache file created successfully.
gtk-update-icon-cache: Cache file created successfully.
gtk-update-icon-cache: Cache file created successfully.
gtk-update-icon-cache: Cache file created successfully.
gtk-update-icon-cache: Cache file created successfully.
gtk-update-icon-cache: Cache file created successfully.
gtk-update-icon-cache: Cache file created successfully.
gtk-update-icon-cache: Cache file created successfully.
==== br-init: Boot root initialization 
235264 blocks
2800 blocks
31040 blocks
0 blocks
92560 blocks
3696 blocks
0 blocks
0 blocks
576 blocks
32 blocks
160 blocks
2832 blocks
272 blocks
384 blocks
11344 blocks
0 blocks
0 blocks
==== slim-br-config: Slim CD boot root configuration 
==== br-config: Boot root configuration 
==== br-arch: Boot root archiving (64-bit) 
397392 blocks
283232 blocks
==== br-arch-32: Boot root archiving (32-bit) 
397392 blocks
205552 blocks
==== slim-post-mod: Slim CD post bootroot image area modification 
==== grub-setup: Grub menu setup 
==== post-mod: Post bootroot image area modification 
Warning: creating filesystem that does not conform to ISO-9660.
Warning: creating filesystem that does not conform to ISO-9660.
==== gen-slim-cont: Generate Slim CD image content list 
==== iso: ISO image creation 
Warning: creating filesystem that does not conform to ISO-9660.
Setting input-charset to 'UTF-8' from locale.
Size of boot image is 4 sectors -> No emulation
  0.68% done, estimate finish Thu Aug  5 17:57:39 2010
  1.36% done, estimate finish Thu Aug  5 17:57:39 2010
  2.05% done, estimate finish Thu Aug  5 17:57:39 2010
...
 98.20% done, estimate finish Thu Aug  5 17:59:06 2010
 98.88% done, estimate finish Thu Aug  5 17:59:05 2010
 99.56% done, estimate finish Thu Aug  5 17:59:06 2010
Total translation table size: 2048
Total rockridge attributes bytes: 31523
Total directory bytes: 200704
Path table size(bytes): 1282
Max brk space used 82000
733225 extents written (1432 MB)
==== usb: USB image creation 
/dev/rlofi/2:	3519000 sectors in 5865 cylinders of 1 tracks, 600 sectors
	1718.3MB in 184 cyl groups (32 c/g, 9.38MB/g, 2368 i/g)
super-block backups (for fsck -F ufs -o b=#) at:
 32, 19232, 38432, 57632, 76832, 96032, 115232, 134432, 153632, 172832,
 3340832, 3360032, 3379232, 3398432, 3417632, 3436832, 3456032, 3475232,
 3494432, 3513632
2932288 blocks
Build completed Thu Aug  5 18:24:38 2010
Build is successful.

Step 5: Use the Media

At this point we have both an ISO and a USB image:

bleonard@opensolaris:~$ ls -lh /rpool/dc/media/
total 2.8G
-rw-r--r-- 1 root root 1.4G 2010-08-05 17:59 OpenSolaris Developer Edition.iso
-r--r--r-- 1 root root 1.7G 2010-08-05 18:24 OpenSolaris Developer Edition.usb

The ISO image can be burned to a DVD or loaded into a hypervisor such as VirtualBox. Note the customized grub menu:


And once the Live DVD is loaded you can see it contains our software:




Step 6: Customizations

Throughout the generation process, the distribution constructor takes checkpoints. These are seen in the output above as:

==== <step>: <description>

For example:

==== grub-setup: Grub menu setup 

You can view the checkpoints as follows:

bleonard@opensolaris:~$ pfexec distro_const build -l all_lang_osde_x86.xml 
/usr/share/distro_const/DC-manifest.defval.xml validates
/tmp/all_lang_osde_x86_temp_11516.xml validates

Step           Resumable Description
-------------- --------- -------------
im-pop             X     Populate the image with packages
im-mod             X     Image area modifications
slim-im-mod        X     Slim CD Image area Modifications
br-init            X     Boot root initialization
slim-br-config     X     Slim CD boot root configuration
br-config          X     Boot root configuration
br-arch            X     Boot root archiving (64-bit)
br-arch-32         X     Boot root archiving (32-bit)
slim-post-mod      X     Slim CD post bootroot image area modification
grub-setup         X     Grub menu setup
post-mod           X     Post bootroot image area modification
gen-slim-cont      X     Generate Slim CD image content list
iso                X     ISO image creation
usb                X     USB image creation

Under the covers you can see that the checkpointing is all handled by ZFS:

bleonard@opensolaris:~$ zfs list -r -t all -s creation rpool/dc
NAME                                       USED  AVAIL  REFER  MOUNTPOINT
rpool/dc                                  7.55G  35.1G   123K  /rpool/dc
rpool/dc/logs                               63K  35.1G    63K  /rpool/dc/logs
rpool/dc/media                            2.80G  35.1G  2.80G  /rpool/dc/media
rpool/dc/build_data                       4.75G  35.1G  1.52G  /rpool/dc/build_data
rpool/dc/build_data@empty                     0      -    23K  -
rpool/dc/build_data@.step_im-pop              0      -    23K  -
rpool/dc/build_data@.step_im-mod              0      -  2.97G  -
rpool/dc/build_data@.step_slim-im-mod         0      -  2.97G  -
rpool/dc/build_data@.step_br-init         2.64M      -  3.00G  -
rpool/dc/build_data@.step_slim-br-config    95K      -  3.11G  -
rpool/dc/build_data@.step_br-config         75K      -  3.11G  -
rpool/dc/build_data@.step_br-arch         9.33M      -  3.12G  -
rpool/dc/build_data@.step_br-arch-32      15.7M      -  3.25G  -
rpool/dc/build_data@.step_slim-post-mod   21.2M      -  3.35G  -
rpool/dc/build_data@.step_grub-setup        54K      -  3.34G  -
rpool/dc/build_data@.step_post-mod          57K      -  3.34G  -
rpool/dc/build_data@.step_gen-slim-cont     28K      -  1.67G  -
rpool/dc/build_data@.step_iso               13K      -  1.67G  -
rpool/dc/build_data@.step_usb               13K      -  1.67G  -

So say, for example, you wanted to edit the grub menu title. After doing so in the manifest file you could resume construction at that step as follows:

bleonard@opensolaris:~$ pfexec distro_const build -r grub-setup all_lang_osde_x86.xml 
/usr/share/distro_const/DC-manifest.defval.xml validates
/tmp/all_lang_osde_x86_temp_3244.xml validates
WARNING: The manifest file, all_lang_osde_x86.xml, has changed 
since Step(s)  im-pop im-mod slim-im-mod br-init slim-br-config br-config br-arch br-arch-32 slim-post-mod grub-setup was generated.
Results may be indeterminate.
...

(Optional) Step 7: Preparing the Live USB

One of the utilities installed with the distribution constructor package is a shell script, usbcopy, for creating a live USB:

bleonard@opensolaris:~$ pfexec usbcopy /rpool/dc/media/OpenSolarisDeveloperEdition.usb 
Found the following USB devices:
0:	/dev/rdsk/c4t0d0p0	<Unknown> MATSHITA	DVD-RAM UJ-844S  1.11
1:	/dev/rdsk/c6t0d0p0	305.2 GB	WD       3200BMV External 1.75
2:	/dev/rdsk/c8t0d0p0	1.9 GB	SMI      USB DISK         1100
Enter the number of your choice: 2

WARNING: All data on your USB storage will be lost.
Are you sure you want to install to
SMI USB DISK 1100, 1900 MB at /dev/rdsk/c8t0d0p0 +  (y/n) y
Copying and verifying image to USB device
Finished 1718 MB in 455 seconds (3.7MB/s)
0 block(s) re-written due to verification failure
Installing grub to USB device /dev/rdsk/c8t0d0s0
Completed copy to USB

Note, I had trouble passing in the file name with spaces to the usbcopy script, so I first renamed it to remove them:

bleonard@opensolaris:~$ pfexec mv /rpool/dc/media/OpenSolaris\\ Developer\\ Edition.usb /rpool/dc/media/OpenSolarisDeveloperEdition.usb

 
  

So, why kind of custom distributions will you create?

Comments:

That's an interesting feature. I translated this one to Russian:
http://solaris11.ru/lib/opensolaris_distribution_constructor/

Posted by Gleb on August 09, 2010 at 09:51 AM GMT #

Awesome, thanks Gleb!

Posted by Brian Leonard on August 09, 2010 at 09:57 AM GMT #

I tried installing Solaris 11 on a Sun Blade 100 today, onto a 500GB disk, which previously held Ubuntu, and watched the installer froth at the mouth, and then die, at the thought that the disk label might be telling the truth. Why is it, that even the trashiest SPARC port of Linux can correctly partition a 500GB disk, while Solaris throws up its hands in horror, and claims the disk label is corrupt? As far as I can see, the trouble appears to be, that number of 'cylinders' is held in the 'format' utility and its derivatives, in a short int and, if you try to describe the disk by hand, it rolls over, if you try to enter a number larger than it can hold. How long would this take to fix, by changing the datatype to an int? Or are we still pretending that there are no disks larger than 130GB? Alternatively, if it's too hard, why not use fdisk/parted or anything else that's not afraid of large numbers?

Posted by Mark Sitkowski on December 15, 2010 at 08:58 PM GMT #

@Mark, without having researched the specifics of the SB100, you may well be running into firmware limitations of that system. Other SPARC systems of that vintage that I'm more familiar with were limited to 130 GB disks.

Posted by Dave Miner on December 27, 2010 at 12:35 PM GMT #

Post a Comment:
  • HTML Syntax: NOT allowed
About

The Observatory is a blog for users of Oracle Solaris. Tune in here for tips, tricks and more as we explore the Solaris operating system from Oracle.

Connect with Oracle Solaris:


Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today