Tuesday Apr 29, 2014

New Format for Solaris IPS Repository Downloads

I haven't contributed anything to this blog in a while, mostly because I haven't contributed anything directly to Solaris in a while. With the release of Oracle Solaris 11.2 Beta, I now have something to write about.

In prior Solaris 11 releases, Oracle offered the downloadable version of the Solaris 11 ISO repository in different formats depending on which download site was being used (Oracle Technical Network, My Oracle Support, etc.). This was due to varying file size limitations on the different sites, as the repository has grown beyond the 4GB boundary. One site contained a single ISO file that was split into several pieces, while another had 2 separate ISO files that had to be mounted and copied to a single repository location. README instructions varied as well. And while offering the entire repository on OTN as a single download sounded like a simple solution, it caused issues on some client systems, depending on the operating system, browser and filesystem.

These varied formats also contributed to service calls, with claims that the ISO images were broken. In reality, it was just that the wrong method was often used for the particular download site. It was an "easy fix" to tell customers that the procedure they used for the media pack was different than for the same bits on My Oracle Support, but we could see why the problem occurred and how much time was spent in unnecessary extra downloads.

Starting with Solaris 11.2, we are now offering a single format for the repository images. Abandoning the ISO format altogether, the repository is now a set of zipped archive files. This format solves the issues mentioned above with the varying formats:

  • each segment is less than 2GB, which allows it to be downloaded on a variety of systems
  • if a single segment fails to download, the entire repo download does not need to be restarted
  • it eliminates the extra steps of combining segments and mounting or burning the resulting ISO
  • customers downloading from My Oracle Support get the same format as OTN customers who get the same format as Oracle Software Delivery Cloud media packs - a single set of instructions for all
  • repository images can continue to grow without concern for any media size limits

In addition to the format change, an installation script is also now available to unzip each segment into the designated repository filesystem. Prior to unzipping, it will check if the target is an existing repository. If it is, a warning will be given with the current level of Solaris contained in the repository. The user can choose to add the new repository packages to the current repository, or quit the application and choose a different target. This will help to prevent accidentally adding software to the wrong local repository. 

The script has a few additional options to make the work a bit easier:

  • validation of the md5 sums of the downloaded zip files
  • validation of the repository after unzipping
  • creation of an ISO image for portability

With the combination of the changed download format and the script to apply the downloads, I think this solution will work out for all customers regardless of system type, Internet connection, or contract status. I won't bother with the cliche "one download format to rule them all."

 For detailed documentation of creating and maintaining repositories, look for these documents on docs.oracle.com:

 Copying and Creating Package Repositories in Oracle Solaris 11.2

 How to Copy a Repository From a zip File

Monday Apr 09, 2012

IPS Package Groups

IPS group packages consist solely of dependencies on other packages that make up a logical grouping of software. These are similar to, but not the equivalent of, Solaris 10 metaclusters. The main difference is that metaclusters are nested subsets ranging from a minimal install to nearly all packages on the media. Group packages have no such hierarchy. They can overlap other groups, or be completely disjoint sets.

A group dependency is set this way in an IPS package manifest file:

    depend fmri=full/pkg/name type=group

Current Solaris Groups

Solaris currently has 4 system groups defined. These are used for different types of installation, and are included in the xml manifest files used by the various Solaris installers:

Package Name

Summary

Description

Default Installation For:

 group/system/solaris-desktop

Oracle Solaris Desktop

Provides an Oracle Solaris desktop environment

Live Media

 group/system/solaris-large-server

Oracle Solaris Large Server

Provides an Oracle Solaris large server environment

Text Installer from media

Automated Installer

 group/system/solaris-small-server

Oracle Solaris Small Server

Provides a useful command-line Oracle Solaris environment

 Zones

 group/system/solaris-auto-install

 Oracle Solaris Automated Installer Client

 Provides an Oracle Solaris Automated Installer client

Text Installer from network

There are also several "feature" groups such as AMP and GNU Developer Tools. These are provided for convenience, but are not used directly by any installers.

Retrieving Group Package Information

A listing of all current groups can be found with the command:

pkg info -r group/*

A listing of all the packages in a group can be obtained with:

pkg contents -o fmri -H -rt depend -a type=group groupname

An example:

$ pkg contents -o fmri -H -rt depend -a type=group solaris-desktop archiver/gnu-tar audio/audio-utilities codec/flac codec/libtheora codec/ogg-vorbis codec/speex communication/im/pidgin etc.

You can determine which package group is currently installed on your system:

$ pkg list group/system/\*  

Output would look like:

NAME (PUBLISHER)                                  VERSION                    IFO
group/system/solaris-desktop                      0.5.11-0.175.0.0.0.0.0    i--

Note that there are not version numbers associated with a group package dependency. The package version that best fits the system will be used, based on other dependencies such as what is listed in incorporation files.

Installing a Group

To Install a group, simple use the group package name as you would any other package:

$ pkg install solaris-small-server

 If you want to exclude a package from installing, you can use the --reject flag:

$ pkg install --reject audio/audio-utilities solaris-desktop

Creating Your Own Group

To create your own group package, you can follow the pkg(5) documentation on how to create a package, and use this action for each package that is part of your group:

    depend fmri=full/pkg/name type=group 

Friday Feb 10, 2012

Solaris and VirtualBox: No Heads Is Better Than One

VirtualBox is a great method of running Solaris in a virtualized environment. However, it shares resources with the host system, and a laptop with 4GB of RAM can't afford to give up too much memory. This is why I usually will run VirtualBox from a server with more RAM and diskspace (and a faster CPU is always nice).

The VirtualBox GUI is easy to use, but trying to run it remotely from the server onto my laptop lacks quite a bit in performance. I'm not a graphics expert, so I can't say why. But I do know that one solution for overcoming remote GUI performance is to use VNC. That allows a whole virtual desktop session to be run on the server. But it also requires running a VNC client locally, and it is a lot of overhead for just being able to run the single VirtualBox application.

Using VBoxManage to Create a VM

I found the best solution for my work is to use the VirtualBox command line interfaces. These will let you work in a terminal window on the server, and just display the virtual console on your local system. The GUI is completely bypassed. The two commands are VBoxManage and VBoxHeadless. The VirtualBox user guide has all the details, but here is how I use them to automate creation of a Solaris virtual machine.

The basic steps for creating a virtual machine (VM) using VBoxManage are as follows:

  • create a virtual hard drive
  • create a VM
  • add memory, NIC and a disk controller
  • link the virtual hard drive with the VM
  • mount an ISO install image

Here is a simple script with the options I used for Solaris 11. These values could be used for Solaris 10 as well, though RAM requirements may differ slightly. It assumes that the ISO image is in the current directory, and it creates the VDI file in the current directory. This example uses the Solaris 11 Live Media (desktop) image, but the other Solaris 11 installation ISOs can also be used, as well as Solaris 10 images.

VMNAME=solaris11
VBoxManage createhd --filename $VMNAME.vdi --size 64000
VBoxManage createvm -name "$VMNAME" -register
VBoxManage storagectl  "$VMNAME" --name "IDE Controller" --add ide
VBoxManage modifyvm  "$VMNAME" --memory "2048" --acpi on --boot1 dvd \
--nic1 nat --pae on --hda "$VMNAME.vdi"
VBoxManage storageattach "$VMNAME" --storagectl "IDE Controller" --port 0 \
--device 1 --type dvddrive --medium sol-11-1111-live-x86.iso

Using the VM with VBoxHeadless

Now you have a VM that can be run remotely using VBoxHeadless. In the server terminal window, type:

$ VBoxHeadless -s solaris11 
Oracle VM VirtualBox Headless Interface 4.1.4
(C) 2008-2011 Oracle Corporation
All rights reserved.

VRDE server is listening on port 3389.

Then on your desktop, use a remote desktop application connect to the server port that VBoxHeadless displays (3389 in the example above, which is the default). You will then get the VM's console on your desktop. At that point you can follow the standard Solaris installation instructions (found here for Solaris 11).

The remote desktop application will depend on the operating system that you are using on your desktop. If you are running Solaris, you can use the rdesktop command. On Solaris 11 this is found in the aptly named rdesktop package (which is simple to install if not already on your desktop).

MyDesktop$ rdesktop myserver:3389

Newer versions of Microsoft Windows has the "Remote Desktop Connection" application which will allow you to specify the server host name and port number.

Adding Guest Additions

After installation of Solaris is complete, you will want to install the VirtualBox Guest Additions. This is also easy to do with the help of VBoxManage. The steps to follow are:

  • shut down the system after installation
  • detach the installation ISO file
  • attach the GuestAdditions ISO file
  • restart the VM
  • install the Guest Additions from the virtual CDROM

Shutting down the system is a bit tricky. If you use the "reboot" option after installation, it will try to boot off of the install media again. You can either halt it through the grub menu, or select "quit" instead of "reboot" and then perform a shutdown through the pulldown system menu or commandline (depending on which type of installation is performed).

Detaching the install ISO and attaching the Guest Additions ISO can be done as follows:

# VBoxManage storageattach "$VMNAME" --storagectl "IDE Controller" --port 0 \
--device 1 --medium emptydrive
# VBoxManage storageattach "$VMNAME" --storagectl "IDE Controller" --port 0 \
--device 1 --type dvddrive --medium /opt/VirtualBox/additions/VBoxGuestAdditions.iso

Then reboot using VBoxHeadless again, and you will find the GuestAdditions ISO mounted in your VM. You then just use the SVR4 pkgadd utility inside your VM to install the Guest Additions package:

# pkgadd -d /media/*/*.pkg all

Then just log off (no need to shut down) and log back on to reap the benefits of the Guest Additions.

Thursday Dec 29, 2011

Recovering From a ZFS Disaster

A few weeks ago I hit a rare bug in an internal Solaris 11 build that caused my laptop to panic when it rebooted. References to ZFS were listed in the panic output. I figured that I just needed to scrub my ZFS filesystem, which would find the error and remove it, and I would be up and running in no time.

I began by burning the Text Installer (available for download from OTN) onto a CD, then booting off of it. I selected the Shell option so I could run the OS off of RAM. I then issued the command:

 # zfs import -f rpool

Right away it panicked. Again. That was not good. I did have a backup that was a few days old, but I really wanted to try to salvage my most current content. In doing some research, I found that I could use mdb (modular debugger) to create a breakpoint at the point where it would normally panic on a zfs error, and then tell it not to.

I was able to boot off of the Live image and used this sequence to mount the /export/home/user partition:

# mdb -kw
> aok/W 0x1
> zfs_recover/E 0x1

# zpool import -f -o readonly=on rpool

Since the Live image is networked, I was able to dump the contents of my home directory to another system on the network. I just needed to set up an empty zfs fileystem remotely.

# zfs send rpool/export/home/user | 
ssh othersystem sudo zfs receive rpool/export/home/user-recovered

Once I had the contents of my user partition copied, I reinstalled my system. This recreated the rpool from scratch. I then did a reversal of the same zfs send command to populate a new home on my laptop:

 # ssh othersystem sudo  zfs send rpool/export/home/user-recovered | 
 zfs receive rpool/export/home/user-recovered

Then as root I destroyed the current rpool/export/home/user and renamed rpool/export/home/user-recovered:

# zfs destroy rpool/export/home/user
# zfs rename  rpool/export/home/user-recovered rpool/export/home/user

All my files were completely restored, without the need to go to my days-old backup copy or attempting an incremental update.

 If I felt I needed the root partition saved, I could have done the same zfs send on the root system as well. Then I could restore it as a new BE, at least for comparing. This would be handy for restoring configuration information like security certificates and printer settings. It could also be used to generate a list of installed packages, in case the reinstallation didn't catch everything.

About

Insights on Solaris from the Release Engineering view. We aren't upstream in development, or downstream in QA. We're in the middle of each Solaris build.

Search

Archives
« September 2015
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today