Tuesday Jun 02, 2009

Minimal OpenSolaris 2009.06 Appliance Image for VirtualBox 2.2.4

With the release of the OpenSolaris 2009.06, I thought it is time to update the Minimal OpenSolaris 2008.11  Appliance OVF image that I had created earlier. The script create_osol2009006_app.sh has been updated to create minimal OpenSolaris 2009.06 Appliance images for VirtualBox. 

How to use the OVF image?

  • Download VirtualBox 2.2.4 and install it on your host platform.
  • Download the OpenSolaris 2009.06 App OVF image zip file and then unzip it.
  • Fire up Virtualbox GUI and  use menu item VirtualBox->File->Import Appliance to import the image (using the  OSOL200906App.ovf file ) into a new VirtualBox VM
  • Start the newly created VM and in few minutes you will be  ready to login into OpenSolaris 2009.06 kernel.The preset login information is user: root with password: opensolaris.

Comments welcome.

Wednesday Apr 08, 2009

Postgres 8.3 Appliance for VirtualBox 2.2

With the release of VirtualBox 2.2 today, exporting and importing appliances becomes lot easier. The new features of VirtualBox 2.2 includes import and export of appliances based on OVF or Open Virtualization Format. 

One of  my earlier blog entry talks about creating appliances. Using the same script mentioned in the earlier blog entry,  I now have a OVF Image  which will work with VirtualBox 2.2  "Import Appliance"  and make it easier to try a virtual PostgreSQL 8.3 database appliance running OpenSolaris under the cover.

The PostgreSQL 8.3 Appliance Image is less than 330MB (unlike a full install of OpenSolaris which could take more than 1500MB).  Just unzip the files in a directory and import them through VirtualBox 2.2 using File->Import Appliance Wizard.  Since an appliance (database in this case) is typically accessed through external application servers, you will want to use Bridge Network instead of NAT.  Detailed instructions to make it accessible (both VirtualBox VM as well as PostgreSQL 8.3)  from external clients are published on Postgres 8.3 Appliance page.

Finally  it is lot easier to finally see the following grub menu on your own VirtualBox 2.2:

 Also for PostgreSQL users/developers who are not familiar with OpenSolaris,  this VM  allows you to try the DTrace probes and ZFS snapshots with PostgreSQL 8.3. More information related to it is  coming soon on Postgres 8.3 Appliance page. Also the PostgreSQL DTrace Toolkit 2009.03.29 is available on PGFoundry.org.

Tip: You can also use my PostgreSQL Monitor Demo to connect to it:

As usual if you have questions, I will be happy to answer them.

Tuesday Dec 16, 2008

PostgreSQL 8.3 Appliance based on OpenSolaris using VirtualBox

Based on Alex Eremin's blog entry on minimal script and some of my own hacking,  I have a script that should allow to create a basic PostgreSQL 8.3 Appliance using OpenSolaris 2008.11 Kernel  in Virtualbox easily with an image which is not bloated (No X, Gnome, etc).

Here is how to try it out:

  • For the time being this requires the  OpenSolaris 2008.11 CD image (650+ MB) only to execute my script create_pg_appliance.sh which is about 6KB. Bear with me till I solve this problem and play along :-)
  • Download Virtualbox and install it
  • Create a New Virtual Box VM with following parameters in the Wizard Screen
    • Call it PostgreSQL 8.3 Appliance
    • Select OS Type "OpenSolaris" 
    • Base Memory  768MB or increase it to 1GB if you can spare your RAM for it
    • Create a New Dynamic Expanding Image with exactly 16.00 GB (Any other wise may not work)
  • Once the VM is created, immediately  click the blue Network link and modify it to select a "Host Based Network"  (in the setup make sure it is connected to the active host interface - wired or wireless depending on the host system)
  • Also Click the CD-ROM image and point it to the osol-200811.iso image that you downloaded earlier
  • Boot up the VM and select the first LiveCD option in the Grub Menu options
  • Select through the defaults to get the full Gnome Desktop
  • Open Firefox in the VM and make  sure your VM  has internet access
  • Open a terminal window and execute the following commands in sequence
    • wget "http://blogs.sun.com/jkshah/resource/create_pg_appliance.sh"
    • pfexec sh -x create_pg_appliance.sh
    • pfexec reboot
  • At this point stop the VM and disconnect the CD image connected to the VM and boot again or  select the  grub entry
    • Boot from Hard disk
  • Once the VM boots from the hard disk, select the  grub menu item
    • PostgreSQL 8.3 Appliance based on OpenSolaris 2008.11
  • The Operating system boots up to give a login prompt. Login as root/opensolaris
  • The PostgreSQL database server is already running with PGDATA in /var/postgres/8.3/data
  • Modify postgresql.conf  to add  listen-address='\*' parameter
  • Modify pg_hba .conf to  allow your clients in pg_hba.conf
  •  Restart the postgresql server instance as follows:
    • svcadm restart postgresql_83:default_32bit
  • Now the PostgreSQL Appliance is ready to be connected from your clients.
  • Remember to take snapshots of rpool/VOSApp/var so you can always revert back to earlier copies if you ever need it.
Image of Grub Menu

Note: In a trial installation on a MacBook Pro the script executed in  about 11 minutes which  includes the time it takes to download packages from pkg.opensolaris.org.  I thought that was fantastic.

Maybe I should also create a minimized LiveCD just to execute the script on Virtualbox.  If you have problems executing the script let me know.

Tuesday Nov 18, 2008

Quest for OpenSolaris based Appliance

Recently I burned a copy of OpenSolaris 2008.11 RC1 and used it with Songbird (using pkg install SUNWsongbird)  and Fluendo MP3 Decoder (which is free for OpenSolaris),  Flash and soon I had a setup in my bedroom which I love to call OpenSolaris Home Theatre Edition that I used to listen to my songs collection and watch online shows that I controlled via Remote Desktop  which to me was a serious contender to those Media Centers out there. 

However I realized that while I wanted to "Pump it up" for my personal usage, I really wanted to "Strip it Down" for business usage. What I meant is in order for someone to try it out with say another Open Source Software it is easier now to say try that Open Source Software pre-installed on OpenSolaris in a Virtualbox Image. However I found it very hard to do it in practice.

Say for example I want to get somebody to try out the latest version of MySQL on OpenSolaris. The easiest thing is to give them a VDI image of preinstalled version of MySQL 5.1 RC  running on OpenSolaris that somebody just double clicks and boom a virutalbox instance starts up with MySQL ready to deliver.

However there is a problem with that theory. The VDI image of OpenSolaris fresh install in a virtualbox instance is about 2.5 - 3 GB. Of course adding MySQL on top of it won't drastically increase the size but I still have a problem with the base size itself. Since the only way that this can work is to use some sort of Peer to Peer File Sharing technology as hosting and hoping people will download this DVD Size downloads without any problems is like going at 10:00am for a Thanksgiving Deal at a store that opens at 6:00am with less than 20 SKU available (Tough Luck!!).

Anyway I started dissecting on how to reduce the size of the VDI image. There are few tricks provided by Virtualbox itself to release zero'ed out sectors using

VBoxManage modifyvdi VDINAME compact

However trying this out on a VDI holding OpenSolaris 2008.11 RC1 candidate did not really help.

The next thing I tried was to list out all the packages that are installed using

pkg list

There are about 550 packages installed as reported by that list. I tried  removing many of the non-essential desktop things (Aha who needs Xorg, Gnome, Firefox, Thunderbird) and reduced it to less than 400 packages. However even with that the VDI images is still not much smaller than 2.5GB. I tried the trick of cloning the disk to another VDI via ZFS replace thinking it will get rid of all the freed up space obtained by removing all the packages but the resulting new VDI image was still 2.5GB.

Looking at ZFS list output I found that for my virtualbox instance with 512MB I have three main zfs file systems defined which gives me the approximate 2.5GB usage.

rpool/ROOT/opensolaris- The legacy mount which is root (/) which is about 1.3 GB

rpool/dump which is about 512MB

rpool/swap which is again about 512MB

Now the goal is to reduce the VDI size without creating complications for the end user trying out the VDI and also still have the capability of going back to the fully installed version of OpenSolaris.

Hence the quest still continues..

Wednesday Feb 27, 2008

Project Indiana and PostgreSQL

As part of Project Indiana which is working on getting a binary distribution of OpenSolaris, there is a new  OpenSolaris Developer Preview 2 release available which is based on Solaris Express build 79a. So why am I interested in this "yet-another-distribution" ?

Currently when people are interested in running PostgreSQL on Solaris, check out where the site points them. It points to download Solaris. The choice is either to download Solaris 10 8/07 (latest production release) or Solaris Express latest builds (bleeding edge technology). So yes its already out there. It also includes PostgreSQL 8.2 in both releases. In fact soon PostgreSQL 8.3 will also be integrated in the latest Solaris Express releases. So what's the problem?

 If you ask me, the problem is the size of the download. I think for a lean-mean database like PostgreSQL whose binary download size is less than 10MB, telling people to download the DVD image of Solaris Express which typically could be around 4GB. That's just too much for what they really need. Coming back to PostgreSQL, what portions of Solaris do you really need to execute PostreSQL to its optimum? Kernel, ZFS, DTrace, Monitoring Tools, maybe some basic X windows to use PgAdminIII. So why force people to download the whole DVD every time there is a new Solaris Express build out there.

Project Indiana does help to cut it down to less than 1/5 th of the original. The download size of the new OpenSolaris Developer Preview 2 is about 650MB which is now a CD image.  It takes out lot of "overhead" and simplifies lot of scenarios and works best when you have a brand new server and want to use the Operating System primarily to run a database on it (the way I see it, your view may differ).

Project Indiana is opening doors for "Database Appliances" with OpenSolaris at its core.

Anyway I took it to do a test drive on my laptop. After installing OpenSolaris Developer Preview 2. It has basic GNOME, Firefox already installed. (And that's how I am writing this current blog entry.) The installation did not have any versions of PostgreSQL installed. I wasn't surprised. The whole idea of Project Indiana is to pull down additional packages that user requires from the network. Of course which means its time to get familiar with the new  Image Packaging System or pkg(5)

First of all I have to search if there are any PostgreSQL related packages on the default server that it refers. (Make sure to use -r option to search on the remote server) (It goes without saying, you need an internet connection to work before you do the following.)

# pkg search -r postgres
basename pkg:/SUNWpostgr-82-client@8.2.4,5.11-0.75:20071114T203533Z
basename pkg:/SUNWpostgr-82-client@8.2.5,5.11-0.79:20080205T163827Z
basename pkg:/SUNWpostgr-82-pl@8.2.4,5.11-0.75:20071114T203552Z
basename pkg:/SUNWpostgr-82-pl@8.2.5,5.11-0.79:20080205T164009Z
basename pkg:/SUNWpostgr-82-tcl@1.5,5.11-0.75:20071114T203557Z
basename pkg:/SUNWpostgr-82-tcl@1.5,5.11-0.79:20080205T164054Z
basename pkg:/SUNWpostgr-82-server@8.2.5,5.11-0.79:20080205T164038Z
basename pkg:/SUNWpostgr-82-server@8.2.4,5.11-0.75:20071114T203552Z
basename pkg:/SUNWpostgr-82-jdbc@8.2.504,5.11-0.75:20071114T203550Z
basename pkg:/SUNWpostgr-82-jdbc@8.2.504,5.11-0.79:20080205T163940Z
basename pkg:/SUNWpostgr-82-l10n-ja@0.5.11,5.11-0.75:20071114T205225Z
basename pkg:/SUNWpostgr-82-l10n-ja@0.5.11,5.11-0.79:20080205T175432Z
basename pkg:/SUNWpostgr-82-server-data-root@8.2.4,5.11-0.75:20071114T203557Z
basename pkg:/SUNWpostgr-82-server-data-root@8.2.5,5.11-0.79:20080205T164023Z
basename pkg:/SUNWpostgr-server@8.1.9,5.11-0.75:20071114T203610Z
basename pkg:/SUNWpostgr-server@8.1.10,5.11-0.79:20080205T164225Z
basename pkg:/SUNWpostgr-82-contrib@8.2.4,5.11-0.75:20071114T203532Z
basename pkg:/SUNWpostgr-82-contrib@8.2.5,5.11-0.79:20080205T163847Z
basename pkg:/SUNWpostgr-82-devel@8.2.4,5.11-0.75:20071114T203535Z
basename pkg:/SUNWpostgr-82-devel@8.2.5,5.11-0.79:20080205T163858Z
basename pkg:/SUNWpostgr-82-libs@8.2.5,5.11-0.79:20080205T163955Z
basename pkg:/SUNWpostgr-82-libs@8.2.4,5.11-0.75:20071114T203550Z
basename pkg:/SUNWpostgr-82-docs@8.2.4,5.11-0.75:20071114T203540Z
basename pkg:/SUNWpostgr-82-docs@8.2.5,5.11-0.79:20080205T163915Z

Good to see that PostgreSQL 8.1 and PostgreSQL 8.2 versions are already available for on-demand download. (I had cheated here. I checked  before executing the command that it did exist at http://pkg.opensolaris.org )  (NOTE: There are plans for PostgreSQL 8.3 download also but will take some time to get it out there. In the meanwhile the Solaris  binaries (i386) can be downloaded from postgresql.org which works on OpenSolaris Developer Preview 2. See below)

Notice that 8.1 has only two packages while PostgreSQL 8.2 has quite many out there. I remember that 8.1 used to have Jumbo packages but patching them was pain since the patches ended up being the whole Jumbo packages and hence for PostgreSQL 8.2 it was broken down into bunch of smaller packages. The list shows the original versions and also patched final versions. Well that causes me some problem here since I am not sure which packages to install and in what order. Fortunately pkg(5) is aware of package dependencies so that solves part of the problem but I still have to make sure all of them have been installed. I figured the best way to narrow down the list is to filter by the 5.11-0.79 stamped packages so I see the package names only once that I want to install.

# pkg search -r postgres |grep  0\\.79  
basename pkg:/SUNWpostgr-82-client@8.2.5,5.11-0.79:20080205T163827Z
basename pkg:/SUNWpostgr-82-pl@8.2.5,5.11-0.79:20080205T164009Z
basename pkg:/SUNWpostgr-82-tcl@1.5,5.11-0.79:20080205T164054Z
basename pkg:/SUNWpostgr-82-server@8.2.5,5.11-0.79:20080205T164038Z
basename pkg:/SUNWpostgr-82-jdbc@8.2.504,5.11-0.79:20080205T163940Z
basename pkg:/SUNWpostgr-82-l10n-ja@0.5.11,5.11-0.79:20080205T175432Z
basename pkg:/SUNWpostgr-82-server-data-root@8.2.5,5.11-0.79:20080205T164023Z
basename pkg:/SUNWpostgr-server@8.1.10,5.11-0.79:20080205T164225Z
basename pkg:/SUNWpostgr-82-contrib@8.2.5,5.11-0.79:20080205T163847Z
basename pkg:/SUNWpostgr-82-devel@8.2.5,5.11-0.79:20080205T163858Z
basename pkg:/SUNWpostgr-82-libs@8.2.5,5.11-0.79:20080205T163955Z
basename pkg:/SUNWpostgr-82-docs@8.2.5,5.11-0.79:20080205T163915Z

# pkg install  SUNWpostgr-82-server
DOWNLOAD                                    PKGS       FILES     XFER (MB)
Completed                                    2/2     166/166   12.96/12.96

PHASE                                        ACTIONS
Install Phase                                289/289

Notice that IPS also downloaded the postgr-82-libs package.

# pkg search postgres             
basename pkg:/SUNWpostgr-82-libs@8.2.5,5.11-0.79:20080205T163955Z
basename pkg:/SUNWpostgr-82-server@8.2.5,5.11-0.79:20080205T164038Z

We can continue installing one package at a time

# pkg install  SUNWpostgr-82-client
DOWNLOAD                                    PKGS       FILES     XFER (MB)
Completed                                    1/1       64/64     3.66/3.66

PHASE                                        ACTIONS
Install Phase                                117/117

However installing multiple packages seems to make more sense to me 

# pkg install SUNWpostgr-82-jdbc SUNWpostgr-82-server-data-root SUNWpostgr-82-contrib SUNWpostgr-82-devel SUNWpostgr-82-pl SUNWpostgr-82-tcl SUNWpostgr-82-docs
DOWNLOAD                                    PKGS       FILES     XFER (MB)
Completed                                    7/7   1514/1514   12.99/12.99

PHASE                                        ACTIONS
Install Phase                              1667/1667
# ls /usr/postgres/8.2
bin  doc  etc  include    jdbc  lib  man    share

Anyway now we are now ready with PostgreSQL 8.2.5 on OpenSolaris  Developer Preview as one would normally use it on Solaris.

For now to use PostgreSQL 8.3 on OpenSolaris Developer Preview is to download the Solaris binaries from postgresql.org  and install it in /opt as follows

# cd /opt
# bunzip2 < /tmp/postgresql-8.3.0-sxde.i386.tar.bz2 | tar xpf -
# ls /opt/postgres/8.3-community/
bin  doc  etc  include    jdbc  lib  man    share

Whatever version you use just make sure you have the right directory in your PATH environment variable. Talking about PATH, you may also want to include the new /usr/gnu/bin along with your favorite path.



Jignesh Shah is Principal Software Engineer in Application Integration Engineering, Oracle Corporation. AIE enables integration of ISV products including Oracle with Unified Storage Systems. You can also follow me on my blog http://jkshah.blogspot.com


« July 2016