Thursday Dec 17, 2009

Switching from Nevada to OpenSolaris

I've fixed another bug (findroot grub menu entry) in the create-be script and added support for additional zpools (i.e. along with rpool, other zpools should now work seamlessly across Nevada and OpenSolaris BEs. NOTE: for additional zpools, the nevada and OpenSolaris BEs should be the exact same build).

The scripts lets you create a new (non COW) OpenSolaris BE on a nevada zfs root based system (or OpenSolaris system). You can use this to transition a Nevada zfs root based system to OpenSolaris. You can also choose to install an arbitrary OpenSolaris build (i.e. if you want to downgrade).

DISCLAIMER: this is totally unsupported by Sun, could mess up your system, etc. etc.

I strongly recommend creating a scratch BE to run this script out of, in case something goes wrong.

You can grab an updated copy here. Remember the DISCLAIMER above.. Back up your data first!

For folks inside of SWAN, here's a cheatsheet...

Write down your IP addr, netmask, gateway, NIS domain

Create & switch to a scratch BE

# lucreate -n scratch-be
# luactivate scratch-be
# init 6
Install the new OpenSolaris BE (this is not an upgrade, it's a fresh install into a BE on the same rpool as your current BEs)
# pkgadd -d /net/girltalk2/export/mrj/pkg-gate/packages/i386/ \\
  SUNWipkg SUNWpython-ply SUNWpython-pycurl
# wget http://blogs.sun.com/mrj/resource/create-be
# chmod a+x create-be
# /root/create-be --build=129 --bename=osol129 --repo=http://ipkg.sfbay/dev --menu="osol129"
If you want to install additional software, i.e. setup a build machine
# zfs set mountpoint="/mnt" rpool/ROOT/osol129
# zfs mount rpool/ROOT/osol129
# pkg -R /mnt set-publisher -O http://ipkg.sfbay/extra extra
# pkg -R /mnt install developer/opensolaris/osnet@0.5.11-0.129
# zfs umount rpool/ROOT/osol129
If you want to be able to build xvm-gate
# zfs set mountpoint="/mnt" rpool/ROOT/osol129
# zfs mount rpool/ROOT/osol129
# pkg -R /mnt install \\
  SUNWgmake@3.81-0.129 \\
  SUNWbcc@0.16.17-0.129 \\
  SUNWgnu-readline@5.2-0.129 \\
  SUNWxwinc@0.5.11-0.129 \\
  SUNWgnome-common-devel@0.5.11-0.129 \\
  SUNWlibtool@1.5.22-0.129 \\
  SUNWgnu-automake-110@1.10-0.129 \\
  SUNWaconf@2.63-0.129 \\
  SUNWgit@1.5.6.5-0.129 \\
  SUNWxvm@3.3.2-0.129
# zfs umount rpool/ROOT/osol129
Reboot into your new be, configure the network, etc. migrate over other BE settings, e.g. sshd config.
# bootadm list-menu
# bootadm set-menu default=....your-osol129-menu-number....
# reboot
: run through sysconfig, reboot
: login
# beadm mount ....your-old-be.... /mnt
# cp /mnt/etc/ssh/sshd_config /etc/ssh/
# cp /mnt/etc/ssh/\*key\* /etc/ssh/
# svcadm refresh ssh;svcadm restart ssh
: migrate over other customizations you might have
# beadm umount ....your-old-be....

Friday Apr 10, 2009

How small can Solaris go (part 2)

First, a couple of answers to some questions from the first post..

James, re: "be able (after a PXE boot) to mount NFS or iSCSI and switch (or layer) root". No. I'm really going for a standalone, extremely stripped down, ramdisk based image right now. No swap. No disk support. This wouldn't be something you would use for a NAS appliance. For that, you would be better off going with a stripped down pkg based opensolaris image. I can get those down to ~350M using a "custom installer"

Benoit, re: "And in a memory usage how light can solaris go?" Good question, lets take a look :-)

When I do get a little free time to play around with this stuff, I generally do it using OpenSolaris xVM domUs since it takes about a second to test then reboot these images.

The last post I did was a i86pc based image.. Here's a domU with the same bits loaded.. We're at ~ 15M for disk usage.

Here's the py file for my domU.. Notice I'm using the dom0's unix (since I'm building the ramdisk based on dom0's bits).

: alpha[1]#; cat /tank/guests/micro/guest.py 
name = "micro"
vcpus = 1
memory = "256"
kernel = "/platform/i86xpv/kernel/unix"
ramdisk = "/tank/guests/micro/ramdisk"
extra = "/platform/i86xpv/kernel/unix"
vif = ['']
on_shutdown = "destroy"
on_reboot = "restart"
on_crash = "preserve"
: alpha[1]#; 

: alpha[1]#; xm create -c /tank/guests/micro/guest.py
Using config file "/tank/guests/micro/guest.py".
Started domain micro
v3.3.2-rc1-pre-xvm chgset 'Mon Apr 06 20:13:29 2009 -0400 18424:97250633e58b'
SunOS Release 5.11 Version onnv-3.3-mrj 32-bit
Copyright 1983-2009 Sun Microsystems, Inc.  All rights reserved.
Use is subject to license terms.
strplumb: failed to initialize drv/dld
# df -lk
Filesystem            kbytes    used   avail capacity  Mounted on
/ramdisk:a             38255   15116   19314    44%    /
/devices                   0       0       0     0%    /devices
/dev                       0       0       0     0%    /dev
ctfs                       0       0       0     0%    /system/contract
proc                       0       0       0     0%    /proc
mnttab                     0       0       0     0%    /etc/mnttab
swap                  205012       0  205012     0%    /etc/svc/volatile
objfs                      0       0       0     0%    /system/object
sharefs                    0       0       0     0%    /etc/dfs/sharetab
#
Now, we are really stripped down (for Solaris)... Lets add in kmdb and enough of mdb to let us do a mdb -K.

I have a very very ugly python script I use to build up my ramdisk...

: alpha[1]#; ./micro.py 
USAGE: ./micro.py cfg disk sizeM
Here's the config file I'm using for my domu after uncommenting out kmdb... Notice I don't even have syscalls in at this point...
: alpha[1]#; cat domu.files
@kmdb32

@kernel32
@i86xpv32
@init32
#@syscall32

#@mount
#@uidcache

#@net32
#/usr/bin/ln

#@devfsadm32
#@basic32
#@ssh32
I just luupgraded my system to b112.. Lets build a new image using that... So how much memory are we using?
: alpha[1]#; ./micro.py domu.files /tank/guests/micro/ramdisk 40
NOTICE: overwriting disk: /tank/guests/micro/ramdisk

: alpha[1]#; xm create -c /tank/guests/micro/guest.py 
Using config file "/tank/guests/micro/guest.py".
Started domain micro
v3.1.4-xvm chgset 'Mon Mar 30 23:29:09 2009 -0700 15914:bb9557896640'
SunOS Release 5.11 Version snv_112 32-bit
Copyright 1983-2009 Sun Microsystems, Inc.  All rights reserved.
Use is subject to license terms.
NOTICE: Invalid iBFT table 0x1
strplumb: failed to initialize drv/dld
# df -lk
Filesystem            kbytes    used   avail capacity  Mounted on
/ramdisk:a             38255   19246   15184    56%    /
/devices                   0       0       0     0%    /devices
/dev                       0       0       0     0%    /dev
ctfs                       0       0       0     0%    /system/contract
proc                       0       0       0     0%    /proc
mnttab                     0       0       0     0%    /etc/mnttab
swap                  204992       0  204992     0%    /etc/svc/volatile
objfs                      0       0       0     0%    /system/object
sharefs                    0       0       0     0%    /etc/dfs/sharetab
# mdb -K

Welcome to kmdb
kmdb: no terminal data available for TERM=
kmdb: failed to set terminal type to `', using `vt100'
Loaded modules: [ scsi_vhci mac xpv_psm ufs unix krtld genunix specfs xpv_uppc
 ]
[0]> ::memstat
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                       8922                34   14%
Anon                          140                 0    0%
Exec and libs                  78                 0    0%
Page cache                    550                 2    1%
Free (cachelist)              708                 2    1%
Free (freelist)             53089               207   84%
Balloon                         0                 0    0%

Total                       63487               247
[0]> 
less than 40M... Not great, but not bad either... There some code changes we could do to get things smaller. But it's not big enough where it would matter at this point... Since I'm building a 40M ramdisk, we need space for that too.. So I would need around 80-90M total for this image..

Lets do a quick test, changing the domU's memory to 80M.

< memory = "256"

> memory = "80"

: alpha[1]#; ./micro.py domu.files /tank/guests/micro/ramdisk 40
NOTICE: overwriting disk: /tank/guests/micro/ramdisk
: alpha[1]#; xm create -c /tank/guests/micro/guest.py 
Using config file "/tank/guests/micro/guest.py".
Started domain micro
v3.1.4-xvm chgset 'Mon Mar 30 23:29:09 2009 -0700 15914:bb9557896640'
SunOS Release 5.11 Version snv_112 32-bit
Copyright 1983-2009 Sun Microsystems, Inc.  All rights reserved.
Use is subject to license terms.
NOTICE: Invalid iBFT table 0x1
strplumb: failed to initialize drv/dld
# mdb -K
WARNING: retrying of kmdb allocation of 0x600000 bytes

Welcome to kmdb
kmdb: no terminal data available for TERM=
kmdb: failed to set terminal type to `', using `vt100'
Loaded modules: [ scsi_vhci mac xpv_psm ufs unix krtld genunix specfs xpv_uppc
 ]
[0]> ::memstat
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                       5766                22   31%
Anon                          140                 0    1%
Exec and libs                   0                 0    0%
Page cache                      1                 0    0%
Free (cachelist)             1259                 4    7%
Free (freelist)             11265                44   61%
Balloon                         0                 0    0%

Total                       18431                71
[0]> 
Interesting.. What about a 64-bit kernel? It's going to bigger of course. But if you need a 64-bit kernel, memory shouldn't be an issue.
: alpha[1]#; xm create -c /tank/guests/micro/guest64.py 
Using config file "/tank/guests/micro/guest64.py".
Started domain micro
v3.1.4-xvm chgset 'Mon Mar 30 23:29:09 2009 -0700 15914:bb9557896640'
SunOS Release 5.11 Version snv_112 64-bit
Copyright 1983-2009 Sun Microsystems, Inc.  All rights reserved.
Use is subject to license terms.
strplumb: failed to initialize drv/dld
# df -lk
Filesystem            kbytes    used   avail capacity  Mounted on
/ramdisk:a             38255   31187    3243    91%    /
/devices                   0       0       0     0%    /devices
/dev                       0       0       0     0%    /dev
ctfs                       0       0       0     0%    /system/contract
proc                       0       0       0     0%    /proc
mnttab                     0       0       0     0%    /etc/mnttab
swap                  195480       0  195480     0%    /etc/svc/volatile
objfs                      0       0       0     0%    /system/object
sharefs                    0       0       0     0%    /etc/dfs/sharetab
# mdb -K

Welcome to kmdb
kmdb: no terminal data available for TERM=
kmdb: failed to set terminal type to `', using `vt100'
Loaded modules: [ scsi_vhci mac xpv_psm ufs unix krtld genunix specfs xpv_uppc
 ]
[0]> ::memstat
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                      15575                60   25%
Anon                          192                 0    0%
Exec and libs                  93                 0    0%
Page cache                    771                 3    1%
Free (cachelist)             1203                 4    2%
Free (freelist)             45653               178   72%
Balloon                         0                 0    0%

Total                       63487               247
[0]> 
OK, now lets do something interesting... Let bring up networking enough so we can ping, etc. In my domu.files file, I'm going to bring in @syscall32, @mount, @net32, and /usr/bin/ln.

One thing you notice when you start playing with this stuff, is that things can grow very fast when you start pulling in user bins (due to all the libraries which can be pulled in too). You would think things like reboot and poweroff would be a small impact. Not so :-)

I have a custom init bin which configures the system and starts up a shell (no SMF, etc). I'll manually configure it though so you can see what I'm doing to get the system up.

Notice I'm setting up a dev link for the NIC.. We don't have devfsadm in this particular ramdisk. Obviously you would pre-create the link on the ramdisk, or pull in the devfsadm bits.. But I though it was an interesting thing to show.

: alpha[1]#; xm create -c /tank/guests/micro/guest.py 
Using config file "/tank/guests/micro/guest.py".
Started domain micro
v3.1.4-xvm chgset 'Mon Mar 30 23:29:09 2009 -0700 15914:bb9557896640'
SunOS Release 5.11 Version snv_112 32-bit
Copyright 1983-2009 Sun Microsystems, Inc.  All rights reserved.
Use is subject to license terms.
NOTICE: Invalid iBFT table 0x1
# mount -o remount,rw /devices/ramdisk:a /
# /sbin/soconfig -f /etc/sock2path
# ifconfig lo0 plumb 127.0.0.1 netmask 255.255.255.0 up
# cd /dev
# ln -s ../devices/xpvd/xnf@0:xnf0 xnf0
# ifconfig xnf0 plumb 192.168.0.91 netmask 255.255.255.0 up
# route add default 192.168.0.1
add net default: gateway 192.168.0.1
# ping 192.168.0.1
192.168.0.1 is alive
# df -lk
Filesystem            kbytes    used   avail capacity  Mounted on
/devices/ramdisk:a     38255   28442    5988    83%    /
/devices                   0       0       0     0%    /devices
/dev                       0       0       0     0%    /dev
ctfs                       0       0       0     0%    /system/contract
proc                       0       0       0     0%    /proc
mnttab                     0       0       0     0%    /etc/mnttab
swap                  198852       0  198852     0%    /etc/svc/volatile
objfs                      0       0       0     0%    /system/object
sharefs                    0       0       0     0%    /etc/dfs/sharetab
# mdb -K

Welcome to kmdb
kmdb: no terminal data available for TERM=
kmdb: failed to set terminal type to `', using `vt100'
Loaded modules: [ scsi_vhci mac xpv_psm ufs unix krtld genunix specfs xpv_uppc
 ]
[0]> ::memstat
Page Summary                Pages                MB  %Tot
------------     ----------------  ----------------  ----
Kernel                      11628                45   18%
Anon                          140                 0    0%
Exec and libs                 129                 0    0%
Page cache                    711                 2    1%
Free (cachelist)             1435                 5    2%
Free (freelist)             49444               193   78%
Balloon                         0                 0    0%

Total                       63487               247
[0]> 

What's next? I'm trying to get VirtualBox (CLI only) running (RDP for external access) :-)

Thursday Mar 12, 2009

How small can Solaris go

I've been playing around lately to see how small of a Solaris image I can get that will boot to a shell prompt... It turns out, it can get pretty small.. And it boots to the prompt in ~ 1 second... Although there's not a lot you can do with it :-)
SunOS Release 5.11 Version onnv-3.3-mrj 32-bit
Copyright 1983-2009 Sun Microsystems, Inc.  All rights reserved.
Use is subject to license terms.
WARNING: Last shutdown is later than time on time-of-day chip; check date.
strplumb: failed to initialize drv/dld
# df -lk
Filesystem            kbytes    used   avail capacity  Mounted on
/ramdisk:a             38255   15146   19284    44%    /
/devices                   0       0       0     0%    /devices
/dev                       0       0       0     0%    /dev
ctfs                       0       0       0     0%    /system/contract
proc                       0       0       0     0%    /proc
mnttab                     0       0       0     0%    /etc/mnttab
swap                 1768156       0 1768156     0%    /etc/svc/volatile
objfs                      0       0       0     0%    /system/object
sharefs                    0       0       0     0%    /etc/dfs/sharetab
# du -sk \*
31      boot
952     dev
1       devices
67      etc
6087    kernel
4849    lib
8       lost+found
1931    platform
11277   proc
122     sbin
1441    system
1       tmp
59      usr
3       var
# 
About

mrj

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today