Tuesday Aug 29, 2006

T2000 link aggregation patch released

dladm is used for creating link aggregations on Solaris 10. This can be used on any nemo based driver. Unfortunatley on your T2000 unless you are running update 2 with the latest patch bundle you are stuck using ipge. The latest e1000g driver is included in KU 118833-20 or higher. Patch 123334 will take care of the transition for you.

Tuesday Aug 08, 2006

Generating a ssl cert and enabling kssl on a T2000

This is a walk through on how you might want to generate your own ssl certs and and use those with a webserver with kssl on Solaris 10. Start by setting up your own certificates

oaf207# cd /opt/SUNWwbsvr/CA
oaf207# openssl
oaf207# mkdir certs crl newcerts private
oaf207# echo "01" > serial
oaf207# cp /dev/null index.txt
oaf207# cp /etc/sfw/openssl/openssl.cnf .
oaf207# vi openssl.cnf

and change 
dir             = /etc/sfw/openssl      # Where everything is kept
dir             = /opt/SUNWwbsvr/CA
oaf207# openssl req -new -x509 -keyout private/cakey.pem -out cacert.pem -days 365 \\
> -config openssl.cnf

Generating a 1024 bit RSA private key
....++++++
...++++++
writing new private key to 'private/cakey.pem'
Enter PEM pass phrase:
Verifying - Enter PEM pass phrase:
------
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [US]:ie
State or Province Name (full name) [Some-State]:dublin
Locality Name (eg, city) []:clontarf
Organization Name (eg, company) [Unconfigured OpenSSL Installation]:sun
Organizational Unit Name (eg, section) []:perf
Common Name (eg, YOUR name) []:testuser
Email Address []:configure_ssl@sun.com

oaf207# openssl req -nodes -new -x509 -keyout newreq.pem -out newreq.pem -days 365 \\
> -config openssl.cnf

Generating a 1024 bit RSA private key
....++++++
..................++++++
writing new private key to 'newreq.pem'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [US]:ie
State or Province Name (full name) [Some-State]:dublin
Locality Name (eg, city) []:clontarf
Organization Name (eg, company) [Unconfigured OpenSSL Installation]:sun
Organizational Unit Name (eg, section) []:perf
Common Name (eg, YOUR name) []:testuser
Email Address []:configure_ssl@sun.com

oaf207#  openssl x509 -x509toreq -in newreq.pem -signkey newreq.pem -out tmp.pem

Getting request Private Key
Generating certificate request

oaf207# openssl ca -config openssl.cnf -policy policy_anything -out newcert.pem \\
> -infiles tmp.pem

Using configuration from openssl.cnf
6789:error:0E06D06C:configuration file routines:NCONF_get_string:no value:/on10/build-nd/G10U2B2/usr/src/common/openssl/crypto/conf/conf_lib.c:329:
group=CA_default name=unique_subject
Enter pass phrase for /opt/SUNWwbsvr/CA/private/cakey.pem:
Check that the request matches the signature
Signature ok
Certificate Details:
        Serial Number: 1 (0x1)
        Validity
            Not Before: Aug  4 15:08:50 2006 GMT
            Not After : Aug  4 15:08:50 2007 GMT
        Subject:
            countryName               = ie
            stateOrProvinceName       = dublin
            localityName              = clontarf
            organizationName          = sun
            organizationalUnitName    = perf
            commonName                = testuser
            emailAddress              = configure_ssl@sun.com
        X509v3 extensions:
            X509v3 Basic Constraints:
                CA:FALSE
            Netscape Comment:
                OpenSSL Generated Certificate
            X509v3 Subject Key Identifier:
                FC:8D:C6:7C:D5:92:13:45:0E:85:74:8F:E1:3C:C8:89:B2:29:89:17
            X509v3 Authority Key Identifier:
                keyid:6F:DF:38:7D:D1:E7:C6:B5:ED:8D:19:57:13:CC:C4:2F:C0:2E:64:C6
                DirName:/C=ie/ST=dublin/L=clontarf/O=sun/OU=perf/CN=testuser/emailAddress=configure_ssl
@sun.com
                serial:00

Certificate is to be certified until Aug  4 15:08:50 2007 GMT (365 days)
Sign the certificate? [y/n]:y


1 out of 1 certificate requests certified, commit? [y/n]y
Write out database with 1 new entries
Data Base Updated

oaf207# ksslcfg create -f pem -i `pwd`/newreq.pem -x 8080 -p \\ 
> /opt/SUNWwbsvr/alias/password 443

oaf207# svcs |grep kssl
online         12:17:41 svc:/network/ssl/proxy:kssl-INADDR_ANY-443
oaf207# 

Thursday Jun 01, 2006

Link aggregation, Jumpstart post install script

Solaris 10 introduced the nemo framework for drivers and Solaris Nevada has more projects which build on said framework. In Update 2 of Solaris 10 support for data link aggregation was added which means we can build fat network pipes from most nics :) without any trunking software.

From the manual for dladm

     The dladm command is used to configure data-links. A config-
     ured  data-link  is  represented  in the system as a STREAMS
     DLPI (v2) interface which  may  be  plumbed  under  protocol
     stacks  such  as  TCP/IP.  Each data-link relies on either a
     single network device or an aggregation of devices  to  send
     packets to or receive packets from a network.
Heres it on a galaxy but you can do the same on a t2000 using the new e1000g driver.
oaf316# ifconfig -a unplumb
oaf316# dladm show-dev

e1000g0         link: up        speed: 1000  Mbps       duplex: full
e1000g1         link: up        speed: 1000  Mbps       duplex: full
e1000g2         link: up        speed: 1000  Mbps       duplex: full
e1000g3         link: up        speed: 1000  Mbps       duplex: full
Now we know what devices are available for our aggregation. We will make a an aggregation of 2 of the nics.
 
oaf316# dladm create-aggr -d e1000g0 -d e1000g3 1
oaf316# dladm show-aggr 1
key: 1 (0x0001) policy: L4      address: 0:14:4f:1:c8:b0 (auto)
           device       address                 speed           duplex  link    state
           e1000g0      0:14:4f:1:c8:b0   1000  Mbps    full    up      standby
           e1000g3      0:14:4f:1:c8:b3   1000  Mbps    full    up      standby
Agrregation completes and the link is in standby mode, next we need to plumb it.
oaf316# ifconfig aggr1 plumb
oaf316# ifconfig aggr1 10.1.10.1 netmask 255.255.255.0 up
regular ifconfig to setup the link. Lets check the device state now.
dladm show-aggr 1
key: 1 (0x0001) policy: L4      address: 0:14:4f:1:c8:b0 (auto)
           device       address                 speed           duplex  link    state
           e1000g0      0:14:4f:1:c8:b0   1000  Mbps    full    up      attached
           e1000g3      0:14:4f:1:c8:b3   1000  Mbps    full    up      attached
We add some more nics to the device while the device is up and running.
oaf316# dladm add-aggr -d e1000g1 -d e1000g2 1
oaf316# dladm show-aggr 1
key: 1 (0x0001) policy: L4      address: 0:14:4f:1:c8:b0 (auto)
           device       address                 speed           duplex  link    state
           e1000g0      0:14:4f:1:c8:b0   1000  Mbps    full    up      attached
           e1000g3      0:14:4f:1:c8:b3   1000  Mbps    full    up      attached
           e1000g1      0:14:4f:1:c8:b1   1000  Mbps    full    up      attached
           e1000g2      0:14:4f:1:c8:b2   1000  Mbps    full    up      attached
Now lets show off and remove a nic from the link
oaf316# dladm remove-aggr -d e1000g0 1
oaf316# dladm show-aggr 1
key: 1 (0x0001) policy: L4      address: 0:14:4f:1:c8:b3 (auto)
           device       address                 speed           duplex  link    state
           e1000g3      0:14:4f:1:c8:b3   1000  Mbps    full    up      attached
           e1000g1      0:14:4f:1:c8:b1   1000  Mbps    full    up      attached
           e1000g2      0:14:4f:1:c8:b2   1000  Mbps    full    up      attached

heres the postinstall script which we use with some of our systems, you will have to change the IP.

# script to setup link aggregation on nics which are not in use
# it trys to ignore unsupported nics.
# This script can be used as part of a jumpstart.
#
# man dladm

PRE=/
[ -f /a/usr/sbin/dladm ] && PRE=/a

PATH=$PRE/usr/bin:$PRE/usr/sbin
export PATH

# ip for configured device to use  eg 10.1.1.1
IP=10.1.1.1

# netmasks for configured device eg 255.255.255.0
Netmasks=255.255.255.0

# set this to "e1000g0 nge0 bge1" etc this can be left blank and we try to use
# other gld nics
NicsToUse=



showError() {
        echo "$0: $1"
        exit 1
}

# exit if no ip or netmask
[ -z "$IP" -o  -z "$Netmasks" ] && showError "IP and Netmasks must be defined"

# check network devices exists
if [ -z "$NicsToUse" ]; then
        ifconfig -a plumb 2>/dev/null
        NicsToUse=`ifconfig -a |awk -F: '//dev/null
        [ `dladm show-link $nic |grep -v -c legacy` ] && vNics="$vNics -d $nic"
done

# no nics supported by dladm
[ -z "$vNics" ] && showError "No supported nics on system"

# configure and plumb device
dladm create-aggr -R $PRE $vNics 1
[ $? != 0 ] && showError "error configuring aggr1 with dladm and $vNics"

ifconfig aggr1 plumb
ifconfig aggr1 $IP netmask $Netmasks up
[ $? != 0 ] && showError "error bringing up aggr1 with ifconfig $IP netmask $Netmasks "

# store nic details
echo $IP >$PRE/etc/hostname.aggr1
IP=`echo $IP|cut -f1-3 -d\\.`.0
echo "$IP $Netmasks" >>$PRE/etc/netmasks

You will also want to increase the number of soft rings used by your aggregations. This can be done via /etc/system or via mdb as the default is 2 per interface.

oaf316# mdb -kw 
Loading modules: [ unix krtld genunix specfs dtrace cpu.AuthenticAMD.15 ufs ip sctp usba fcp 
fctl nca random md lofs zfs nfs sppp crypto cpc fcip logindmux ptm ]
> ip_soft_rings_cnt/W 8
ip_soft_rings_cnt:              0x2             =       0x8
> $q
This increases the number of soft rings from 2 to 8. If your aggregation is already plumbed you will need to replumb it to take advantage of the extra rings. To make this permanent you will need to add it to /etc/system
set ip:ip_soft_rings_cnt=8
Be warned this will do it for each link after the next reboot. More to come on t2000 and link aggregation.

Monday Sep 19, 2005

When smart bios gets anoying

We recently got some new hardware which has some nice logic in the bios which says "if I have a mbr and no one has pressed F12 to select network boot, then silently boot from the default boot device". This is not the behaviour I am used, normally if the default boot device is set to network and there are no offers then boot from the next device in the boot list. Unfortunatley the default boot device on these machines is hard coded to the disk!, this is really annoying when your trying to jumpstart the system remotely and have no physical access.

There are 2 options since Solaris with grub is installed on the system, we can wipe the boot record and force a boot from the default device or modify the grub menu.
Wiping the mbr - pro: easily done, cons if something goes wrong theres no way back
Modify Grub - pro: can revert back to default os, cons need hands on if things go wrong

This assumes you have a tftp server configured for jumpstart and are using dhcp, all you do is add the following to the menu on the system to be reinstalled in /boot/grub/menu.lst. Typically I place the entry below the timeout entry ie as the first entry and set the default boot entry to 0

#
# default menu entry to boot
default 0
#
# menu timeout in second before default OS is booted
# set to -1 to wait for user input
timeout 10
#

dhcp
root (nd)
kernel  /I86PC.Solaris_11-15/multiboot kernel/unix - install dhcp  -B console=keyboard,install_config=IP_INSTALL_SERVER:/PATH_TO_JUMPSTART_CONFIG/MACHINE_NAME,sysid_config=IP_INSTALL_SERVER:/PATH_TO_JUMPSTART_CONFIG/MACHINE_NAME,install_media=IP_INSTALL_SERVER:/PATH_TO_INSTALL_MEDIA/OS_BUILD,install_boot=IP_INSTALL_SERVER:/PATH_TO_INSTALL_MEDIA/OS_BUILD/boot
module /I86PC.Solaris_11-15/x86.miniroot

The entries for kernel and module can be found in the menu.lst.MAC-ADDRESS on your tftp server after running add_install_client. The above entry is used when you have a jumpstart profile, to use an interactive install (why?) the entries would be as follows
#
# default menu entry to boot
default 0
#
# menu timeout in second before default OS is booted
# set to -1 to wait for user input
timeout 10
#
dhcp
root (nd)
kernel  /I86PC.Solaris_11-15/multiboot kernel/unix - install dhcp 
module /I86PC.Solaris_11-15/x86.miniroot

Simply what happens is, dhcp tell grub to configure the network device based on the dhcp settings. We set the root filesystem to be the network devices tftp directory. Load the kernel from the directory with the args of install and dhcp .

note
this will only work if you have a network device which is supported by the mini root.

Friday Sep 16, 2005

Modifying miniroot to boot via pxe

My laptop had network card which is almost supported by Solaris (read oem version of well known card), as of late I've been doing some work with jumpstart and my laptop has become my test machine. Since my card is unsupported I need to modify the miniroot so that I can pxe install the system. And I also need to add the entry as part of my postinstall script to allow me use the networking on my system.

The process of modifying the install miniroot is straight forward once you remember that the system is little endian so do the mods on a little endian box.


gunzip < solaris_build/boot/x86.miniroot >/tmp/miniroot
lofiadm -a /tmp/miniroot
/dev/lofi/2
# mount /dev/lofi/2 /mnt
#   echo 'iprb "pci8086,1050"\\n' >>/mnt/etc/driver_aliases
# tail -2 driver_aliases
iprb "pci8086,1050"
umount /mnt
# lofiadm -d /dev/lofi/2
# gzip < /tmp/miniroot > solaris_build/boot/x86.miniroot

Wiping bootenv.rc on x86

At some point over the last week I managed to swap the contents of my bootenv.rc for "hello world", well not quite, it actually contained bootfile='kernel/amd64/unix'. Theres nothing wrong with the entry per say, the problem is ALL the other entries had been over written. The system stayed up and continued running until I made the mistake of rebooting the box.

The symptoms of my over zealous redirect showed the following on the screen \*snip\* bios boot.... grub boot loader.. starts default boot of multiuser \*snip\* SunOS Release 5.11 Version XXX 64-bit Copyright 1983-2005 Sun Microsystems, Inc. All rights reserved. Use is subject to license terms. few lines of panic info and then back to bios, repeat as required. The solution was to boot a single user session, ie the failsafe session in gnome and edit the root partition. When booting the failsafe mini rooot asked if I wanted to mount the OS it found on the system. I agreed and my regular root filesystem was mounted as /a. I examined the contents of /a/boot/solaris/bootenv.rc and noticed the problem. I examined the contents of the miniroot bootenv.rc

\*snip\*

setprop kbd-type US-English
setprop ata-dma-enabled 1
setprop atapi-cd-dma-enabled 1
setprop ttyb-rts-dtr-off false
setprop ttyb-ignore-cd true
setprop ttya-rts-dtr-off false
setprop ttya-ignore-cd true
setprop ttyb-mode 9600,8,n,1,-
setprop ttya-mode 9600,8,n,1,-
setprop lba-access-ok 1
setprop prealloc-chunk-size 0x2000
setprop input-device 'keyboard'
setprop output-device 'screen'
\*/snip\*
ok no big problem, copy the file to /a/boot/solaris/bootenv.rc and just add the bootpath and bootfile. The bootpath can be got from the current device tree
ls -l /dev/dsk/c0d1s0
lrwxrwxrwx   1 root     root          50 Sep 16 10:46 /dev/dsk/c0d1s0 -> ../../devices/pci@0,0/pci-ide@7,1/ide@0/cmdk@1,0:a
based on this we add
bootpath='/pci@0,0/pci-ide@7,1/ide@0/cmdk@1,0:a'
to the bootenvrc, since we want to boot 64bit add the entry which caused the problem in the 1st place
bootfile='kernel/amd64/unix'
almost forgot to set the kbd type
setprop kbd-type 'UK-English'
reboot and were back....
About

nickyv

Search

Categories
Archives
« April 2014
MonTueWedThuFriSatSun
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
    
       
Today