In the zone with SVM and JET...

We now have a decent OS image, our root disk layout finished and mirrored, along with the associated metadbs live. You DID test at this point and make sure that the template file checks out and the target machine installs, right?

The partitions that we need to use for zone allocations are created for us, slice s6 on each of our 8 disks. The root disk and the root mirror disk have about 80G in s6, and the remaining 6 disks have about 143G in s6. We need to glue all of that free disk space together into a giant mirrored metadevice. We'll call it d100. In our sds_metadevices_to_init variable, we need to add:

d101 d102 d100
d101:4:1:c1t0d0s6:1:c1t2d0s6:1:c1t4d0s6:1:c1t6d0s6
d102:4:1:c1t1d0s6:1:c1t3d0s6:1:c1t5d0s6:1:c1t7d0s6
d100:-m:d101

This created metadevices d101, d102 and d100. Metadevice d101 would be defined as a 4-way concatenation of partitions c1t0d0s6, c1t2d0s6, c1t4d0s6, and c1t6d0s6. Metadevice d102 would be defined as a 4-way concatenation of partitions c1t1d0s6, c1t3d0s6, c1t5d0s6, and c1t7d0s6. We didn't see a real need for striping in this configuration for performance, and we want to retain the option of stacking more disk partitions on top of the concatenation at some point, if that becomes an issue. Remember that these disks are on a single controller, with the throttles and bottlenecks associated with that hardware limitation.

The two metadeviced d101 and d102 were then mirrored into a new metadevice named d100. This could have been accomplished by the command lines:

# metainit d101 4 1 c1t0d0s6 1 c1t2d0s6 1 c1t4d0s6 1 c1t6d0s6
# metainit d102 4 1 c1t1d0s6:1 c1t3d0s6 1 c1t5d0s6 1 c1t7d0s6
# metainit d100 -m d101
# metattach d100 d102

Now that we have a ~500GB disk partition called d100, we need to slice out the soft partitions. Since we were unable to use ZFS, using SVM softpartitions gives us the maximum flexibility possible when we slice out the zone partitions. For now, we will be creating fairly standard sizes and mountpoints for the zones to be created on, but that might change later.

The first zone (100) is for the Message Queue, and will contain 4 filesystems (two, plus Live Upgrade space). Zones 101 through 135 will use the same 6 partitions and sizes. Naming for the zone space metadevices is d[zone number][partition number].

	d1000 d1001 d1002 d1003
           d1000:-p:d100:4500M
           d1001:-p:d100:4g
           d1002:-p:d100:4500M
           d1003:-p:d100:4g
	d1010 d1011 d1012 d1013 d1014 d1015
           d1010:-p:d100:3200M
           d1011:-p:d100:1800M
           d1012:-p:d100:2g
           d1013:-p:d100:1g
           d1014:-p:d100:3200M
           d1015:-p:d100:1800M
    ...
	d1350 d1351 d1352 d1353 d1354 d1355
           d1350:-p:d100:3200M
           d1351:-p:d100:1800M
           d1352:-p:d100:2g
           d1353:-p:d100:1g
           d1354:-p:d100:3200M
           d1355:-p:d100:1800M"

This is the equivilant of running over 200 "metainit d#### -p d100 [size]" commands by hand. I created this section with a quick and dirty shell script that did a loop, counting up each zone, and containing a loop with echo's that created the lines for that zone. Ugly, but quick and effective.

# Creating filesystems
#
# If you wish to newfs any UFS filesystems, then you can do them from here.
# As a side effect, the devices need not be DiskSuite ones....
#
# sds_newfs_devices:	Space separated list of devices to newfs - full paths
#
# If you wish to specify extra newfs options, then use a custom script to
# perform the newfs etc. THIS WILL ONLY CREATE UFS FILESYSTEMS!
#
sds_newfs_devices=" /dev/md/dsk/d1000 
		/dev/md/dsk/d1001 
		/dev/md/dsk/d1002 
		/dev/md/dsk/d1003 
		/dev/md/dsk/d1010 
		/dev/md/dsk/d1011 
		/dev/md/dsk/d1012 
		/dev/md/dsk/d1013 
     ...
		/dev/md/dsk/d1350 
		/dev/md/dsk/d1351 
		/dev/md/dsk/d1352 
		/dev/md/dsk/d1353 
		/dev/md/dsk/d1354
		/dev/md/dsk/d1355"

This tells JET to newfs all of those metadevices that we just softpartitioned out of the d100 space. This is the most time-consuming piece of the machine build, lasting about 10-15 minutes. Again, this section was initially created with a quick and dirty shell script full of "for" loops and "echo" statements.

#
# sds_mount_devices:	space separated list of tuples that will be used
#			to populate the /etc/vfstab file
#
#	tuples of the form
#
#	blockdevice:mountpoint:fsckpass:mountatboot:options
#		blockdevice:	/dev/dsk/.... or /dev/md/dsk/....
#		mountpoint:	/export/big_raid_device	
#		fsckpass:	5th column from /etc/vfstab
#		mountatboot:	"yes" or "no"
#		options:	7th column from /etc/vfstab
#
sds_mount_devices="
        /dev/md/dsk/d1000:/zones/ZONE100/ROOT:2:yes:-
        /dev/md/dsk/d1001:/zones/ZONE100/var:2:yes:-
        /dev/md/dsk/d1002:/zones/ZONE100/ROOT_LU:2:yes:-
        /dev/md/dsk/d1003:/zones/ZONE100/VAR_LU:2:yes:-
        /dev/md/dsk/d1010:/zones/ZONE101/ROOT:2:yes:-
        /dev/md/dsk/d1011:/zones/ZONE101/var:2:yes:-
        /dev/md/dsk/d1012:/zones/ZONE101/logs:2:yes:-
        /dev/md/dsk/d1013:/zones/ZONE101/apps:2:yes:-
        /dev/md/dsk/d1014:/zones/ZONE101/ROOT_LU:2:yes:-
        /dev/md/dsk/d1015:/zones/ZONE101/VAR_LU:2:yes:-
   ...
        /dev/md/dsk/d1350:/zones/ZONE135/ROOT:2:yes:-
        /dev/md/dsk/d1351:/zones/ZONE135/var:2:yes:-
        /dev/md/dsk/d1352:/zones/ZONE135/logs:2:yes:-
        /dev/md/dsk/d1353:/zones/ZONE135/apps:2:yes:-
        /dev/md/dsk/d1354:/zones/ZONE135/ROOT_LU:2:yes:-
        /dev/md/dsk/d1355:/zones/ZONE135/VAR_LU:2:yes:-"

This section creates the /etc/vfstab entries for the partitions that we just created. Yes, we are mounting the Live Upgrade space for both the global zone, and for our full root local zones, but that keeps folks from stealing space from my d100 or mounting those allocated partitions for other nefarious uses. The space is there and mounted into the global zone for when I need to use it for LU. Philosophical arguments on this item > /dev/null and as always, your mileage may vary.

Now we should be ready to cook our JET template and give it a shot:


jetserver# pwd
/opt/SUNWjet/Templates

jetserver# ../bin/make_client -f myhost
Gathering network information..
        Client: 10.68.75.9 (10.68.75.0/255.255.255.0)
        Server: 10.68.75.23 (10.68.75.0/255.255.255.0, SunOS)
Solaris: client_prevalidate
         Clean up /etc/ethers
Solaris: client_build
Creating sysidcfg
Creating profile
Adding base_config specifics to client configuration
Adding flash specifics to client configuration
FLASH: Modifying client profile for flash install
FLASH: Removing package/cluster/usedisk entries from profile
Adding eiscd specifics to client configuration
EISCD: EISCD: No profile changes needed
Adding sds specifics to client configuration
SDS: Configuring preferred metadevice numbers
Solaris: Configuring JumpStart boot for myhost
         Starting SMF services for JumpStart
Solaris: Configure bootparams build
cleaning up preexisting install client "myhost"
removing myhost from bootparams
updating /etc/bootparams
Force bootparams terminal type
-Restart bootparamd
Running '/opt/SUNWjet/bin/check_client  myhost'
        Client: 10.68.75.9 (10.68.75.0/255.255.255.0)
        Server: 10.68.75.23 (10.68.75.0/255.255.255.0, SunOS)
Checking product base_config/solaris
Checking product flash
FLASH: Checking nfs://10.68.75.23/export/flars/SPARC-App-Tier-20.flar
Checking product custom
Checking product eiscd
Checking product sds
--------------------------------------------------------------
Check of client myhost

-> Passed....
jetserver#

YAY!!! We now have a template that (according to JET) might actually work. Let's go over to the target machine and give it a shot:

sc> break -y
sc> console

Enter #. to return to ALOM.

{0} ok 
{0} ok 
{0} ok boot net - install

Boot device: /pci@0/pci@0/pci@1/pci@0/pci@2/network@0  File and 
args: - install
1000 Mbps full duplex  Link up
Requesting Internet Address for 0:14:4f:d3:9a:0
Requesting Internet Address for 0:14:4f:d3:9a:0
Requesting Internet Address for 0:14:4f:d3:9a:0

... blah blah blah ...

Using sysid configuration file 
10.68.75.23:/opt/SUNWjet/Clients/myhost/sysidcfg
Search complete.
Discovering additional network configuration...


Completing system identification...

Starting remote procedure call (RPC) services: done.
System identification complete.
Starting Solaris installation program...
Searching for JumpStart directory...
Using rules.ok from 10.68.75.23:/opt/SUNWjet.
Checking rules.ok file...
Using begin script: Utils/begin
Using derived profile: Utils/begin
Using finish script: Utils/finish
Executing JumpStart preinstall phase...
Executing begin script "Utils/begin"...
Installation of myhost at 09:45 on 19-Jun-2008
Loading JumpStart Server variables
Loading JumpStart Server variables
Loading Client configuration file
Loading Client configuration file
FLASH: Running flash begin script....
FLASH: Running flash begin script....
CUSTOM: Running custom begin script....
CUSTOM: Running custom begin script....
Begin script Utils/begin execution completed.
Searching for SolStart directory...
Checking rules.ok file...
Using begin script: install_begin
Using finish script: patch_finish
Executing SolStart preinstall phase...
Executing begin script "install_begin"...
Begin script install_begin execution completed.

So far so good, it found the right stuff. Now it should set up the disks and extract my flar.

Processing profile
	- Opening Flash archive
	- Validating Flash archive
	- Selecting all disks
	- Configuring boot device
	- Using disk (c1t0d0) for "rootdisk"
	- Configuring / (c1t0d0s0)
	- Configuring swap (c1t0d0s1)
	- Configuring /GZ_VAR_LU (c1t0d0s3)
	- Configuring /GZ_ROOT_LU (c1t0d0s4)
	- Configuring /var (c1t0d0s5)
	- Configuring  (c1t0d0s7)
	- Configuring  (c1t2d0s7)
	- Configuring  (c1t3d0s7)
	- Configuring  (c1t4d0s7)
	- Configuring  (c1t5d0s7)
	- Configuring  (c1t6d0s7)
	- Configuring  (c1t7d0s7)
	- Configuring  (c1t0d0s6)
	- Configuring  (c1t2d0s6)
	- Configuring  (c1t3d0s6)
	- Configuring  (c1t4d0s6)
	- Configuring  (c1t5d0s6)
	- Configuring  (c1t6d0s6)
	- Configuring  (c1t7d0s6)
	- Deselecting unmodified disk (c1t1d0)

Verifying disk configuration
	- WARNING: Changing the system's default boot device in 
the EEPROM

Verifying space allocation
	NOTE: 1 archives did not include size information

Preparing system for Flash install

Configuring disk (c1t0d0)
	- Creating Solaris disk label (VTOC)

Configuring disk (c1t2d0)
	- Creating Solaris disk label (VTOC)

Configuring disk (c1t3d0)
	- Creating Solaris disk label (VTOC)

Configuring disk (c1t4d0)
	- Creating Solaris disk label (VTOC)

Configuring disk (c1t5d0)
	- Creating Solaris disk label (VTOC)

Configuring disk (c1t6d0)
	- Creating Solaris disk label (VTOC)

Configuring disk (c1t7d0)
	- Creating Solaris disk label (VTOC)

Creating and checking UFS file systems
	- Creating / (c1t0d0s0)
	- Creating /GZ_VAR_LU (c1t0d0s3)
	- Creating /GZ_ROOT_LU (c1t0d0s4)
	- Creating /var (c1t0d0s5)

Beginning Flash archive processing

Predeployment processing
16 blocks
16 blocks
16 blocks

No local customization defined

Extracting archive: patchedflar
	Extracted    0.00 MB (  0% of 1302.60 MB archive)
	Extracted    1.00 MB (  0% of 1302.60 MB archive)
	Extracted    2.00 MB (  0% of 1302.60 MB archive)
   ...
	Extracted 1302.00 MB ( 99% of 1302.60 MB archive)
	Extracted 1302.60 MB (100% of 1302.60 MB archive)
	Extraction complete

Postdeployment processing

No local customization defined

Customizing system files
	- Mount points table (/etc/vfstab)
	- Unselected disk mount points (/var/sadm/system/data/vfstab.unselected)
	- Network host addresses (/etc/hosts)
	- Environment variables (/etc/default/init)

Cleaning devices

Customizing system devices
	- Physical devices (/devices)
	- Logical devices (/dev)

Installing boot information
	- Installing boot blocks (c1t0d0s0)
	- Installing boot blocks (/dev/rdsk/c1t0d0s0)
	- Updating system firmware for automatic rebooting

Installation log location
	- /a/var/sadm/system/logs/install_log (before reboot)
	- /var/sadm/system/logs/install_log (after reboot)

Flash installation complete
Executing JumpStart postinstall phase...
Executing finish script "Utils/finish"...

YAY!!! Still going well. Now we'll see if the JET stuff actually works. At this point, we see lots of information about moving the scripts around, copying stuff in, and setting up for the reboots and activities that happen between those reboots. After the first reboot, we see:


Boot device: /pci@0/pci@0/pci@2/scsi@0/disk@0,0:a  File and args:
SunOS Release 5.10 Version Generic_127127-11 64-bit
Copyright 1983-2008 Sun Microsystems, Inc.  All rights reserved.
Use is subject to license terms.
myhost
/dev/rdsk/c1t0d0s4 is clean
/dev/rdsk/c1t0d0s3 is clean
JumpStart (/var/opt/sun/jet/post_install/S99jumpstart) started @ Thu 
Jun 19 09:52:42 MST 2008
Loading JumpStart Server variables
Loading Client configuration file
Running additional install files for reboot Platform/1
NFS Mounting Media Directories
Mounting nfs://10.68.75.23/export/install/patches on 
/var/opt/sun/jet/js_media/patch
Mounting nfs://10.68.75.23/export/install/pkgs on 
/var/opt/sun/jet/js_media/pkg
CUSTOM: Running 001.custom.001.set_etc_system
SDS: Running 002.sds.001.create_fmthard
SDS: Running 002.sds.001.set_boot_device
SDS: Running 002.sds.002.create_metadb
SDS: Running 002.sds.003.create_root_mirror
SDS: Running 002.sds.007.create_user_devices
SDS: Running 002.sds.008.create_filesystems
SDS: Running 002.sds.009.create_vfstab_entries
NFS Unmounting Media Directories
Unmounting /var/opt/sun/jet/js_media/pkg
Unmounting /var/opt/sun/jet/js_media/patch
Save existing system entries
Copying file /etc/system to /etc/system.prejs2
Jun 19 10:11:03 myhost reboot: rebooted by root

Jun 19 10:11:04 myhost syslogd: going down on signal 15

Jun 19 10:11:04 rpc.metad: Terminated

syncing file systems... done
rebooting...
Resetting...

YAY!!! Our disk configurations all executed without errors or warnings. Now let's see if it comes back clean after the reboot.

Boot device: rootdisk  File and args: 

SunOS Release 5.10 Version Generic_127127-11 64-bit
Copyright 1983-2008 Sun Microsystems, Inc.  All rights reserved.
Use is subject to license terms.
 myhost
/dev/md/rdsk/d4 is clean
/dev/md/rdsk/d3 is clean
/dev/md/rdsk/d1000 is clean
/dev/md/rdsk/d1001 is clean
/dev/md/rdsk/d1002 is clean
/dev/md/rdsk/d1003 is clean
...
/dev/md/rdsk/d1352 is clean
/dev/md/rdsk/d1353 is clean
/dev/md/rdsk/d1354 is clean
/dev/md/rdsk/d1355 is clean

... [lots of other activity deleted]

SDS: Running 003.sds.001.attach_mirrors
SDS: Running 003.sds.002.attach_user_mirrors
Disable & delete SMF tag svc:/site/jetjump
JumpStart is complete @ Thu Jun 19 10:15:55 MST 2008

myhost console login: 

Success.


bill.


Comments:

Post a Comment:
Comments are closed for this entry.
About

mrbill

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today