Part 8: Glassfish V3 Pet Catalog sample DEMO in VM Template - Cleaning

OpenSolaris 200906 JeOS Prototype was designed as PURE CLI enviroment providing main building stone for creation of Virtual Appliances or VM Templates

In this example I will DEMOnstrate creation of Evaluation type Virtual Appliance / VM Template which will leveradge OpenSolaris 200906 JeOS prototype and just in Dec/09 released Glassfish v3, Java EE 6 and Netbeans 6.8 with simple Java EE 6 demo. This entry is part of Virtual Appliances & OVF workshop.

New Glassfish V3 introduce an unique network repository based modular structure, so we can more easily create really simple initial DEMO setup which user can extend by installation more samples right from Glassfish V3 repository or from Netbeans GUI.


Step 4: Clean-up the template

I this step will be using a Live CD media and/or Source repository from OpenSolaris 200906 Virtualization Assistant Live-CD Proof of Concept

Step 4a: Motivations for Clean-up

Clean us is done because we want:

  • Have smaller image (To have smaller donwload size)
  • Have it more clear (No old log with irrelevant issues)
  • Have remove sensitive info (No password and so in logs or in history)

Step 4b: Clear layered app SW

1. Temporally disable our APPs

osol@osol-jeos:~$ svcadm disable -t osol-domain1
osol@osol-jeos:~$ svcadm disable -t sun-mysql51:osol-mysql51
osol@osol-jeos:~$ svcadm disable -t http:apache22

2. Clear Appache

osol@osol-jeos:~$ ls /var/apache2/2.2/logs/\*log
/var/apache2/2.2/logs/access_log  /var/apache2/2.2/logs/error_log
osol@osol-jeos:~$ pfexec su -
root@osol-jeos:~# echo -n >/var/apache2/2.2/logs/access_log
root@osol-jeos:~# echo -n >/var/apache2/2.2/logs/error_log
root@osol-jeos:~# exit

3. Clear Glassfish V3

osol@osol-jeos:~$ cd ./unbundled/glassfishv3/
osol@osol-jeos:~/unbundled/glassfishv3$ find . -name \*.log
./glassfish/domains/domain1/logs/jvm.log
./glassfish/domains/domain1/logs/server.log
osol@osol-jeos:~/unbundled/glassfishv3$ echo -n >./glassfish/domains/domain1/logs/jvm.log
osol@osol-jeos:~/unbundled/glassfishv3$ echo -n >./glassfish/domains/domain1/logs/server.log

4. Clear MySQL

osol@osol-jeos:~$ cd ./unbundled/webstack16
osol@osol-jeos:~/unbundled/webstack16$ find . -name \*log
./var/mysql/5.1/data/osol-jeos.log
osol@osol-jeos:~/unbundled/webstack16$ echo -n > ./var/mysql/5.1/data/osol-jeos.log
osol@osol-jeos:~/unbundled/webstack16$ ./bin/pkg purge-history

PS: Now you now where all logs are , so you can document it to user :-)

Step 4c: Clean OS and default user OSOL data

1. I will use clean.os.runtime from VA Live-CD PoC

http://src.opensolaris.org/source/xref/jeos/va-live-media/jeos/clean.os.runtime/jeos-os-cleanup.sh

pfexec su - root
root@osol-jeos:~# cd ~
root@osol-jeos:~# wget http://src.opensolaris.org/source/raw/jeos/va-live-media/jeos/clean.os.runtime/jeos-os-cleanup.sh
root@osol-jeos:~# chmod 0777 jeos-os-cleanup.sh
Warning: Execution of this cleaning stript is soft - no exits or error checks!
Deleting logs in /var/svc/log/\*
File size application-GlassFish-domain1:default.log made 0
File size application-GlassFish-osol-domain1:default.log made 0
File size application-database-sun-mysql51:osol-mysql51.log made 0
....
File size system-zones:default.log made 0
Deleting root user related information home dirs
Deleting osol user related information home dirs
Cleanig OS level logs and logn infos
Delete old ZFS stream stuf if exist
cannot destroy 'rpool@stream': dataset does not exist
no snapshots destroyed

2. Final manual check

root@osol-jeos:~# pkg purge-history
root@osol-jeos:~# du -s -h /var/pkg/catalog/
2.2M    /var/pkg/catalog/
root@osol-jeos:~# ls /var/pkg/download
ls: cannot access /var/pkg/download: No such file or directory

Step 4d: Important: Time to make a backup now

root@osol-jeos:~# pfexec init 0

Step 4e: Shrink installed system disk with ZFS Streaming

Basic idea here is then ZFs SEnd/receive will shrink disk (serialize data and remove deleted stuff)

I will use Live-CD media from OpenSolaris 200906 Virtualization Assistant Live-CD Proof of Concept

osol-0906-VirtAssit-proto-x-1.0a.isoVirtualization Assistant Live-CD X86 media (Optimized for 32bit - works on 64bit too) {113MB}

I will utilize stream.l2d.zfs and stream.sadr.zfs modules.

You can also try manually this procedure:

5993 Document procedure how to Shrink VDisk with ZFS send/recevie for P2V or V2V

First I will optimize disk with local ZFS SEND/RECEIVE streaming

From stream.l2d.zfs readme:

 1 This script will copy all content of original disk with full ZFS Send / Receive
      2 
      3 This script will primary do SHINK (compress) ZFS alocated space to by on one 
      4 continuous place (ZFS Send/Reveice serialize disk data), as a side effect you 
      5 can also change size ZFS container - you can change size of Virtual Disk.
      6 
      7 ------------------------------------------------------------------------------
      8 
      9 This script is local version of ZFS Shring procedure developed as part of JeOS:
     10 
     11 5993 Document procedure how to Srink VDisk with ZFS send/recevie for P2V or V2V
     12 http://defect.opensolaris.org/bz/show_bug.cgi?id=5993
     13 
     14 In future this script will be improved to incorporate planned ZFS Flash feature.
     15 
     16 -------------------------------------------------------------------------------
     17 
     18 You need to attach original disk as second disk and create a new disk as first 
     19 bootable one, t. m. on first bootable IDE or SCSI position to make grub happy:
     20 
     21 1. Backup original disk with OpenSolaris installation
     22 
     23 2. In VM configuration move this disk to another position on IDE or SCSI
     24 
     25 - On VMWare click on Edit VM options, point to disk and click advanced, port.
     26 - On VirtualBox just change a port to which disk is attached in main VM screen.
     27   
     28 3. Create a new disk with required size and attach it to 1 position on channel
     29 
     30 
     31 Original disk need to be disconnected before VM reboot otherwise boot loginc in
     32  kernel will be dis-oriented and system will fail to boot :-)
     33   
     34 -------------------------------------------------------------------------------
     35 
     36 Read stream-l2d.sh stript for more info.
     37 
     38 ToDo: 
     39  - Add more error checking and exit levels as Accumulated Error Codes
     40  - Ready for Ldoms (SPARC), test it

1a. Procedures stream-l2d modules itself

  • Move original OS disk
  • Add new disk, be sure is first on bus
  • Boot from VA Live-CD media (Click ESC in VMware to see media boot menu)
  • Log there as osol/justone1
  • Switch to root pfexec su -
root@virt-assist:~# cd /usr/share/virt_assist/
root@virt-assist:/usr/share/virt_assist/$ cd ./stream.l2d.zfs/
root@virt-assist:/usr/share/virt_assist/stream.l2d.zfs#
root@virt-assist:/usr/share/virt_assist/stream.l2d.zfs# ./stream-l2d.sh

For this module you need VM configured with: VCPU=1 Memory=1024MB VDISK=8GB NET
This module uses custom multi disk configuration, see more in module README.txt
----------------- General Show VM Config Information START -------------------
System: SunOS virt-assist 5.11 snv_111b i86pc i386 i86pc Solaris
This VM have configured: VCPUs: 1 Memory size: 1024 Megabytes
Mem Usage Memory: 1024M phys mem, 749M free mem
Virtual Disks present:
Total number of disks: 3
---------------------------------
 num |    name|  ctype|size [MB]|
---------------------------------
   1 |\* c3t0d0|   scsi|     8192|
   2 |  c3t1d0|   scsi|       80|
   3 |  c3t2d0|   scsi|     8192|
---------------------------------
ZFS zpools present (allready imported ones):
no pools available
----------------- General Show VM Config Information END ---------------------
INFO: We are on platform: i386
You need to pass a disk name of disk you want to zfs stream content of rpool to.
This disk must be first in system and marked my BIOS as bootable (\* in list)
stream-l2d.sh [-zerodisk] diskname
  stream-l2d.sh -zerodisk c3t0d0

This disks are visible on system:
Disk discovery
Total number of disks: 3
---------------------------------
 num |    name|  ctype|size [MB]|
---------------------------------
   1 |\* c3t0d0|   scsi|     8192|
   2 |  c3t1d0|   scsi|       80|
   3 |  c3t2d0|   scsi|     8192|
---------------------------------

root@virt-assist:/usr/share/virt_assist/stream.l2d.zfs# ./stream-l2d.sh c3t0d0

INFO: We are on platform: i386
Importing zpool rpool and streamer
NAME       USED  AVAIL  REFER  MOUNTPOINT
streamer  2.30G  5.51G  83.5K  /streamer
FastFix for BE is bussy error during shapshot (wrong shutdown)
Clean zpool zpool from old stream snapshot
cannot destroy 'streamer@stream': dataset does not exist
no snapshots destroyed
Getting streamer DUMP and SWAP size infos
Orig DUMP bs=128K size=384M
Orig SWAP bs=4K size=512M (SYS PAGESISE=4096)
Creating a full snapshot of streamer
Remove snapshots for dump and swap (we will recreate them)
Creating new zpool rpool on selected drive
  Creating a FDISK partition on whole disk
Test TI started in real mode...
Target type specified: fdisk
fdisk target prepared successfully
fdisk target created successfully
  Creating VTOC table
Test TI started in real mode...
Target type specified: VTOC
Config file not specified, default VTOC will be created
VTOC target prepared successfully
VTOC target created successfully
   Creating a ZPOOL rpool
Test TI started in real mode...
Target type specified: ZFS root pool - create
ZFS root pool target prepared successfully
ZFS root pool target created successfully
Send and Receive full ZFS stream
stream.zfs: 1.52GB 0:01:09 [         <=>         ]
Recreating DUMP and SWAP with orig values
Update bootfs on zpool rpool
Clean streams and export streamer zpool ('beadm' must see only one 'opensolaris' BE)
Update Grub/Boot and Reconfigure systems
   Activate BE
   Mount BE
   For x86 update (reinstall) grub
Creating GRUB menu in /a
Installing grub on /dev/rdsk/c3t0d0s0
stage1 written to partition 0 sector 0 (abs 4096)
stage2 written to partition 0, 271 sectors starting at 50 (abs 4146)
   Force Reconfigure (safe option :-)
   Umount BE
Now after system shutdown you need to disconnet orignial disk so you can boot to new one
If you wan to boot original disk, you need to boot Live-CD rename zpool back it with cmd:
   'zpool import -f streamer rpool'
Streamer is done, halt VA Live-CD with 'init 0'

Process take 1 minute and Shrinked disk have 1459MB

Second I will optimize disk with NFS based ZFS SEND/RECEIVE streaming

From stream.l2d.zfs readme:

  1 This script will copy all content of original disk with full ZFS Send / Receive
      2 
      3 This script will primary do SHINK (compress) ZFS alocated space to by on one 
      4 continuous place (ZFS Send/Reveive serialize disk data), as a side effect you 
      5 can also change size ZFS container - you can change size of Virtual Disk.
      6 
      7 ------------------------------------------------------------------------------
      8 
      9 This script is NFS version of ZFS Shring procedure developed as part of JeOS:
     10 
     11 5993 Document procedure how to Srink VDisk with ZFS send/recevie for P2V or V2V
     12 http://defect.opensolaris.org/bz/show_bug.cgi?id=5993
     13 
     14 In future this script will be improved to incorporate planned ZFS Flash feature.
     15 
     16 -------------------------------------------------------------------------------
     17 
     18 For this module you need VM configured with: VCPU=1 Memory=1024MB VDISK=8GB NET
     19 
     20 This module utilize PV (Pive Viwer) to show process on terminal (dev 2 = err) 
     21 so you will not see any progess if you will redirect err output to log file
     22 
     23 -------------------------------------------------------------------------------
     24 This module is non disruptive , there is no zpool renaming like in local one.
     25 
     26 First you need save a ZFS stream of installed system to writable NFS path:
     27     
     28 You need to pass a RW NFS path (hostname with path) for zfs stream archive.
     29 Use root user, NFS path must be writable by root and must allready exist
     30 ZFS stream and all it needed control information will be saved in this dir.
     31 stream-save.sh RW-NFS-PATH
     32  stream-save.sh jsc-ai.czech.sun.com/buildpool/store
     33 
     34 Read stream-save.sh stript for more info.
     35 
     36 Second you can restore OS installation from saved on NFS ZFS stream with   
     37 
     38 You need to pass a disk name of disk you want to zfs stream content of rpool to.
     39 This disk must be first in system and marked my BIOS as bootable (\* in list)
     40 You need to pass a NFS path (hostname with path) for zfs stream archive.
     41 ZFS stream and all it needed control information will be saved in this dir.
     42 At the and 'reconfig.hw.v2v' = HW reconfiguration module will be called
     43  stream-restore.sh [-zerodisk] diskname 'RW-NFS-PATH'
     44  stream-restore.sh [-zerodisk] diskname jsc-ai.czech.sun.com/buildpool/store
     45  	           
     46 -------------------------------------------------------------------------------
     47 
     48 ToDo: 
     49  - Add more error checking and exit levels as Accumulated Error Codes
     50  - Ready for Ldoms (SPARC), test it

2a. Procedures stream-sadr module itself (PART one store ZFS stream)

  • Create a Writable NFS store
  • Boot from VA Live-CD media (Click ESC in VMware to see media boot menu)
  • Log there as osol/justone1
  • Switch to root pfexec su -
root@virt-assist:~# cd /usr/share/virt_assist/stream.sadr.zfs/
root@virt-assist:/usr/share/virt_assist/stream.sadr.zfs# ./stream-save.sh jsc-ai.czech.sun.com/buildpool/store/gfv3
For this module you need VM configured with: VCPU=1 Memory=1024MB VDISK=8GB NET
This module uses custom multi disk configuration, see more in module README.txt
----------------- General Show VM Config Information START -------------------
System: SunOS virt-assist 5.11 snv_111b i86pc i386 i86pc Solaris
This VM have configured: VCPUs: 1 Memory size: 1024 Megabytes
Mem Usage Memory: 1024M phys mem, 740M free mem
Virtual Disks present:
Total number of disks: 2
---------------------------------
 num |    name|  ctype|size [MB]|
---------------------------------
   1 |\* c3t0d0|   scsi|     8192|
   2 |  c3t1d0|   scsi|       80|
---------------------------------
ZFS zpools present (allready imported ones):
no pools available
----------------- General Show VM Config Information END ---------------------
INFO: We are on platform: i386
Importing zpool rpool (forced)
NAME    USED  AVAIL  REFER  MOUNTPOINT
rpool  2.30G  5.51G  83.5K  /rpool
FastFix for BE is bussy error during shapshot (on wrong VM shutdown)
Clean zpool zpool from old stream snapshot
cannot destroy 'rpool@stream': dataset does not exist
no snapshots destroyed
Creating a full snapshot of rpool
Remove snapshots for dump and swap (we will recreate them)
Getting rpool DUMP and SWAP size infos
Orig DUMP bs=128K size=384M
Orig SWAP bs=4K size=512M (SYS PAGESISE=4096)
Saving SWAP and DUMP information
Using path /net/jsc-ai.czech.sun.com/buildpool/store/gfv3/stream.vars.sh
7z compression need 640MB of Memory, activating SWAP on rpool as safe option
You will now see SWAP activated in Memory stats:
Memory: 1024M phys mem, 730M free mem, 512M total swap, 512M free swap
Send with full ZFS stream to 7z compressor and selected nfs /net share dir
Using path /net/jsc-ai.czech.sun.com/buildpool/store/gfv3/stream.zfs.7z
stream.zfs: 1.52GB 0:24:37 [    <=>                           ]
Visually check if 'zfs send' don't report any error
Visually check if '7z compression' don't report any error
rpool stream.zfs.7z archive created as:
Sizes Real: 1635175080B Compres: 321465691B Path:
    '/net/jsc-ai.czech.sun.com/buildpool/store/gfv3/stream.zfs.7z'
Clean zpool zpool from stream snapshot
Streamer is done, halt VA Live-CD with 'init 0'

Process takes 25 minute and Shrinked 7ZIP compressed ZFS stream have 307MB

PS: If you need to use a SSH to VA Live-CD access, copy a SSHkeys from JeOS download archives

ssh -i sshvmkeyrsa.key osol@NWAM_ip
Comments:

Post a Comment:
  • HTML Syntax: NOT allowed
About

Hands-on experience with Virtualization

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today