Thursday May 12, 2016

Oracle VM 3.4 script: schedule and move VMs to a different repository

Moving VMs from one repository to an other can be something boring, mostly if your VM has many vdisks or if you have to move many vms.

On Oracle VM to completely move a VM you need to create a clone-customizer and specify target destination for each virtual-disk.

So, on activities like:

  • storage replacement (new repositories) ***
  • huge list of vms to move to a new repository
  • vms to move to a new repository with tens of virtual-disks
  • scheduled VM moving

a batch approach can help to address them with an automated script able to create the clone-customizer and move the entire vm (vm.cfg file and all virtual-disks) to the destination repository. 

*** In case of storage replacement this script does not manage the Oracle VM poolfs migration.

Moving a VM is a tricky operation and more checks are executed by the script before starting with it; these are all the checks executed by the script before starting the "move" operation:

  • Execute the first connection to the manager using "expect", needed in case of reboot of Oracle VM Manager or client host
  • Check VM Name is unique: you can have different VMs with the same name but we need to identify which one we have to move
  • Check Server-Pool name and VM membership: verify that VM we are going to move is part of the pool specified
  • Check target repository name: verify that target repository specified exists
  • Check OVS that guest the vm: verify which Oracle VM Server actually guest the VM and if this server has target repository presented
  • Check OVMCLI Release and features available: this script is compatible with Oracle VM 3.3 and 3.4
  • Check VM and all its components are not already on target repository: verify that everything is not already in place
  • Verify disk space available on target repository: verify that on target repository there is enough space to guest the VM
  • Check VM Status (running / stopped) and options specified (VM shutdown and VM restart)

Once all these checks are verified the script executes following steps:

  • Create Clone Customizer for VM moving
  • Prepare storage mappings for Clone Customizer created
  • Prepare network mappings for Clone Customizer created
  • Move VM to target repository and delete Clone Customizer
  • Verify VM moving completed successfully
  • Start the moved vm and verify it's started (optional)

Script "MoveVm.sh" v0.1 can be downloaded here.

"MoveVm.sh" script  has following characteristics:

  • Compatible with:
    • Oracle VM 3.3
    • Oracle VM 3.4
  • Requires
    • "expect" installed on the Linux system executing the script
    • able to reach port 10000 of Oracle VM Manager host

Script flowchart graphically is:

To see script syntax, just execute the script without any option:

The script won't move "Virtual Cd-Rom" (ISO files) and/or manage physical disks.

In the following example we have one VM (ol67test) where:

  • vm.cfg file is on repository named local_02
  • 3 virtual-disks are on repository named local_02
  • 1 virtual-cdrom is on repository named local_02
  • 1 physical-disk is owned by the guest
  • 2 vnic(s) are configured on two different networks

Following example will move the entire VM (vm configuration file and all virtual-disks) to repository named local_01.

Now the "moved" VM is configured with:

  • vm.cfg file is on repository named local_01
  • 3 virtual-disks are on repository named local_01
  • 1 virtual-cdrom remains on repository named local_02
  • 1 physical-disk is owned by the guest
  • 2 vnic(s) are configured as before

Log execution of this operation can be downloaded here

Obviously all this stuff can be scheduled and, thanks to this OVMCLI script, moving VMs won't be anymore a tedious operation.

Feedback and questions are obviously appreciated. 

Monday May 02, 2016

Oracle VM 3.4: create custom install ISO with updated UEK4 kernel

Oracle VM Server 3.4.1 ISO package is dedicated to server installation but, to try it, maybe someone needs to install on a desktop/laptop environment (direct install on bare metal).

It could happen that the default UEK4 kernel part of Oracle VM Server 3.4.1 build 1351, does not contain a particular fix and/or driver needed for old or newest hardware; a common problem, for example, could be related to particular graphic card available on modern desktop/laptop systems and while the installation correctly terminates, you aren't able to get console access to the system.

So, the target of this guide, is to supply a step-by-step guide to build  your custom "Oracle VM Server" ISO package with an updated UEK4 kernel release where, maybe, the fix needed is already in place.

In the example below, I'll pick-up updated kernel-uek packages for Oracle VM Server 3.4.1, actually available by yum.oracle.com ; the machine used for all this steps is based on "Oracle Linux 6.7 x86_64".

RPMs/Packages needed to execute all the operation:

  • createrepo
  • rsync
  • mkisofs
Now we can start with the step-by-step how-to: 
  • Mount the default "Oracle VM Server 3.4.1" ISO
# mount -o loop /u01/OVS-3.4.1-1351.iso /media

  • Clone ISO content to a r/w directory
# mkdir /u01/new_iso
# rsync -avz /media/ /u01/new_iso/

  • Remove RPMs that will be replaced with updated release 

# cd /u01/new_iso/
# cd Packages/
# rm -f kernel-uek-4.1.12-32.1.3.el6uek.x86_64.rpm
# rm -f kernel-uek-firmware-4.1.12-32.1.3.el6uek.noarch.rpm

  • Copy updated RPMs (you can download them from yum.oracle.com if needed)
# cd /u01/new_iso/Packages/
# cp /root/kernel-uek-* .

  • Rebuild file "listing" with updated entries due to changes above and verify differences
# cd /u01/new_iso/Server/
# ls Packages/ > listing.new
# diff listing listing.new
# mv listing.new listing


  • Prepare to update YUM repository configuration and metadata files 
# cd /u01/new_iso/Server/repodata/
# mv bdaef80f41c05cc39420d77d91edea91c3d726cb003ba168ce74b3a5580f06ba-comps-ovs-core.xml comps-ovs-core.xml

  • Update YUM metadata files and sqlite DBs
# cd /u01/new_iso/Server/
# createrepo -g repodata/comps-ovs-core.xml .

  • Generate new ISO build pointing to the temporary directory
# cd /u01/new_iso/
# mkisofs -o ../OVS-3.4.1-b1351_custom.iso -b isolinux/isolinux.bin -c isolinux/boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table -J -R -V "OVS 3.4.1 custom-build" .

  • Test your custom ISO and verify the system correctly installs

  • Update your Oracle VM Server 3.4.1 system pointing to yum.oracle.com
# cd /etc/yum.repos.d
# wget https://blogs.oracle.com/scoter/resource/blog21/ovm34_latest_x86_64.repo
# yum update -y

  • "yum update -y" example 

Feedback and question are obviously appreciated. 

Tuesday Apr 26, 2016

Oracle VM 3.4: ovm-info command to get details within VM

Continuing on the path of the first "ovm-disk" command, here you can find one new tool available for your Linux guest-vms; target of this utility, named "ovm-info", is to supply main information of the running guest on:

  • VM information, like Hostname, VM-Name, VM-Type and repository that house it.
  • CPU and Memory information (Max and used)
  • Disk information like domU device and the corresponding dom0 device (virtual or physical)
  • Network information like domU device and OVM Network configuration (id, dom0 net device and vlan-tag)
  • Other informations like H/A enabled or not, Hugepages enabled or not and actions on different operations (reboot, crash etc)

So with the same approach of the other script, specific for detailed reports on vdisks, we can install and execute this command on our running Linux guest and obtain this information by executing this command on the guest itself:

"ovm-info" command/script, once executed, query the Oracle VM Manager and obtains detailed info related to the device specified.

There are some requirements to have it correctly working and these are: 

  1. "expect" installed on the Linux VM
  2. VM able to reach port 10000 of Oracle VM Manager host
  3. Compatible with Oracle VM 3.3 and 3.4

The package containing all-the-stuff can be downloaded here ; into this tarball-gzipped file you'll find:

  • README_FIRST.txt (please, read it, it won't take more than 2 minutes)
  • FirstConn.exp (expect script that executes first connection to the Oracle VM Manager CLI port)
  • SetupSsh.exp (expect script that setup the self-authentication between the VM and Oracle VM Manager CLI)
  • install.sh (bash script to execute to correctly setup the self-authentication)
  • ovm-info (script that will be customized and copied by the install.sh process)

NB: Once the setup is completed a copy of the command "ovm-info" will be saved under "/usr/local/bin" and you can remove the install package and its directory.

Here you can find an example of the installation output (click on the image to get the entire log).

Syntax:

# ./install.sh <ovmm_host> <ovmm_oracle_linux_user_password> <ovmm_admin_manager_password>


Once the setup is completed, following success message will appear:

"OVMCLI Session successfully connected with key-based authentication!!!"

Now you can remove the package and the installation folder and verify the "ovm-info" command path:


Example output on "General Information" option (-g):


Example output on "CPU and Memory" option (-c):


Example output on "Virtual Disk" option (-d):


Example output on "Network Information" option (-n):

Example output on "Other Information" like H/A, HugePages and actions, option (-o):

Last option "Show all VM information" (-a) query for all information and option above sequentially.

Feedback and question are obviously appreciated. 

Saturday Apr 02, 2016

Oracle VM 3.4: install Oracle VM Server on EFI VirtualBox machine

Between the others, Oracle VM 3.4 introduce the option to install the Server on an EFI system.

Following steps will show how-to install Oracle VM Server 3.4.1 on a VirtualBox VM EFI based; these steps could be interesting also because new modern HW is EFI based and newest models does not accept anymore to have legacy-BIOS available.

First of all we have to create one VM dedicated to the installation; here you can find all the options selected and configuration in place:

General => Basic

  • Name: <desired VM name>
  • Type: Linux
  • Version: Oracle (64-bit) 

System => Motherboard

  • Base Memory: <desired memory amount>
  • Boot Order: <leave default>
  • Chipset: <leave default>
  • Pointing Device: <leave default>
  • Extended Features: <all selected> - see EFI

System => Processor 

  • Processor(s): <choose desired number of vcpu>
  • Execution Cap: <choose desired vcpu capacity>
  • Extended Features: <enabled>

    System => Acceleration

    • Paravirtualization Interface: Legacy
    • Hardware Virtualization: <all selected>

    Storage => Storage Tree

    • Controller: SATA
    • Devices
      • 1 vdisk dedicated to Oracle VM Server installation (I suggest 100GB dynamic vdisk)
      • virtual cd-rom with Oracle VM Server 3.4 ISO connected

    Now you can proceed to install Oracle VM Server; installation is very easy and there is no particular option to select.

    Once installed you'll see that the system, at first reboot, will automatically start the Oracle VM Server dom0; the problem really happens if you are going to stop the VM.

    In fact, once the VM stopped, if you are going to start it you'll see that the UEFI Interactive Shell v2.0 will be presented:

    To correctly boot Oracle VM Server 3.4.1 we need to specify the grub-efi configuration file with:

    fs0:\EFI\redhat\grubx64.efi

    Once Oracle VM Server 3.4 has completed the boot process we can now work to have it automatically booting.

    To have it, execute following steps:

    1. Create "startup.nsh" file (default executed on UEFI VirtualBox VM) with:
      • echo "\EFI\redhat\grubx64.efi" > /boot/efi/startup.nsh
    2. Verify that file "startup.nsh" is executable:
      • ls -l /boot/efi/startup.nsh

    Example:

    From now on, Oracle VM Server will automatically boot on this VirtualBox EFI system without any kind of interaction. 

        Thursday Mar 24, 2016

        Oracle VM 3.4: guest hot-backup on different repository (v0.8)

        Not more than a couple of hours ago Oracle VM 3.4 has been announced on Virtualization Blog.

        Here you can find the new version ( 0.8 ) of the HotCloneVm.sh script.

        Download it here

        Once downloaded, you'll find a zipped file containing:

        • "SetupSsh.sh" script to setup key-based authentication to Oracle VM Manager 3 CLI
        • "HotCloneVm.sh" script to execute guests hot-backups and proceed to apply retention policies
        • "HotCloneVM.pdf" containing a real user-guide to implement this backup strategy
        • "README_FIRST.txt" containing some tips to configure ssh timeout(s)  

        Fixes introduced in 0.8:

        • Fixed Oracle VM CLI release check 

        New features available in 0.7a:

        • Compatible with Oracle VM 3.3 / 3.4
        • Script is now able to identify Oracle VM Manager release and evaluate features enabled or not
        • New options for backup-type available: FULL, SNAP and OVA (OVA available only with 3.4 OVM Release)
          • FULL => HotClone will create a full vdisk backup on a further repository
          • SNAP => HotClone will create an ocfs2 reference-link snapshot of the vm on the same repository
          • OVA  => HotClone will create a packaged OVA file on a further repository
        • Retention is now applied on different backup-types
        • New definition of backup vm name:
          • $guest-OVA for OVA backup types
          • $guest-SNAP for ocfs2-reflink backup types
          • $guest-FULL for full-backup types
        • Retention continues to be applied also on old backups ( $guest-CLONE )

        The hot-backup obtained by this script is always a crash-consistent backup of the running vm; if the VM owns also physical-disks, the backup will contain only virtual-disks and so, a manual intervention is needed to have it correctly booting/starting.

        Here you can find one high-level picture of the target of this script:

        The idea, here, is to have low-retention backups available on one NFS repository and medium/long retention on a Tape-Library / Further solution (see NDMP); an example could be to have latest 8/15 days (from 8 to 15) on the NFS share and oldest backups on the Tape-Library / NDMP.

        So the retention can be managed between the two solutions (HotCloneVm script and Tape Library/NDMP) where:

        • HotCloneVm.sh will manage the low-retention on the NFS Repository
          • both Incremental and Full approaches are available with "SNAP" and "FULL"
        • Tape Library / NDMP will manage the long-retention on the Tape Library / NDMP solution
          • both Incremental and Full approaches can be managed by Tape Library / NDMP

        Having the low-retention on the NFS will allow, moreover, to have a fast restore point for:

        • Single VM file restore or a list of them
          • Just start the VM, that resides on the NFS repository, with a temporary IP address and get files needed.
        • Entire VM restore
          • Stop the original VM and Start a snapshot and/or Clone/Move the FULL backup from the NFS
          • You can also evaluate to start the vm on the NFS repository to be even faster

        Another option, available only with 3.4 release, is to have backups in OVA format; this option will allow to have backups that can be exported/imported to further Oracle VM Pools; for example you can use OVA exported to have the same VM available on a different Oracle VM Pool (like a clone from Production to Development) or on your laptop/desktop using VirtualBox.

        First of all you need to setup an ssh-key exchange to obtain a self-authenticated system:

        [scoter@ovmm: ~/Documents/OVM_34/HotCloneVm_0.7a]# ./SetupSsh.sh
        #####################################################################################
        You have to specify <guest id> or <guest name>:
        Use SetupSsh.sh <Oracle VM Manager host> <Linux oracle user password> <Oracle VM Manager Password> 
        Example:
        SetupSsh.sh ovm-mgr.oracle.local oracle Welcome1
        ##########################################################################################
        [scoter@ovmm: ~/Documents/OVM_34/HotCloneVm_0.7a]# 

        NB: If you have already configured ssh-key exchange vs OVMCli you don't need to execute "SetupSsh.sh" script.

        To identify syntax to be used, you have only to execute the command without any option:

        [scoter@myhost: ~/]# ./HotCloneVm.sh
        #####################################################################################
         You have to specify <guest id> or <guest name>:
         Use HotCloneVm.sh <Oracle VM Manager password> <Oracle VM Manager host> <guest name> <Oracle VM Server Pool> <target Repository> <Backup Retention> <backup_type>
         <backup_type> options are (no case-sensitive):
                - FULL => HotClone will create a full vdisk backup on a further repository
                - SNAP => HotClone will create an ocfs2 reference-link snapshot of the vm on the same repository
                - OVA  => HotClone will create a packaged OVA file on a further repository
        Example:
         HotCloneVm.sh Welcome1 ovm-mgr.oracle.local vmdb01 myPool repotarget 8d (retention will be 8 days)
         HotCloneVm.sh Welcome1 ovm-mgr.oracle.local vmdb01 myPool repotarget d8 (retention will be 8 days)
         HotCloneVm.sh Welcome1 ovm-mgr.oracle.local vmdb01 myPool repotarget 8c (retention will be 8 copies)
         HotCloneVm.sh Welcome1 ovm-mgr.oracle.local vmdb01 myPool repotarget c8 (retention will be 8 copies)
        ##########################################################################################

        As usual, your feedback will be really appreciated. 

        About

        Simon Coter is a Principal Product Manager for Oracle VM and VirtualBox.

        Search

        Categories
        Archives
        « June 2016
        SunMonTueWedThuFriSat
           
        1
        2
        3
        4
        5
        6
        7
        8
        9
        10
        11
        12
        13
        14
        15
        16
        17
        18
        19
        20
        21
        22
        23
        24
        25
        26
        27
        28
        29
        30
          
               
        Today