X

Technical articles, news, and insights
for Oracle's Infrastructure Software offerings

Oracle VM 3.4: guest hot backup on different repository (v0.8a)

Simon Coter
Director of Product Management

Updated release of the script is now available at: Hands on Backup Utilities for Oracle VM 3.4

Not more than a couple of hours ago Oracle VM 3.4 has been announced on Virtualization Blog.

Here you can find the new version ( 0.8a ) of the HotCloneVm.sh script.

 

Once downloaded, you'll find a zipped file containing:

  • "SetupSsh.sh" script to setup key-based authentication to Oracle VM Manager 3 CLI
  • "HotCloneVm.sh" script to execute guests hot-backups and proceed to apply retention policies
  • "HotCloneVM.pdf" containing a real user-guide to implement this backup strategy
  • "README_FIRST.txt" containing some tips to configure ssh timeout(s)  

Fixes introduced in 0.8a on March, 10th 2017:

  • Fixed an issue on retention policy for VMs containing "_" in the vm-name

Fixes introduced in 0.8:

  • Fixed Oracle VM CLI release check 

New features available in 0.7a:

  • Compatible with Oracle VM 3.3 / 3.4
  • Script is now able to identify Oracle VM Manager release and evaluate features enabled or not
  • New options for backup-type available: FULL, SNAP and OVA (OVA available only with 3.4 OVM Release)
    • FULL => HotClone will create a full vdisk backup on a further repository
    • SNAP => HotClone will create an ocfs2 reference-link snapshot of the vm on the same repository
    • OVA  => HotClone will create a packaged OVA file on a further repository
  • Retention is now applied on different backup-types
  • New definition of backup vm name:
    • $guest-OVA for OVA backup types
    • $guest-SNAP for ocfs2-reflink backup types
    • $guest-FULL for full-backup types
  • Retention continues to be applied also on old backups ( $guest-CLONE )

The hot-backup obtained by this script is always a crash-consistent backup of the running vm; if the VM owns also physical-disks, the backup will contain only virtual-disks and so, a manual intervention is needed to have it correctly booting/starting.

Here you can find one high-level picture of the target of this script:

The idea, here, is to have low-retention backups available on one NFS repository and medium/long retention on a Tape-Library / Further solution (see NDMP); an example could be to have latest 8/15 days (from 8 to 15) on the NFS share and oldest backups on the Tape-Library / NDMP.

So the retention can be managed between the two solutions (HotCloneVm script and Tape Library/NDMP) where:

  • HotCloneVm.sh will manage the low-retention on the NFS Repository
    • both Incremental and Full approaches are available with "SNAP" and "FULL"
  • Tape Library / NDMP will manage the long-retention on the Tape Library / NDMP solution
    • both Incremental and Full approaches can be managed by Tape Library / NDMP

Having the low-retention on the NFS will allow, moreover, to have a fast restore point for:

  • Single VM file restore or a list of them
    • Just start the VM, that resides on the NFS repository, with a temporary IP address and get files needed.
  • Entire VM restore
    • Stop the original VM and Start a snapshot and/or Clone/Move the FULL backup from the NFS
    • You can also evaluate to start the vm on the NFS repository to be even faster

Another option, available only with 3.4 release, is to have backups in OVA format; this option will allow to have backups that can be exported/imported to further Oracle VM Pools; for example you can use OVA exported to have the same VM available on a different Oracle VM Pool (like a clone from Production to Development) or on your laptop/desktop using VirtualBox.

First of all you need to setup an ssh-key exchange to obtain a self-authenticated system:

[scoter@ovmm: ~/Documents/OVM_34/HotCloneVm_0.7a]# ./SetupSsh.sh
#####################################################################################
You have to specify <guest id> or <guest name>:
Use SetupSsh.sh <Oracle VM Manager host> <Linux oracle user password> <Oracle VM Manager Password> 
Example:
SetupSsh.sh ovm-mgr.oracle.local oracle Welcome1
##########################################################################################
[scoter@ovmm: ~/Documents/OVM_34/HotCloneVm_0.7a]# 

NB: If you have already configured ssh-key exchange vs OVMCli you don't need to execute "SetupSsh.sh" script.

To identify syntax to be used, you have only to execute the command without any option:

[scoter@myhost: ~/]# ./HotCloneVm.sh
#####################################################################################
 You have to specify <guest id> or <guest name>:
 Use HotCloneVm.sh <Oracle VM Manager password> <Oracle VM Manager host> <guest name> <Oracle VM Server Pool> <target Repository> <Backup Retention> <backup_type>
 <backup_type> options are (no case-sensitive):
        - FULL => HotClone will create a full vdisk backup on a further repository
        - SNAP => HotClone will create an ocfs2 reference-link snapshot of the vm on the same repository
        - OVA  => HotClone will create a packaged OVA file on a further repository - available from 3.4
Example:
 HotCloneVm.sh Welcome1 ovm-mgr.local vmdb01 myPool repotarget 8d FULL (retention will be 8 days)
 HotCloneVm.sh Welcome1 ovm-mgr.local vmdb01 myPool repotarget d8 SNAP (retention will be 8 days)
 HotCloneVm.sh Welcome1 ovm-mgr.local vmdb01 myPool repotarget 8c OVA  (retention will be 8 copies)
 HotCloneVm.sh Welcome1 ovm-mgr.local vmdb01 myPool repotarget c8 FULL (retention will be 8 copies)
##########################################################################################

As usual, your feedback will be really appreciated. 

Join the discussion

Comments ( 35 )
  • guest Friday, October 7, 2016

    I am running OVM 3.3.3 This script doesn't work there on OVM Manager server

    ./HotCloneVm.sh xxxxx ovmm erpdevdb01 OVM-PROD ERP-DEV-REP01 8d FULL

    ./HotCloneVm.sh: line 64: ${7,,}: bad substitution


  • Simon Friday, October 7, 2016

    Hi "Guest",

    can you report the complete log of the execution ?

    Please execute the script with:

    sh -x ./HotCloneVm.sh <parameters>

    Thanks

    Simon


  • guest Tuesday, October 11, 2016

    With a little modification in the code, you can have more speed in you script:

    # Execute first-connection in case of reboot of Oracle VM Manager, specifying admin password

    ssh admin@$mgrhost -p 10000 -oBatchMode=yes "set OutputMode=Verbose"

    if [ $? != 0 ]; then

    /usr/bin/expect FirstConn.exp $password $mgrhost

    fi


  • Simon Tuesday, October 11, 2016

    Hi guest,

    you mean on the first connection ?

    First connection is something particular: if you restart your Oracle VM Manager service (or your Oracle VM Manager server) the first connection must happen supplying a password (key won't be usable for the very first connect from an host) so it's needed to execute it by using "expect".

    So due to the fact that this script is going to be scheduled we cannot have a fault due to the restart of Oracle VM Manager service and I evaluated to have the very first connection always using "expect".

    In fact, all the other connections are executed without it.

    Let me know if I correctly understood your comment.

    Thank you for the feedback.

    Simon


  • DAVIDE SERENO Friday, December 23, 2016

    I used your scripts to create a snap copy of 2 virtual machines. Is there any way to apply retention policy (8 days)?

    I created both copies on december the 2th, using an 8 day retention, but copies of virtual machines are still under "unassigned virtual machines" in ovm manager.


  • Simon Friday, December 23, 2016

    Hi Davide,

    to have the retention applied you have to keep scheduled your script.

    If, for example, you run the script today to get a new backup the retention will be applied (and so old backup removed).

    So, the retention does work if the script runs.

    Simon


  • guest Thursday, January 19, 2017

    Hello,

    I just to try to use your script, it works perfectly for me, but I have a question, if I need to backup also information about network interfaces what I should add to script? Because I have the VMs with 2 and more network interfaces and whan backup from this WMs are done, they are no have the network interfaces.

    Thanks


  • Simon Thursday, January 19, 2017

    Hi Guest,

    vNics has been ignored as default because if you start a clone-vm (or a snap) the same won't create any type of problem to the source-vm.

    If the snap also owns vNICs a possible snap/backup started could create outage for the guest-source.

    I see two different options for you:

    - edit the HotClone script where you can add vNICs to the clone-customizer

    - create your own script that add vNICs to a VM, based on your reqs

    Thanks

    Simon


  • Michal Tuesday, January 24, 2017

    I have LAB with all Storage/Repositories on NFS.

    When I executed HotCloneVm.sh with "3d FULL" got Error on OVMM GUI:

    OVMRU_007013E Cannot thin clone: disc_xp1, to: repo-nr1. The clone target does not support thin cloning. [Tue Jan 24 15:59:30 CET 2017]

    Delete VM Clone Definition: vDisks-xp1-20170124-1600 from VM: xp1 - Success.

    VM with xp1 name is currently Running.

    Is this normal situation ?


  • Simon Tuesday, January 24, 2017

    Hi Michal,

    if you open the PDF attached to the script, between all the details, you'll see that one of the requirements is that the source VM has to reside on an OCFS2 repository.

    Unfortunately there is no option to get a low-level snap on NFS repositories today.

    Thanks

    Simon


  • Michal Thursday, February 2, 2017

    Hi Simon,

    I've used your script with options FULL and SNAP. Looks like this step is done. Now I see Unassigned Virtual Machines with schema:

    vm_name-FULL-date-time

    vm_name-SNAP-date-time

    I know that restore from FULL is easy - clone this or move to main repository.

    But.. I do not know HOW to implement SNAPSHOT from vm_name-SNAP-date-time ?

    You are writing "Stop the original VM and Start a snapshot"

    so ... I've stopped original VM and ... ?

    Could you explain whole process "Start a snapshot" ?

    Its by your script or GUI OVM ? :/

    Thanks.


  • Simon Friday, February 3, 2017

    Hi Michal,

    you can have two different type of clones: SNAP and FULL.

    In case of restore from FULL you have two options:

    1. start the FULL clone from the repository where it resides (different from the source VM repository)

    2. move-back the FULL clone to the source-VM repository and then start it

    In case of restore from SNAP you only need to stop the source-VM and start the SNAP-vm (I would also suggest you to rename the vm-name to something different).

    While revert-back to a known SNAP/FULL clone you also have to:

    - re-add vNICs as the source-vm

    - re-add possible physical-disks that originally were presented to the source-VM

    That said, I'm going to work on this script and introduce a couple of improvments like, for example, the ability to manage the restore/revert-back for a VM.

    Simon


  • surendra kota Monday, March 6, 2017

    Hi Simon,

    My name is Surendra working as an OVM administrator for 5 years now.

    I have gone through the blog on “guest hot backup on different repository”, found very helpful for the hot guest backups, in fact felt happy about this feature of hot clone/copy/ova.

    1.

    However I would like to understand is there a way that we can automate this as a backup job schedule including retention policies ?

    2.

    Can we use this as for snapshot purpose as in vmware, because I see the snap option here is creating a complete vm copy into unassigned virtual machine folder, instead can we create a child vd like in vmware ?


  • Simon Monday, March 6, 2017

    Hi Surendra,

    1) you can schedule this script using your own scheduling tool and you can also use crontab available on the Oracle VM Manager machine if you want.

    2) you can create snap(s) using the script (or also by using Oracle VM Manager BUI); the "simple clone" on an OCFS2 repository is a real copy-on-write of the original VM (using OCFS2 reference-link); that said, there is no automation related to the snapshot(s) created so, in case you want to revert to a previous snap, you have to stop the original VM and then start the snapshot created.

    Hope it helps

    Simon


  • Michal Tuesday, March 7, 2017

    Hi Simon,

    Is there an option to use some password wallet tool (e.g. Oracle Wallet) or any other more secure method to provide Oracle VM admin password (in execution of HotCloneVm.s)? Keeping password in clear text is not a good option from security point of view.

    Thank You,

    Michal


  • Dar&iacute;o Wednesday, March 8, 2017

    Hi Simon,

    First of all, I wish congratulate you for the great help you always give us.

    I'm doing a FULL backup and everything goes well until show the message: "Waiting for Vm moving to complete ......", gets into a loop.

    I have checked that:

    - Create the clone of the machine.

    - Move the clone to another repository.

    But it stands still in this step it producing the loop.

    Thanks for your help.

    Darío.


  • Simon Wednesday, March 8, 2017

    Hi Michal,

    the script does not really need the password to access Oracle VM Manager.

    In fact, the "SetupSsh" script, part of the package, executes the ssh-key adding to have password-less access to Oracle VM Manager.

    The only problem is related to the first access: if the Oracle VM Manager has been restarted, the first access by the CLI will ask for a password even if the ssh-key exchange is already in place.

    So, I had to introduce the first login, based on expect, to execute the first access (and so the password that is needed).

    That said, if you schedule the script using proper scheduling tools (like crontab, for example) the password will be written only into the scheduling tool; so, suppose to you crontab, you can schedule with following syntax:

    HotCloneVm.sh Welcome1 localhost ca-osbovm ca-pool ca-nfs-backup 8c SNAP > backup_XXX_$YYY_$today.log

    and, into the log file generated, you won't see the password used to access Oracle VM Manager.

    Obviously I'll take into consideration your suggestion and, never say never, maybe it could be something on which we could work.

    Thanks

    Simon


  • Simon Wednesday, March 8, 2017

    Hi Dario,

    by executing a FULL backup you are also going to move the entire initial snapshot to a different repository (that usually is an NFS one).

    So, based on the size of source virtual-disks and based on network/disk performance, you'll have to wait "x" time to have the snap completely moved to the NFS repository.

    So, just to give you an example, here the output of the log:

    OVM> moveVmToRepository Vm name=ca-osbovm-CLONE-20170308-1420 CloneCustomizer=ca-osbovm-20170308-1420 targetRepository=ca-nfs-backup

    Waiting for Vm moving to complete......10 seconds

    Waiting for Vm moving to complete......20 seconds

    Waiting for Vm moving to complete......30 seconds

    Waiting for Vm moving to complete......40 seconds

    Waiting for Vm moving to complete......50 seconds

    Waiting for Vm moving to complete......60 seconds

    Waiting for Vm moving to complete......70 seconds

    Waiting for Vm moving to complete......80 seconds

    Waiting for Vm moving to complete......90 seconds

    Waiting for Vm moving to complete......100 seconds

    Waiting for Vm moving to complete......110 seconds

    Waiting for Vm moving to complete......120 seconds

    Waiting for Vm moving to complete......130 seconds

    Waiting for Vm moving to complete......140 seconds

    Waiting for Vm moving to complete......150 seconds

    Waiting for Vm moving to complete......160 seconds

    Waiting for Vm moving to complete......170 seconds

    Waiting for Vm moving to complete......180 seconds

    Waiting for Vm moving to complete......190 seconds

    Waiting for Vm moving to complete......200 seconds

    Command: moveVmToRepository Vm name=ca-osbovm-CLONE-20170308-1420 CloneCustomizer=ca-osbovm-20170308-1420 targetRepository=ca-nfs-backup

    Status: Success

    Time: 2017-03-08 14:24:50,758 CET

    JobId: 1488979282137

    where you can see that the operation "moveVmToRepository" starts and then, the script wait for the job completion before proceed with the next steps; so there is a loop that, each 10 seconds, verifies that the "moveVm" job has been completed or not.

    This had been introduced into one of the latest releases of the script just to avoid a possible timeout (session timeout) for the backup script.

    If you want to check the real status of the operation execution, you can check the job running by using Oracle VM Manager or the CLI interface (show job id=<your_id>).

    FYI, the machine above, on my environment, is 5GB of real used disk-space and I had to wait 200seconds on my env.

    Hope it helps.

    Simon


  • Dar&iacute;o Wednesday, March 8, 2017

    Thanks for your quick response.

    Ok, I understand what you are saying me, but I start receiving the "waiting messages" once the machine has been migrated to the NFS repository and never stop :S

    OVM> moveVmToRepository Vm name=pruebaovm-CLONE-20170308-1314 CloneCustomizer=pruebaovm-20170308-1314 targetRepository=MT_REPO_NFS

    admin@ovmvm.domain.local's password:

    Waiting for Vm moving to complete......10 seconds

    admin@ovmvm.domain.local's password: Command: moveVmToRepository Vm name=pruebaovm-CLONE-20170308-1314 CloneCustomizer=pruebaovm-20170308-1314 targetRepository=MT_REPO_NFS

    Status: Success

    Time: 2017-03-08 13:44:13,751 WET

    JobId: 1488979143477

    Waiting for Vm moving to complete......20 seconds

    Waiting for Vm moving to complete......30 seconds

    Waiting for Vm moving to complete......40 seconds

    Waiting for Vm moving to complete......50 seconds

    Waiting for Vm moving to complete......60 seconds

    .

    .

    .


  • Andrey Friday, March 10, 2017

    Hi Simon.

    Many thanks for your help in organizing a hot backup of virtual machines.

    I've tested your script many times in version 0.8, but not once so that the script would delete virtual machines after the expiration of the retention period.

    I put the script in crontab in a test environment with a full backup every 15 minutes, with the condition of storing the last two copies, but the script does not remove extra backups.

    50,05,20,35 * * * * /HotCloneVm_0.8/HotCloneVm.sh Jet12345 10.0.4.215 siteB_vm SiteB-pool Backup_repo c2 FULL > /siteB_backup.log

    Could you tell me what I'm doing wrong? Where do I send the script execution log with the "set -x" options for you?

    By the way, in my test environment the script of version 0.6 works fine, creates and removes backups. But with version 0.8 I had problems only with delete expired backups - the rest of the functionality works fine.

    Thank you in advance.


  • Simon Friday, March 10, 2017

    Hi Andrey,

    I posted an updated release of the script (0.8a) with the fix for your problem.


  • guest Tuesday, March 14, 2017

    Hi Simon, thanks for your blog.

    I have executed SetupSsh.sh and everything is correct, it returns the message "OVMCLI Session Succesfully connected with key-based authentication!!!". But, when I run HotClone.sh, it validates the first time and the rest of operations asks me for the admin@ovmmgr password. How I can solve this? Thank you.


  • Simon Tuesday, March 14, 2017

    Hi,

    I suspect that the is an ssh issue.

    Are you executing the script locally on the host that guest Oracle VM Manager or remotely ?

    The script "SetupSsh.sh" creates an "authorized_keys" file on the linux user "oracle" at the following path:

    /home/oracle/.ssh/ovmcli_authorized_keys

    You can see how-to debug ssh access at this link, for example:

    http://www.linuxquestions.org/questions/linux-server-73/passwordless-ssh-doesn%27t-work-but-works-when-debugging-4175517920/

    but there are so many that you can see on how-to debug ssh-access based on key authentication.

    Hope this helps.

    Simon


  • guest Friday, March 17, 2017

    Hi, I executed the script in OVManager host. Also, I executed this and works correctly (https://docs.oracle.com/cd/E18476_01/doc.220/e35316/advanced_option.htm#ELCHK125). But I can't get work the script. I hope to get it, I will tell you. Thank you.


  • Luis Tuesday, July 18, 2017
    Hi Simon.

    I have your process working fine months and this Friday began to fail.

    After move the Full clone to a back repository the process is hang.

    Waiting for Vm moving to complete......330 seconds
    Waiting for Vm moving to complete......340 seconds
    Command: moveVmToRepository Vm name=DIIFAP03-CLONE-20170718-1238 CloneCustomizer=DIIFAP03-20170718-1238 targetRepository=ENGIE_BCK_REP
    Status: Success
    Time: 2017-07-18 12:44:38,095 CEST
    JobId: 1500374330581
    OVM> Connection closed.
    Waiting for Vm moving to complete......350 seconds
    Waiting for Vm moving to complete......360 seconds
    Waiting for Vm moving to complete......370 seconds
    Waiting for Vm moving to complete......380 seconds
    Waiting for Vm moving to complete......390 seconds

    The copy was fine but it still in the loop, it's like this "while" statement can't stop:

    # Wait until moveVmToRepository job completed

    job_id=`ssh admin@$mgrhost -p 10000 list job |grep -i "Move Vm" |grep $guest-CLONE-$today |cut -d ":" -f2|cut -d " " -f1`
    done=0
    i=$each_cycle
    while [ $done -lt 1 ]; do
    echo "Waiting for Vm moving to complete......$i seconds"
    let i=i+$each_cycle
    sleep $each_cycle
    done=`ssh admin@$mgrhost -p 10000 show job id=$job_id |grep -c "Summary Done = Yes"`
    done


    Could you give an idea how to solve it?

    I have not backup for 3 days.

    Thank's for your help.

    Luis
  • Luis Tuesday, July 18, 2017
    Hi again Simon,

    I think I have a cue about what is happening, the line

    job_id=`ssh admin@$mgrhost -p 10000 list job |grep -i "Move Vm" |grep $guest-CLONE-$today |cut -d ":" -f2|cut -d " " -f1`

    It's getting NULL, because

    [root@ovmbu002 ~]# ssh admin@10.248.52.2 -p 10000 list job
    OVM> list job
    Command: list job
    Status: Failure
    Time: 2017-07-18 12:57:37,872 CEST
    Error Msg: GEN_000000:An exception occurred during processing: No such object (level 1), cluster is null:
    Tue Jul 18 12:57:37 CEST 2017
    OVM> Failed to complete command(s), error happened. Connection closed.

    But I don't know why OVMCLI is falling.

    Any idea

    Regards

    Luis
  • Simon Wednesday, July 19, 2017
    Hi Luis,

    sorry for my late reply but I had been busy this week.
    Yep, the problem is due to what you discovered.
    I would suggest you to open a SR to MyOracle Support on this issue (No such object (level 1), cluster is null) to get a fix for it.
    Once did, the script will continue to work again.
    Let me know if you need further details.
    Thanks

    Simon
  • Savo Tuesday, August 15, 2017
    Hi Simon,

    I Have the same problem like Dario. I choose FULL backup with 1d retention period. The VM moves to target repository but the job from ssh never finishes and i am getting on the output the following:
    Waiting for Vm moving to complete...... seconds

    And this never stops.

    Also, vm backup never get's to the Unsigned VM folder, and the old VM does not get deleted.

    Version of Oracle VM Manager and hosts is 3.4.2.

    Any help is appreciated.

    Regards,
    Savo
  • Simon Wednesday, August 16, 2017
    Hi Savo,

    you should check the logs of your Oracle VM Manager installation; my suspect is that something is not correctly working on Oracle VM Manager.

    Simon
  • Savo Friday, August 18, 2017
    Hi Simon,

    Thanks for the answer. I found what the problem was.

    The error is due corrupted Oracle Vm manager database.


    Problem is solved with restoring database from an earlier backup.

    It is very strange that also a few people from here have the same problem, also have corrupted Oracle VM database.
  • Jaime Tuesday, October 17, 2017
    Hi, I have had exactly the same problem as Savo. I have done a rebuild of the database, and I have repopulated servers and storage and perfect! thanks for this very good script
  • Remco Wednesday, January 31, 2018
    Thanks a bunch for the effort you put into this Simon, the script works like a charm, love it!
  • Simon Wednesday, January 31, 2018
    Hi Remco,

    you're welcome.
    Consider that soon the updated release of the script, with many enhancements will be available.
    Thanks

    Simon
  • Nick Monday, March 12, 2018
    Hello Simon and thank you for your efforts.

    I want to take one backup everyday and put it in one folder and other backup every 15 days and put it in another folder. If I make the first one with the switch c10 and the other one d180 for the same VM, will they execute like two different scripts or will it still delete images after the 10th one?

    Cheers!
  • Simon Tuesday, March 13, 2018
    Hi Nick,

    I would suggest you to upgrade to the latest version of the script (v1.0) available at:

    https://blogs.oracle.com/scoter/hands-on-backup-utilities-for-oracle-vm-34

    and then use the "Preserve" option.
    If you want to save it on two different repositories you should create your own script that manage the vm configuration file change.

    Simon
Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.Captcha

Recent Content

Oracle

Integrated Cloud Applications & Platform Services