Saturday May 26, 2012

example of transcendent memory and oracle databases

I did some tests with tmem using an Oracle Database 11gR2 and swingbench setup. You can see a graph below. Let me try to explain what this means.

Using Oracle VM 3 with some changes booting dom0 (additional parameters at the boot prompt) and with UEK2 as a guest kernel in my VM, I can make use of autoballooning. What you see in the graph below is very simple : it's a timeline (horizontal)of how much actual memory the VM is using/needing. I created 3 16GB VMs that I wanted to run on a 36GB Oracle VM server (so more VM memory than we have physically available in the server). When I start a 16GB VM (vertical) the Linux guest immediately balloons down to about 700Mb in size. It automatically releases pages to the hypervisor that are not needed, it's free/idle memory otherwise. Then I start a database with a 4GB SGA, as you can see, the second I start the DB, the VM grows to just over 4GB in size. Then I start swingbench runs, 25, 50, 100, 500, 1000 users. Every time such a run starts, you can see memory use/grab go up, when swingbench stops it goes down. In the end after the last run with 1000 users I also shut down the database instance and memory drops all the way to 700Mb.

I ran 3 guests with swingbench and the database in each and through dynamic ballooning and the guests cooperatively working with the hypervisor, I was able to start all 3 16GB VMs and there was no performance impact. When there was free memory in the hypervisor, cleancache kicked in and guests made use of those pages, including deduping and compression of the pages.

If you want to play with this yourself, you can run this command in dom0 to get decent statistics out of the setup : xm tmem-list --long --all | /usr/sbin/xen-tmem-list-parse. It will show you the compression ratio, the cache ratios etc. I used those statistics to generate the chart below. This yet another example of how, when one can control both the hypervisor and the guest operating system and have things work together, you get better and more interesting results than just black box VM management.

tmem>

from the research department - ramster in Linux 3.4 staging drivers

Some developers on my team have spent quite a few years on doing research and development on new ways of doing memory management better across servers, both physical and virtual. You often hear me talk about cooperative memory management in the context of Oracle VM. I believe it is important to be able to do memory overcommit the right way and have both the guest OS and the hypervisor know what's going on, instead of having the hypervisor do things behind the guest's back. Transparent memory overcommit causes unpredictable behavior for applications and while this might be of little concern when consolidating lots of small VMs that aren't doing much interesting. When you have applications that are, by nature, memory intensive, such as an Oracle database or any sort of production environment, then there need to be better ways to do this. Better, here, meaning -> more efficient, more predictable and more cooperative between host and guest.

It's a long road to getting all these stars aligned but we are very close now in terms of Linux kernel features to help with this. Over the last few years Dan Magenheimer has been working hard on what's called transcendent memory, a collection of features that include cleancache, frontswap and ramster. Now with Linux 3.4, we have tmem, cleancache and ramster in kernel upstream. Frontswap is still pending but we're working on it.

As a refresher, you can read more about cleancache in this writeup I did just bout a year ago. Frontswap is more tricky, because here we can use transcendent memory as a swap device. Pages swapped out cannot just disappear, they have to be persistent until the OS decides to no longer need them. In other words, with cleancache we provide extra memory(extra cache) but with the understanding that that extra memory might just disappear. With frontswap, we provide swap space through the same mechanism, with the understanding that it's not going to just disappear. One place where frontswap is different from regular swap, is its size. A swapdevice is pre-created with a specific size, a file acting as a swap device or a physical/logical volume. They do not grow or shrink dynamically. frontswap is more dynamic. The advantage of frontswap is that you can use ram on a remote server or in the hypervisor as very fast storage. It can also do compression of memory pages using zcache.

If you want to play with ramster, Dan wrote up a simple how-to.
Dan also wrote up a presentation which you can find here.

Looking forward to frontswap getting into the kernel and many of these features get into customers hands through the Unbreakable Enterprise Kernel and as products evolve and features get management also into Oracle VM.

Thursday May 24, 2012

The case for Oracle VM

As I mentioned a few days ago, at a customer visit earlier this month we were discussing Oracle VM and Oracle Linux advantages and the customer was surprised to hear how many features we include in our offerings. My previous write up was about Oracle Linux, this one is about Oracle VM.

In many ways, the Oracle VM solution has similar advantages. For us to deliver a solution and for our customers and partners to have a stack that's built to work together across products. Because Oracle VM subscriptions include the right to use a number of products, such as Oracle Enterprise Manager 12c and Oracle Enterprise Manager OpsCenter, we end up with an impressive, all-in set of features. Often times however this is not obvious to our users or evaluators and it is important to look at the total package and not just sub components. What I am going to describe is included in our Oracle VM subscription.

First of all, pricing... our virtualization and infrastructure management solution has a very simple model :

  • Oracle VM premier limited (for 1 or 2 socket servers) : $599 annual support subscription per server
  • It doesn't matter how many VMs you run, it doesn't matter how much RAM is in the box, it doesn't matter how many cores or threads the sockets have. 1 or 2 sockets in the box. That's the price. Annual support subscription fee. No license. Including all the products I will be detailing below.
    So if you have a 2 socket server with 256GB RAM... $599. case closed. 24/7 support.

  • Oracle VM premier (for larger than 2 socket servers) : $1199 annual support subscription per server
  • Same deal. If you have a 4 socket server, or even an 8 socket server with 4 TB of RAM... $1199 subscription. period. No virtual RAM, no per 2 sockets, no multiple options, no starter pack for 2 servers... Simple pricing. 24/7 support.

    Also, as with Oracle Linux, if you purchase an Oracle Server with the Premier support for Systems, it also includes everything I am talking about without the need for extra per server subscriptions.

    Oracle VM support subscriptions for both premier and premier limited comes with the following products and right to use :

  • Oracle VM server
  • Oracle VM server is the product that is installed on each server. It contains the hypervisor and the management console/agent to talk to Oracle VM Manager. This is effectively what you get a support subscription for. A tiny installation of about 200Mb worth of RPMs on a local hard drive or flash drive.

  • Oracle VM Manager
  • Oracle VM Manager is our centralized management console. It is an Oracle WebLogic Server 11g application that uses an Oracle Database as data repository. You typically only need 1 instance to manage a very large set of servers and virtual machines. Oracle VM Manager is the management hub for Oracle VM. It has a web-based admin interface and the management product can be installed on a Linux server, typically outside the Oracle VM server pools. We do not charge extra for this, the use and support of Oracle VM Manager is included in the server subscriptions. More details further down...

  • Oracle Enterprise Manager 12c Cloud Control
  • OEM 12c is our complete management product which can manage from application to disk. It contains management for our various applications, database, middleware, operating systems, virtualization layer, it offers configuration management, lifecycle management, physical and virtual environments, cloud portal and so on. The product consists of many management packs, the components that manage Oracle Linux and Oracle VM (cloud portal, private cloud, self-service portal, VM management, IaaS layer services, cloud API....) are included with Oracle VM subscriptions. These are not extra licensed components, they're part of the package!

  • Oracle Enterprise Manager Ops Center 12c
  • EMOC is a very powerful management product that's focused on infrastructure management, our Oracle systems (X86 and SPARC) (firmware, switches, hardware fault analysis, My Oracle Support integration, bios configuration etc...), our Operating Systems (Oracle Solaris, Oracle Linux), Virtualization products (Solaris Containers, Oracle VM SPARC, Oracle VM x86) and a connector into OEM12c to tie them together. Again, when you have an Oracle VM server subscription, you have the right to use EMOC as part of the stack included with its feature set It also contains an IaaS layer and a cloud API and cli.

    Both EM12c and EMOC connect to Oracle VM Manager instances behind the scenes. They can discover or sync up the information from an existing Oracle VM Manager/Server configuration and you can literally switch back and forth. So you start out installing a bunch of Oracle VM servers, install an Oracle VM manager instance, discover the servers, then install EM12c and/or EMOC, discover Oracle VM Manager and now you have access to their feature sets on top. You can still connect to / log into Oracle VM Manager and so on.

    So when we say "Oracle VM" and when you compare "Oracle VM" without various competitors, we really provide you with all of the above, and our feature checklist and offering is all of the above. Often times people just use Oracle VM Manager/Server without also including the use of EMOC and/or EM12c. Our competitors have many products that are all separately licensed pieces and bits that have to be installed together. Look at the solution bundle. We have no per VM, per 96GB, per socket, per feature pricing... remember, $599 and $1199 subscription.

    Here's a few features that are part of all this :

  • Cloud Infrastructure administrator portal
  • Set up and manage the lowerlevel infrastructure, hypervisor, provisioning of storage...

  • Self service administrator
  • Set up higher-level services, users, roles, access boundaries (assign roles and users to zones)
    Set up charge back plans, publish software components (VMs, full application stacks for self service users)

  • self service user portal
  • end user, developer can log in and see their assigned infrastructure quota and create application setups based on software component from the libraries

  • Consolidation planning physical server assets and P2P or P2V advisories
  • Since EM12c has a view of physical and virtual servers and applications inside, it can provide very detailed planning of which systems are good targets to consolidate, because it has an application view and a complete end to end stack overview, it is much more detailed and capable than simple VM analysis.

  • Zone management (multiple pools of servers into a larger grouping)
  • Depending on administrators needs its possible to configure larger zones (pools) of systems as units

  • Storage management through Oracle VM Storage Connect
  • We can manage iscsi, fibre channel, NFS servers from our management console. It is possible to just provision large chunks of storage and provide credentials to Oracle VM Manager and we can handle the details behind the scenes. Add a new lun? no problem, we will do it. EM12c and EMOC expose this functionality and connect through Oracle VM Manager to the storage.

  • Oracle VM assemblies and templates
  • pre-created virtual machines that include complete Oracle product installations, database, siebel, peoplesoft, ebusiness suite etc... these can eb downloaded from the Oracle website and uploaded into Oracle VM Manager or the EM12c software library for easy deployment of complete solution stacks.

    With assemblies we provide complete multi-tiered environments, including scale-out options. Need more webservers, need more database nodes? We can add/remove them on the fly, based on the application and business need, not just based on what the blackbox VM is doing.

  • Chargeback
  • It is possible to create charge back plans, either your own or using existing models. Whether it's basic charge back for virtualization or depending on management packs and oracle products up the stack for other products as well.

  • build, test, deploy
  • Because of the templates and configuration management for applications support, we can let developers build and create solutions which then get uploaded into the software library for use in product deployments. Since EM12c understands the entire product stack, application to disk, we can automate and reproduce not just simple VM deployments but include application configuration on top as well. Creating a VM with an OS is easy. Deploying a VM with an OS is easy. Deploying and configuring an entire software stack with various applications and database(s) is error prone and can take a lot of time. We can automate it for you, reproduce it, exactly the same steps, every time. Top to bottom.

  • Various abstraction layers
  • Through EM12c, EMOC and Oracle VM Manager you have an easy to use, graphical user interface.
    Through EM12c and EMOC you have access to a complete cloud API.
    EMOC offers an IaaS RESTful interface, a JAVA API library and a complete CLI.
    EM12c offers an IaaS RESTful interface

  • complete monitoring and management
  • EM12c manages and monitors every target in the stack, the applications, the operating systems, the VMs, the hypervisors. You can define jobs, metrics, thresholds, corrective actions. Because we collect information across the entire stack, you can set up auditing rules, you can compete systems top to bottom, you can do change management across the entire infrastructure, physical and virtual. To make it easy, it is possible to create groups of systems to manage as a unit. Configuration management is now more critical than ever. In a cloud infrastructure the number of VMs, and as such, the number of deployed applications, grows exponentially. We keep track of every aspect through a single source/database :) of truth.

  • Roles, LDAP directory services
  • Complete integration of LDAP with EM12c and EMOC. The ability to define fine grained access to all these components. Storage, servers, functions.
  • integration with My Oracle Support
  • Patch advisories, service request management, downloading new templates into the software library

  • standard virtualization features
  • Not to forget the fact that this obviously also includes Live migration, Dynamic Power Management (DPM), Dynamic Resource Management(DRS), storage management, network management(setting up bonding, creating virtual networks, vlan support, built-in HA (VM failover, server failover), cpu pinning, clustered filesystem for storage repositories, direct lun/storage access,...

    Installs on the largest X86 servers available. 4TB RAM, 160 threads, virtual machines up to 2TB of ram and up to 128 virtual CPUs or as many virtual CPUs the guest operating system can handle. Support for Linux, Oracle Solaris, Windows (including Windows logo signed drivers).

    So this is the Oracle VM offering, this is a summary (and I am not even trying to be complete here) of our features. Need an API, need a cli, need a UI, need a cloud portal, need monitoring or management, need roles and responsibilities, need centralized account management, we have it all. So when you compare Oracle VM with our competition out there... compare the included feature set in the subscription.

    A few useful links :

  • EMOC API/CLI reference
  • EM12c API
  • Oracle Virtualization page
  • Saturday May 19, 2012

    using ovm_utils

    Now that we released Oracle VM 3.1 and I just added ovm_utils 0.5.2 on ARU, it's time to give a few examples/use cases of the utilities :)

    The utilities are available for download from http://support.oracle.com search for patch name 13602094.

    It's a small 2.5Mb zip file which you should install on the manager node and unzip in /u01/app/oracle/ovm-manager-3. There are manpages for the utilities as well. These are installed in /u01/app/oracle/ovm-manager-3/ovm_utils/man/man8. To make it easy, simply add this to your MANPATH : export MANPATH=/u01/app/oracle/ovm-manager-3/ovm_utils/man.

    To use the utilities you need the admin username and password for the manager instance and in general you can just use localhost as the hostname. If you want to be able to connect to a remote manager instance, you have to first enable SSL on the manager node. By default we listen only on 127.0.0.1. There is an option in the utils to make a secure connection, use -X, but again, you first have to enable SSL for this. You can enable secure connections using ovm_utils and the addkeystore command. We have a support note for this.

    So the format is something like : ./ovm_managercontrol -u admin -p mypassword -h localhost -c [options...]..
    There are quite a few commands in the various utilities at this point. I broke the utilities up into categories :br

  • ovm_managercontrol : generic manager level functions
  • yuminfo : display yum configuration
    setupyum : configure yum
    addkeystore : configure the SSL keystore
    keystoreinfo : display keystore file and path
    createcpugroup : create a CPU Compatibility group
    removecpugroup : delete CPU Compatibility group
    addservertocpugroup : add server to CPU Compatibility group
    removeserverfromcpugroup : remove server from CPU Compatibility group
    listcpugroups : list all CPU Compatibility groups and its servers
    setsessiontimeout : set VM Console session timeout (Oracle VM 3.1+)
    getsessiontimeout : display VM Console session timeout(Oracle VM 3.1+)

  • ovm_repocontrol : generic repository level functions
  • list : list all repositories
    status : query repository status
    info : display repository information
    refresh : send a repository refresh command
    fixrepo : fix a failed repository create
    create : create repository on a storage device

  • ovm_poolcontrol : generic server pool level functions
  • status : display the current status of the server pool
    info : display server pool info such as memory, utilization, servers,...
    list : list all server pools registered with this Oracle VM Manager instance
    events : list all latest event types for this server pool
    addserver: add server to this server pool (-S server name)
    removeserver: remove server to this server pool (-S server name)
    refresh : refresh storage in the server pool

  • ovm_servercontrol : generic server level functions
  • start : use IPMI to power-on the server
    stop : shutdown the server
    kill : use IPMI to power-off the server
    mainton : turn on maintenance mode for this server
    maintoff : turn off maintenance mode for this server
    status : display the current status of the server (Running,Stopped,...)
    info : display server info such as memory, utilization, server-pool,...
    lock : temporarily lock the server disallowing any updates to happen
    upgrade : send an upgrade command to the server
    list : list all servers registered with this Oracle VM Manager instance
    events : list all latest event types for this server
    discover : discover server with provided hostname or IP address and Oracle VM agent password (-P)
    listnfsexports : list NFS exports on this server
    deletenfsexport : delete an NFS export on this server
    createnfsexport : create a new NFS export on this server

  • ovm_vmcontrol : generic Virtual Machine level functions
  • start : start the Virtual Machine
    stop : Power-off the Virtual Machine
    suspend : Suspend a running Virtual Machine
    kill : Kill the Virtual Machine
    resume : Resume a suspended Virtual Machine
    restart : Restart a running Virtual Machine (Power-off, start)
    status : display the current status of the Virtual Machine (Running, Stopped,Suspended)
    info : display Virtual Machine info such as memory, utilization, server-pool,...
    lock : temporarily lock the server disallowing any updates to happen
    events : list all latest events for this Virtual Machine requiring acknowledgement
    list : list all Virtual Machines registered with this Oracle VM Manager instance
    vcpuset : bind virtual cpu’s to physical threads for this Virtual Machine
    vcpuget : list bindings of virtual cpus to physical threads for this Virtual Machine
    settags : store a set of comma separate custom tags for a Virtual Machine
    gettags : retrieve list of tags for this Virtual Machine
    fixcfg : fix Virtual Machine Configuration file
    delete : delete Virtual Machine

  • ovm_vmdisks : a utility to list the virtual disk info for a given Virtual Machine
  • ovm_vmmessage : utility to send/receive message to/from a Virtual Machine using the Oracle VM VM API
  • Here a a few (hopefully) useful examples:

    # ovm_managercontrol -u admin -p Manager -h localhost -c getsessiontimeout
    # ovm_managercontrol -u admin -p Manager -h localhost -c setsessiontimeout -T 60

    Session timeout is the Oracle VM Console session timeout (connect to a Virtual Machine console). When you have a slow network, the default timeout of 30 seconds might be too short and this lets you set this to a longer (or shorter value).

    # ovm_servercontrol -u admin -p Manager -h localhost -c listnfsexports -s server1
    Oracle VM Server Control utility 0.5.1.
    Connected.
    Command : listnfsexports
    NFS Exports :
            id         : 0004fb0000230000978a28e2dc85e06b
            client     : ca-srs1
            options    :
            repository : SATA_ST3500320AS_5QM1EYTX
            path       : /OVS/Repositories/0004fb000003000031fac25b24740cca/
    ---
            id         : 0004fb00002300004b161fd1a0d106b7
            client     : ca-vdi1
            options    : rw,no_root_squash
            repository : SATA_ST3500320AS_5QM1EYTX
            path       : /OVS/Repositories/0004fb000003000031fac25b24740cca/
    ---
    Exit... 

    nfs exports are very useful and new in 3.1. It is now possible to export a local ocfs2 repository on a FC, iscsi or local disk device to be exported from an Oracle VM server to the outside world. You define which repository and which client can mount the repository. listnfsexports lists the exports defined on a specific Oracle VM Server.

    # ovm_servercontrol -u admin -p Manager -h localhost -c createnfsexport -s server1 
    -C ca-vdi1 -o rw,no_root_squash -r SATA_ST3500320AS_5QM1EYTX
    Oracle VM Server Control utility 0.5.1.
    Connected.
    Command : createnfsexport
    Creating nfs export for repository 'SATA_ST3500320AS_5QM1EYTX' on server.
    Created repository export
    Exit... 

    Here I have a repository named SATA_ST3500320AS_5QM1EYTX (this happens to be a local disk on server1) and I am exporting that repo through nfs to external server ca-vdi1. So now, on ca-vdi1, I can do mount server1:/OVS/Repositories/0004fb000003000031fac25b24740cca /mnt and I can backup(or restore) files.

    # /ovm_servercontrol -u admin -p Manager -h localhost -c deletenfsexport -s server1 
    -i 0004fb0000230000978a28e2dc85e06b
    Oracle VM Server Control utility 0.5.1.
    Connected.
    Command : deletenfsexport
    Deleting nfs export with ID '0004fb0000230000978a28e2dc85e06b'
    Deleted repository export
    Exit... 

    Simply deleting a repository. You need to specify the UUID of the nfs export, which you can find using listnfsexports.

    # ovm_repocontrol -u admin -p Manager -h localhost -c create -r foo -s wopr5 
    -i 3600144f057ef8a0000004fb3194b0001
    Oracle VM Repository Control utility 0.5.1.
    Connected.
    Command : create
    Found LUN with id '3600144f057ef8a0000004fb3194b0001'
    Creating Filesystem...
    Creating repository... Please wait...
    Create repository completed.
    Exit... 

    Create a repository on the LUN with a specific UUID. This works around slow storage where we hit a 2 minute timeout.

    # ovm_vmcontrol -u admin -p Manager -h ovmm -c vcpuget -v apitest
    Oracle VM VM Control utility 0.3.9.
    Connected.
    Command : vcpuget
    Current pinning of virtual CPUs to physical threads : 2,3
    

    Pinning virtual CPU's to physical threads.

    # ovm_vmcontrol -u admin -p Manager -h ovmm -c vcpuset -v apitest -s 5,6
    Oracle VM VM Control utility 0.3.9.
    Connected.
    Command : vcpuset
    Pinning virtual CPUs
    Pinning of virtual CPUs to physical threads '5,6' 'apitest' completed.
    A restart of Virtual Machine 'apitest' is required. 
    

    # vm_servercontrol  -u admin -p Manager -h localhost -s server -c events -A all
    Oracle VM Server Control utility 0.3.10.
    Connected.
    Command : events
    Server Events
    Acknowledging Virtual Machine events
            Virtual Machines on this server :
    VM : el5u4_32_pvm_1
    VM : el6u1_64_hvm_1
    VM : el5u7_64_hvm_1
    VM : el5u7_64_pvm_1
    VM : el6u1_64_pvm_1
    VM : el6u2bd5_32_pvhvm_1
    VM : el6u1_32_pvm_1
    VM : uek-boot-el5-64
    Exit..

    Acknowledging events on a given server, including any guest events.

    Friday May 18, 2012

    Oracle VM 3.1 released

    Today we formally announced the release of Oracle VM Server for x86 3.1. It is available for download from http://edelivery.oracle.com/linux. The software consists of 2 ISO images, the Oracle VM 3.1 Manager iso and Oracle VM 3.1 server iso... You can find the documentation library on the documentation website.

    Quite a few enhancements went into this release :

  • Oracle VM Manager UI enhancements to improve ease-of-use.
  • We continue to reduce the number of clicks to do actions and also have very easy to use and very streamlined configuration wizards and setup wizards. For instance for create VM or Create Server pool.

  • Storage enhancements for backup and recovery purposes.
  • It is now possible to do an nfs export on an Oracle VM server of a local/shared SAN/iscsi repository. When you have a filesystem based (ocfs2) repository with Oracle VM 3, only servers that are part of the Server pool can access the filesystem, with this new enhancement, you can define an nfs client hostname/IP address which will be allowed to do a remote nfs mount of this volume for backup purposes (or restore).

  • Using Unbreakable Enterprise Kernel 2 (2.6.39+) as the dom0 kernel
  • We now use the exact same Linux kernel for Oracle Linux with 2.6.39 (uek2) and for Oracle VM 3.1. This is possible because of all the work we have done together with Citrix/Xensource people over the last few years to get the necessary support for Xen and Linux merged into Linux. As I have mentioned in a previous blog entry, Linux mainline contains everything needed to have Linux be a complete dom0 management kernel for Xen without any additional patches. Unlike some of the FUD out there about "Xen" not being part of Linux, we have always made clear that Xen is a hypervisor. And Linux runs on top as the management/control domain kernel giving a great level of separation/isolation. In enterprise environments this is a big deal and helps us create solid, stable, secure solutions.

    The advantage for customers is that we do an incredible amount of testing on uek2 in terms of performance, network, disk io and scalability/stability in general and now Oracle VM automatically benefits from this additional testing. On top of that, for our partners, they do not have to worry about testing 2 different products in terms of drivers, since we share the exact same code base we have the exact same drivers for our hardware/storage vendors.

  • Clone many times
  • Simply the ability to say, clone a template X times in one step.

    You can find more information in the documentation set and in the release notes.

    Wednesday May 16, 2012

    More on templates and first boot configuration

    A little while back I wrote something up about building appliances (physical/virtual) using Oracle Linux. We build a whole set of Oracle VM templates and release them on our edelivery site. But some of what we use inside these templates can be more generically applied... Let me explain..

    A few years ago we created a Linux rpm called oraclevm-template-config. This is a very small package that contains a number of scripts that esssentially do the following:

  • install a service oraclevm-template-config that checks at boot time whether it needs to run or not
  • provides a default first boot configuration service for root password, network setup
  • provides a default cleanup script so that you can re-run this on an existing system, at next boot
  • provides a small set of library calls to use those functions in your own scripts
  • allows you to define your own first boot configuration scripts for your own applications to run
  • regenerates ssh host keys
  • all input was done on the local console or through kernel boot line parameters
  • These things are very straightforward and we wrote it specifically to build Oracle VM templates but there was nothing specific to Oracle VM or virtualization there. The newer version of this, which we released along with Oracle VM 3.x is very much tied into Oracle VM. In the new version (3.x) we integrate the first boot scripts and the configuration scripts with the Oracle VM VM api where it's now possible to send remote VM messages to configure the VM. So in order to automate deployment and have very integrated and fancy assemblies or Oracle VM based templates, this is the way to go. If, however, you want something very simple, with lcoal console (physical or virtual), where it can be on a physical server where you want to reboot and reconfigure and install, whether this is a VirtualBox VM or an Oracle VM VM or, for that matter any other Virtualization technology based VM. the older version is quite usable. I tend to use that one for the VirtualBox templates that I create all the time.

    We just published this rpm on http://public-yum.oracle.com. The rpm can be directly downloaded for Oracle Linux 5, here. Or when you have a system registered with public-yum or with ULN it is also in the add-ons channel. If you have a yum repo config then just enable the addon channel and you can simple type yum install oraclevm-template-config.

    So what are some of the basics you can do :

    initial install By default when you install template config, you have a service that provides the basic, configure root password, setup local network at what's called firstboot. To enable this to happen at the next reboot (which one can then call firstboot), just simple type /usr/sbin/oraclevm-template --enable. There is a simple default cleanup script which un-configures the network and removes ssh keys : /usr/sbin/oraclevm-template --cleanup. Keep in mind that when you run this script, the network will reset to dhcp and you will lose your network connection if you are in a remote ssh session.

    So once you do this and reboot your server or your VM. The next reboot will automatically start /usr/sbin/oraclevm-template --config and go through the questionnaire.

    The base configuration script is /etc/sysconfig/oraclevm-template. If you enable the service, you will see RUN_TEMPLATE_CONFIG=YES. This means next boot it will run. There is also a log file /var/log/oraclevm-template.log.

    If all you need is a simple way of packaging a VM or configure a physical server that deals with network setup, the above is enough. If you have your own software installed and it can handle IP changes without reconfiguration, then you can create an appliance and once firstboot happens, the configuration is set, network is setup, your application comes up and you're done. However, in some cases you want to run your own scripts and integrate with this template configuration. We also handle that case...

    If you write your own shell script (or really any script) that you install on the system, you can add that script to /etc/sysconfig/oraclevm-template and instead of doing the default firstboot run, it will at firstboot (when you call --enable) execute your script instead. Simply by setting TEMPLATE_CONFIG_SCRIPT=/myscript.sh. Inside that script you can do whatever you want... You can also provide a cleanup script for --cleanup, using TEMPLATE_CLEANUPS_SCRIPT=mycleanupscript.sh.

    Keep in mind that we do not automatically run --cleanup, this is a manual step. I would recommend doing the following :

  • /usr/sbin/oraclevm-template --enable
  • /usr/sbin/oraclevm-template --cleanup
  • history -c
  • shutdown -h now
  • We provide a few default function you can include in your own scripts. These functions are defined in /usr/lib/oraclevm-template/functions just source them in your bash script.

    ovm_configure_network [dhcp [hostname]] | [static] | [ip_adddress netmask gateway dns_server hostname]

    You can call this like ovm_configure_network "dhcp" in your script and it will configure your network that way. If you set it to static then there will be a dialog for entering ip, hostname, gateway, netmask, dns, or if you provide that in your function call it will just set the changes without dialog.

    ovm_cleanup_os
    A simple OS cleanup. This resets the network to dhcp and cleans up ssh keys.

    ovm_disable_firewall
    disable iptables

    press_anykey
    Shows user input to enter a key in your script execution flow

    ovm_error_message Displays an error message to the screen and also to /var/log/oraclevm-template.log

    ovm_info_message Displays an informational message to the screen and also to /var/log/oraclevm-template.log

    ovm_warn_message Displays a warning message to the screen and also to /var/log/oraclevm-template.log

    All of the above is very simple, yet very powerful. Like I mentioned in the beginning, I use this all the time when creating both VirtualBox and Oracle VM server templates and it makes my life so much easier. People often write these on their own but I wanted to share how to do this so, when you use Oracle Linux, and/or Oracle VM Server or Oracle VM VirtualBox or anything else, and you're building a cool appliance, physical or virtual, use it and you don't even need a support subscription to download it:)...

    Friday May 04, 2012

    building an appliance? physical ? virtual? production quality? use Oracle Linux

    Many companies are building appliances these days. Whether they are virtual appliance (VMs) or physical appliances, they all have an embedded OS inside. Now, I have looked at many of them, in particular virtual appliances and a few things struck me as interesting :

  • many had a random version of Linux included. Nothing bad about any of the distributions used at all, but just a hodgepodge of debian, fedora, opensuse, centos, etc... Almost all were marked "trial" or "demonstration"
  • a number of these appliances had a known commercial version of Linux, were marked trial edition and had restrictions on use (use for 30 days and so on).
  • So what are some problems with these scenarios?

    Well, first of all, even though there is absolutely nothing wrong at all with using community-based and community-maintained distributions (they do a great job), in a commercial setting, or when you build a product that lives on top of it, you need to have the ability to get formal support. Maybe not everyone needs this, but many companies do not have the knowledge (nor should they necessarily) to support the Linux distribution itself and they would want a reasonable formal service level for when something goes wrong, or when to expect something to get released.

    Secondly, in the case of using a commercial Linux distribution, there are distribution restrictions and even access restrictions. Certain Linux distribution vendors that offer formal support subscriptions will not give you access to the binaries without a subscription for support services and they also do not let you re-distribute without special agreements or making lots of modifications (or even restrict modifications).

    Enter Oracle Linux...

    What do you get?

  • freely downloadable ISO images for all the releases and major updates (Oracle Linux 4(4.4, 4.5, 4.6,4.7,4.8), Oracle Linux 5(5.1, 5.2, 5.3, 5.4, 5.5, 5.6, 5.7,5.8),Oracle Linux 6 (6.1, 6.2)). Both source code and binaries.No support subscription needed to download
  • freely re-distributable Both source code and binaries (no contract or support subscription needed to re-distribute, including logos and trademarks)
  • freely available bugfix errata and security errata, released at same time as we release them to customers with support subscriptions Both source code and binaries. In other words, every RPM released through a free public yum repository
  • the ability to purchase a support subscription for what you build, ship, support to your customers without any change to your appliance. No need to have a trial version or demo version using a community-edition Linux and a production version based on a commercial-edition Linux
  • use it in production, go for it, whatever the use case is, use it, if you want support, get a support subscription, if you don't need support, that's fine.
  • the ability to have a support subscription that offers on-line zero-downtime, rebootless security and bugfix updates as part of Oracle Linux Premier support with Oracle Ksplice technology
  • if you run with Centos or Red Hat Enterprise Linux, just replace the rpm list with the Oracle Linux RPMs and you're done.. so easy to switch
  • heavily tested on the biggest servers and biggest IO intensive, process intensive and network intensive workloads out there
  • do this for installs on physical servers, for Virtual Machines (whether this be Oracle VM Virtual Appliances, VMWare Virtual Appliances or something else, it doesn't matter)
  • One Enterprise Linux distribution, for all of the above. We make it easy for you. Grab the code, binaries and source, use it, distribute it, build your environments with it, freely, no contracts needed. Need our help, get a support subscription. Choice, open. Virtual, physical, cloud. Not just obfuscated tar balls. No license or activation key, good consistent SLAs for releasing security updates, well tested,... Run Oracle Linux in-house in test and development environments, run it in production environments, use it for customer systems, distribute it, any or all of the above. One distribution that you can manage across all the use cases. No need to manage different versions even if they're similar, no need to make different distribution choices based on your use case and pay/not pay.

    http://www.oracle.com/linux
    http://public-yum.oracle.com
    http://edelivery.oracle.com/linux

    Thursday May 03, 2012

    understanding memory allocation in oracle vm / xen

    As a follow up to my previous blog about cpu topology, I wanted to add a little bit about memory topology and memory allocation in the hypervisor. Most systems these days that are multi-socket are considered NUMA. Even though over the years, the NUMA-factor has gone down drastically,there still is a small amount of memory locality involved.

    My test setup is a dual socket server with 36GB memory. You can see this in Oracle VM Manager as part of the server info or directly on the server with xm info :

    # xm info 
    ..
    total_memory           : 36852
    free_memory            : 25742
    ..
    

    I have a few VMs running on this server which is why you see memory be lower than total. The 16GB VM is running with tmem enabled and because of that is not using up all memory but only the base memory needed to be active for the workload it's running.

    # xm list
    Name                                        ID   Mem VCPUs      State   Time(s)
    0004fb00000600001668dac79108cb84             2  4096     4     -b----    129.9
    0004fb0000060000804bac06a5087809             1  4096     4     -b----    129.4
    0004fb0000060000db9c71d539c940ed             3 16000     4     -b----     28.3
    Domain-0                                     0  1244    24     r-----    188.0
    

    Let's start with a clean slate and look at some statistics. The following commands will dump detailed memory information on your server :

    # xm debug-key u ; xm dmesg. Basically debug info for NUMA memory info. xm dmesg will show you the debug output.

    
    (XEN) 'u' pressed -> dumping numa info (now-0xFE:A1CFFF69)
    (XEN) idx0 -> NODE0 start->0 size->4980736
    (XEN) phys_to_nid(0000000000001000) -> 0 should be 0
    (XEN) idx1 -> NODE1 start->4980736 size->4718592
    (XEN) phys_to_nid(00000004c0001000) -> 1 should be 1
    (XEN) CPU0 -> NODE0
    (XEN) CPU1 -> NODE0
    (XEN) CPU2 -> NODE0
    (XEN) CPU3 -> NODE0
    (XEN) CPU4 -> NODE0
    (XEN) CPU5 -> NODE0
    (XEN) CPU6 -> NODE0
    (XEN) CPU7 -> NODE0
    (XEN) CPU8 -> NODE0
    (XEN) CPU9 -> NODE0
    (XEN) CPU10 -> NODE0
    (XEN) CPU11 -> NODE0
    (XEN) CPU12 -> NODE1
    (XEN) CPU13 -> NODE1
    (XEN) CPU14 -> NODE1
    (XEN) CPU15 -> NODE1
    (XEN) CPU16 -> NODE1
    (XEN) CPU17 -> NODE1
    (XEN) CPU18 -> NODE1
    (XEN) CPU19 -> NODE1
    (XEN) CPU20 -> NODE1
    (XEN) CPU21 -> NODE1
    (XEN) CPU22 -> NODE1
    (XEN) CPU23 -> NODE1
    (XEN) Memory location of each domain:
    (XEN) Domain 0 (total: 318627):
    (XEN)     Node 0: 282976
    (XEN)     Node 1: 35651
    
    The above output shows that the first 12 cpu's are bound to memory node 0 and the next 12 to memory node 1. The info shows how many pages of RAM are available on each node NODE0 start->0 size->4980736 and NODE1 start->4980736 size->4718592. the Dom0 domain is about 1.2Gb of RAM and it has some memory allocated on each NODE (it also has all of it's 24 vcpu's allocated across all threads in the box). Now let's start a VM.

    # xm list
    Name                                        ID   Mem VCPUs      State   Time(s)
    0004fb0000060000804bac06a5087809             4  4096     4     r-----      8.8
    Domain-0                                     0  1244    24     r-----    240.9
    
    # xm debug-key u ; xm dmesg
    ...
    (XEN) Memory location of each domain:
    (XEN) Domain 0 (total: 318627):
    (XEN)     Node 0: 282976
    (XEN)     Node 1: 35651
    (XEN) Domain 4 (total: 1048576):
    (XEN)     Node 0: 1048576
    (XEN)     Node 1: 0
    
    You can see that the newly started VM (domain 4) has 4Gb allocated on node 0.
    # xm vcpu-list 4
    Name                                ID  VCPU   CPU State   Time(s) CPU Affinity
    0004fb0000060000804bac06a5087809     4     0     0   -b-       4.8 0-3
    0004fb0000060000804bac06a5087809     4     1     3   -b-      26.1 0-3
    0004fb0000060000804bac06a5087809     4     2     2   -b-       3.5 0-3
    0004fb0000060000804bac06a5087809     4     3     1   -b-       2.4 0-3
    
    The VM also has its virtual CPUs bound to node 0. Let's start another VM.

    # xm vcpu-list 6
    Name                                ID  VCPU   CPU State   Time(s) CPU Affinity
    0004fb00000600001668dac79108cb84     6     0    19   r--       2.2 19-23
    0004fb00000600001668dac79108cb84     6     1    23   r--      24.6 19-23
    0004fb00000600001668dac79108cb84     6     2    20   -b-       1.4 19-23
    0004fb00000600001668dac79108cb84     6     3    22   -b-       1.1 19-23
    
    # xm debug-key u ; xm dmesg
    ...
    (XEN) Memory location of each domain:
    (XEN) Domain 0 (total: 318627):
    (XEN)     Node 0: 282976
    (XEN)     Node 1: 35651
    (XEN) Domain 4 (total: 1048576):
    (XEN)     Node 0: 1048576
    (XEN)     Node 1: 0
    (XEN) Domain 6 (total: 1048576):
    (XEN)     Node 0: 0
    (XEN)     Node 1: 1048576
    
    As you can see, this domain 6 has vCPUs bound to node 1, and Xen automatically also allocates memory from node 1. To ensure memory locality. It tries hard to keep memory and CPU as local as possible. Of course when you run with many VMs with many vCPUs then memory allocation will be spread out across multiple nodes.

    After starting a 16Gb VM on this server (domain 7), now that 8Gb is allocated, you will see that this 16Gb VM's memory allocation is across the 2 memory nodes :

    (XEN) Memory location of each domain:
    (XEN) Domain 0 (total: 318627):
    (XEN)     Node 0: 282976
    (XEN)     Node 1: 35651
    (XEN) Domain 4 (total: 1048576):
    (XEN)     Node 0: 1048576
    (XEN)     Node 1: 0
    (XEN) Domain 6 (total: 1048576):
    (XEN)     Node 0: 0
    (XEN)     Node 1: 1048576
    (XEN) Domain 7 (total: 4097012):
    (XEN)     Node 0: 2524148
    (XEN)     Node 1: 1572864
    

    Wednesday May 02, 2012

    figuring out cpu topology in oracle vm

    I have had a few people in the past ask about CPU topology on Oracle VM and how to figure out which CPUs, cores or threads are used by a VM... Here's a quick overview :

    1) get a summary of your server

    The xm info command prints out a whole bunch of information about your system.

    Look for the lines below in the output for detail on your systems CPUs.

    # xm info
    ...
    nr_cpus                : 8
    nr_nodes               : 1
    cores_per_socket       : 4
    threads_per_core       : 2
    cpu_mhz                : 3200
    ...
    
    This server has a single socket with 4 cores and 2 threads per core. Total of 8 "CPUs". so cpu0..7 really is thread 0..7.

    # xm info
    ...
    nr_cpus                : 12
    nr_nodes               : 1
    cores_per_socket       : 6
    threads_per_core       : 2
    ...
    
    This server has a single socket with 6 cores and 2 threads per core. Total of 12 "CPUs".

    2) get the cpu topology

    The xenpm command lets you print out the thread/core/socket topology on a given server :

    # xenpm get-cpu-topology
    CPU     core    socket  node
    CPU0     0       0       0
    CPU1     0       0       0
    CPU2     1       0       0
    CPU3     1       0       0
    
    In the above example, it's a single socket machine with 2 cores and 2 threads per core. cpu 0 is thread 0 of core 0, cpu 1 is thread 1 of core 0, cpu 2 is thread 0 of core 1, cpu 3 is thread 1 of core 1. If you were to have cpus="0,1" in the VM config file, you would be running on core 0. If you were to have cpus="0-3" in the VM config file, you would actually run a VM on both cores.

    # xenpm get-cpu-topology
    CPU     core    socket  node
    CPU0     0       0       0
    CPU1     0       0       0
    CPU2     1       0       0
    CPU3     1       0       0
    CPU4     2       0       0
    CPU5     2       0       0
    CPU6     3       0       0
    CPU7     3       0       0
    
    In the above example, you have a single socket server with 4 cores and 2 threads per core. cpu 0 -> thread 0 of core 0, cpu 1 -> thread 1 of core 0 etc.If you were to have cpus="4-7" in the VM config file, you would run the VM on cores 2 and 3.

    # xenpm get-cpu-topology
    CPU     core    socket  node
    CPU0     0       0       0
    CPU1     0       0       0
    CPU2     1       0       0
    CPU3     1       0       0
    CPU4     2       0       0
    CPU5     2       0       0
    CPU6     8       0       0
    CPU7     8       0       0
    CPU8     9       0       0
    CPU9     9       0       0
    CPU10    10      0       0
    CPU11    10      0       0
    
    In the above example, you see a single socket server with 6 cores hyperthreaded.

    3) Show the guest/VM vcpu bindings to physical threads : Use the command xm vcpu-list to list a summary of which virtual CPU is running on which physical CPU.

    # xm vcpu-list 1
    Name                                ID  VCPU   CPU State   Time(s) CPU Affinity
    0004fb00000600007c351fa24276c63f     1     0     5   -b-    4673.6 5-6
    0004fb00000600007c351fa24276c63f     1     1     5   -b-    4534.0 5-6
    
    If you add the domain ID to the command xm vcpu-list 1, you get the information for just that guest or VM. In the above example, you have a guest with 2 virtual CPUs both running, at this time, on physical cpu (thread in my case) 5. The column CPU Affinity show's 5-6 which means that both virtual CPUs could be running on either thread 5 or 6. This shows that the guest is pinned on those 2 threads. Combined with the information of xenpm get-cpu-topology I can then see that in my case CPU 5 is thread 1 of core 2 and CPU 6 is thread 1 of core 8. So this 2 vcpu guest is pinned to 2 separate physical cores.

    # xm vcpu-list
    Name                                ID  VCPU   CPU State   Time(s) CPU Affinity
    0004fb00000600007c351fa24276c63f     1     0     5   -b-    4676.8 5-6
    0004fb00000600007c351fa24276c63f     1     1     5   -b-    4537.0 5-6
    Domain-0                             0     0     0   -b-     932.1 any cpu
    Domain-0                             0     1     6   -b-    1168.0 any cpu
    Domain-0                             0     2     7   -b-    1010.8 any cpu
    Domain-0                             0     3    11   -b-     903.0 any cpu
    Domain-0                             0     4     8   -b-     494.2 any cpu
    Domain-0                             0     5     9   r--     773.8 any cpu
    Domain-0                             0     6     1   -b-     522.7 any cpu
    Domain-0                             0     7     2   -b-     785.1 any cpu
    Domain-0                             0     8     4   -b-     473.8 any cpu
    Domain-0                             0     9     3   -b-     728.1 any cpu
    Domain-0                             0    10    10   -b-     490.8 any cpu
    Domain-0                             0    11     0   r--    1219.6 any cpu
    
    This is the same system but xm vcpu-list without argument. It also shows the dom0 guest. As you can see in this example, dom0 can run on any physical thread and the CPU Affinity is any cpu which implies any virtual CPU can be scheduled on any physical thread so no pinning or partitioning.

    4) using ovm_utils ovm_vmcontrol lets a user or admin set and get the cpu/vcpu bindings for a VM through Oracle VM Manager in 3.0.

    # ./ovm_vmcontrol -u admin -p Manager1 -h localhost -v apitest -c vcpuget
    Oracle VM VM Control utility 0.4.2.
    Connected.
    Command : vcpuget
    Current pinning of virtual CPUs to physical threads : 5,6
    
    In the above example, I have a VM named apitest and using the action vcpuget it shows me that virtual cpus of this guest are bound to threads 5 and 6. This happens to be the VM in the previous example which showed that this was core 2 and core 8 (3).

    Now let's bind the vcpu's to core 0, I just run the following command :

    # ./ovm_vmcontrol -u admin -p Manager1 -h localhost -v apitest -c vcpuset  -s 0
    Oracle VM VM Control utility 0.4.2.
    Connected.
    Command : vcpuset
    Pinning virtual CPUs
    Pinning of virtual CPUs to physical threads  '0' 'apitest' completed.
    
    Now, running the xm vcpu-list command, I can see that this immediately took effect (and also will on subsequent startups of the VM) :
    # xm vcpu-list 1
    Name                                ID  VCPU   CPU State   Time(s) CPU Affinity
    0004fb00000600007c351fa24276c63f     1     0     0   -b-    4687.6 0
    0004fb00000600007c351fa24276c63f     1     1     0   -b-    4547.2 0
    
    The VM now has CPU Affinity 0 for both virtual CPUs.

    Friday Apr 13, 2012

    More Oracle VM templates for PeopleSoft and Oracle Enteprise Manager

    Just as I wrote up a blog promoting the Oracle VM Ebusiness suite templates, we also pushed out 2 other products :

  • Oracle Enterprise Manager Cloud Control 12c
  • Oracle PeopleSoft FSCM 9.1 and PeopleTools 8.52.03
  • They can be downloaded from edelivery.

    Same advantage... you download the template, import it and you have a completely pre-installed set of products. That's application deployment, not just VM deployment. That's flexibility across the stack, not just a hypervisor, not just virtualization, but a complete solution stack.

    Oracle Ebusiness Suite 12.1.3 Oracle VM templates

    Steven Chan just published a great blog entry that talks about the release of a new set of Oracle VM templates.

    Oracle Ebusiness Suite 12.1.3.

    You can find the blog post here.

    Templates are available for:

  • E-Business Suite 12.1.3 Vision (64-bit)
  • E-Business Suite 12.1.3 Production (32-bit)
  • E-Business Suite 12.x Sparse Middle Tiers (32-bit and 64-bit)
  • Thanks Steven!

    Why does this stuff matter? Well, in general, virtualization (or cloud) solutions provide an easy way to create Virtual Machines. Whether it's through a "cloud api" or just a virtualization API. But all you end up with, in the end, is still just a Virtual Machine... Maybe with an OS pre-installed/pre-configured. So you have flexibility of moving VMs around and providing a VM but what about the actual applications (anything more than a very basic app)?

    The application administrator then still has to go and install and configure the OS for that application and install the application and its patches and basic configuration so that the application user then can go in. Building gold images for complex software stacks that are not owned by the users/admins is always very difficult.

    With our templates, we provide a number of things :

  • Oracle Linux pre-installed and pre-configured with the minimum required packages for that application to run. (so it's secure)
  • Oracle Linux can be distributed and used for free or with a support subscription. There is no trial license, there is no registration key, no alpha version or community version versus enterprise version. You get what we provide in our engineered systems, what we provide support for, without change. Supported out of the box. No virtual Trial appliances, no prototypes, no POC. What you download is production ready without change.
  • The applications are installed by the developers of the application. The database team builds database templates, the applications engineering team builds applications templates. The first boot/configuration scripts ask for the basic information such as hostname, ip address, user passwords and then go off and set everything up correctly.
  • All tested together -> application -> operating system -> hypervisor. not 3 (or more) products from 3(or more) different companies.
  • Thursday Apr 05, 2012

    Some Oracle VM 3 updates

    Today we did another patch set update for Oracle VM 3 (3.0.3-build 240). This can be downloaded from My Oracle Support as patch ID 13614645. There are quite a few updates in here and I highly recommend any Oracle VM 3 customer or user to install this update.

    This patch can be installed on top of Oracle VM 3.0 versions 3.0.2 and 3.0.3. The patch is cumulative for 3.0.3. So if you already installed patch update 1 (3.0.3-150) then this will just be incremental on top of that and brings you to 3.0.3-build 227. There is a readme file which contains the patchlist in the patch info.

    The following patches are released on ULN for Oracle VM server 3.0 :

  • initscripts-8.45.30-2.100.18.el5.x86_64 The inittab file and the /etc/init.d scripts.
  • kernel-ovs-2.6.32.21-45.6.x86_64 The Linux kernel
  • kernel-ovs-firmware-2.6.32.21-45.6.x86_64 Firmware files used by the Linux kernel
  • osc-oracle-ocfs2-0.1.0-35.el5.noarch Oracle Storage Connect ocfs2 Plugin
  • osc-plugin-manager-1.2.8-9.el5.3.noarch Oracle Storage Connect Plugin Infrastructure
  • osc-plugin-manager-devel-1.2.8-9.el5.3.noarch Oracle Storage Connect Plugin Development
  • ovs-agent-3.0.3-41.6.x86_64 Agent for Oracle VM
  • xen-4.0.0-81.el5.1.x86_64 Xen is a virtual machine monitor
  • xen-devel-4.0.0-81.el5.1.x86_64 Development libraries for Xen tools
  • xen-tools-4.0.0-81.el5.1.x86_64 Various tooling for the manipulation of Xen instances
  • Errata emails will be sent in the next few days with details on the above updates. Or you will find them here.

    I also did an update of my Oracle VM utilities to 0.4.0. They are also available from My Oracle Support, patch ID 14736239.
    These utils can be unzipped and installed on the server running Oracle VM Manager. Typically in /u01/app/oracle/ovm-manager-3/ovm_utils. There is a set of man pages in /u01/app/oracle/ovm-manager-3/ovm_utils/man/man8. There now are 6 commands :

  • ovm_vmcontrol : VM level operations
  • ovm_servercontrol : server level operations
  • ovm_vmdisks : virtual disk/physical location mapping for VM disks
  • ovm_vmmessage : message passing utility between the manager and the VM tools (in the Oracle VM templates)
  • ovm_repocontrol : repository level operations
  • ovm_poolcontrol : pool level operations

    Some of the new changes :

  • at a pool level, acknowledge events and cascade to servers and virtual machines with outstanding events
  • at a pool level, do a rescan of the storage for fibrechannel/iscsi disks if you add new devices (it does this operation then on every running server)
  • at a repository level, fixup a device if it had a failed create repository
  • at a repository level, refresh the repository and this will update the free space in the UI for ocfs2 repositories
  • at a server level, acknowledge server events and cascade to virtual machines if needed
  • at a VM level, acknowledge VM events
  • at a VM level, bind vcpus to cores with vcpuset/vcpuget

    Please see the man pages and remember that these tools are just written As Is - no SRs... (per the documentation) Hopefully they are useful.

  • Saturday Feb 04, 2012

    Changing database repositories in Oracle VM 3

    At home I have a small atom-based server that was running Oracle VM Manager 3, installed using simple installation. Simple installation is the option where you just enter a password and the Oracle VM Manager installer installs : Oracle XE database, WebLogic Server and the Oracle VM Manager container. The same password is used for the database user, Oracle VM Manager database schema user, weblogic user and admin user for the manager instance.

    The manager instance stores its data as objects inside the database. To do that, there is something called a datasource defined in weblogic during installation. It's basically a jdbc connection from weblogic to the database. This DS requires the following information : database hostname, database instance name, database listener port number, schema username and schema password. In my default install this was localhost, XE, 1521, ovs, mypassword.

    Now that I re-organized my machines a bit, I have a larger server that runs a normal database 11.2.0.3, which I also happen to use for EM12c. So I figured I would take some load off the little atom server, keep it running Oracle VM Manager but shut down XE and move the schema over to my dedicated database host. This is a straightforward process so I just wanted to list the steps.

    1) shut down Oracle VM Manager so that it does not continue updating the repository.
    as root : /etc/init.d/ovmm stop
    
    2) export the schema user using the exp command for Oracle XE
    as oracle : 
    cd /u01/app/oracle/product/11.2.0/xe
    export ORACLE_HOME=`pwd`
    export ORACLE_SID=XE
    export PATH=$ORACLE_HOME/bin:$PATH
    exp
    (enter user ovs and its password)
    export user (option 2)
    export everything including data
    this will create (by default) a file called expdat.dmp
    copy this file over to the other server with the other database
    The schema name is also in /u01/app/oracle/ovm-manager-3/.config (OVSSCHEMA)
    
    3) shutdown oracle-xe as it's no longer needed  
    as root : /etc/init.d/oracle-xe stop
    
    4) import the ovs user into the new database. I like to do it as the user. 
    I just simply pre-create the schema before starting import
    as oracle : 
    sqlplus '/ as sysdba'
    create user ovs identified by MyPassword;
    grant connect,resource to ovs;
    at this point, run the imp utility on the box to import the expdat.dmp
    import asks for username/password, enter ovs and its password
    import yes on all data and tables and content.
    
    At this point you have a good complete repository. 
    Now let's make the Oracle VM Manager weblogic instance point to the new database.
    
    5) on the original system, restart weblogic
    as root :/etc/init.d/ovmm start
    wait a few minutes for the instance to come online
    
    6) use the ovm_admin tool
    as oracle : 
    cd /u01/app/oracle/ovm-manager-3/bin
    ./ovm_admin --modifyds orcl wopr8 1521 ovs mypassword
    My new host name for the 11.2.0.3 database is called wopr, 
    the database instance is orcl and listener is still 1521 with schema ovs
    The admin tool asks for a password, this is the weblogic user password. 
    In a simple install, this would be the same as your admin or ovs account password.
    
    7) restart to have everything take effect.
    as root : 
    /etc/init.d/ovmm stop  ; sleep 5 ;/etc/init.d/ovmm start ;
    
    8) edit the config file and update the new data 
    vi /u01/app/oracle/ovm-manager-3/.config 
    modify :
    DBHOST=
    SID=
    LSNR=
    OVSSCHEMA=
    and leave the rest as is. 
    
    that should do it !
    
    

    Sunday Oct 02, 2011

    Oracle VM 3.0.2 patch update

    Last Friday (9/30) we uploaded the first patch for Oracle VM 3.0 to My Oracle Support. You can download the upgrade ISO from the website. Just look for patch 13036236. When you download the patch file p13036236_30_Linux-x86-64.zip(about 50MB), just unzip the file and mount the included ISO image somewhere, or burn a CD and as user root start the runUpgrader.sh script.

    The upgrade utility will be able to upgrade from an existing Oracle VM 3.0.1 installation.
    As always with an upgrade, it's recommended to do a full database repository backup. Or just run the exp utility against your Oracle VM Manager repository database and export the schema that's used to store the repository (defaults to ovs). Just in case.

    Oracle VM Manager 3.0.1 has to be up and running for this process to start, the upgrade process will undeploy the application containers and replace them with the newer version. Most of the information to upgrade will be automatically detected. In version 3.0.1 we save the installation configuration to a config file in /u01/app/oracle/ovm-manager-3/.config. The only thing you will need to provide is the password for the database repository schema.

    We do export the metadata into xml files and the upgrade script backs up these files as well, it would be possible to import this backup into the database again, however I would still recommend and exp of the database, it will be more convenient.(/tmp/ovm-manager-3-backup-(date).zip)
    After the xml files are created, the upgrade tool will transform the data into the new format, delete the old data in the database and import the new version.

    To upgrade the Oracle VM servers, you can register with the ovm3_3.0.2_x86_64_base channel on the Unbreakable Linux Network(ULN) and yum upgrade the servers.

    The detailed documentation can be found here.

    If you do not already have Oracle VM 3.0.1 installed, a new installer ISO image for Oracle VM 3.0.2 will also be made available in the next several days. If you can't wait, download 3.0.1 + the 3.0.2 patch and you're good to go as well.
    About

    Wim Coekaerts is the Senior Vice President of Linux and Virtualization Engineering for Oracle. He is responsible for Oracle's complete desktop to data center virtualization product line and the Oracle Linux support program.

    You can follow him on Twitter at @wimcoekaerts

    Search

    Categories
    Archives
    « April 2014
    SunMonTueWedThuFriSat
      
    1
    2
    3
    4
    5
    6
    7
    9
    10
    11
    12
    13
    14
    15
    16
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
       
           
    Today