Thursday May 24, 2012

The case for Oracle VM

As I mentioned a few days ago, at a customer visit earlier this month we were discussing Oracle VM and Oracle Linux advantages and the customer was surprised to hear how many features we include in our offerings. My previous write up was about Oracle Linux, this one is about Oracle VM.

In many ways, the Oracle VM solution has similar advantages. For us to deliver a solution and for our customers and partners to have a stack that's built to work together across products. Because Oracle VM subscriptions include the right to use a number of products, such as Oracle Enterprise Manager 12c and Oracle Enterprise Manager OpsCenter, we end up with an impressive, all-in set of features. Often times however this is not obvious to our users or evaluators and it is important to look at the total package and not just sub components. What I am going to describe is included in our Oracle VM subscription.

First of all, pricing... our virtualization and infrastructure management solution has a very simple model :

  • Oracle VM premier limited (for 1 or 2 socket servers) : $599 annual support subscription per server
  • It doesn't matter how many VMs you run, it doesn't matter how much RAM is in the box, it doesn't matter how many cores or threads the sockets have. 1 or 2 sockets in the box. That's the price. Annual support subscription fee. No license. Including all the products I will be detailing below.
    So if you have a 2 socket server with 256GB RAM... $599. case closed. 24/7 support.

  • Oracle VM premier (for larger than 2 socket servers) : $1199 annual support subscription per server
  • Same deal. If you have a 4 socket server, or even an 8 socket server with 4 TB of RAM... $1199 subscription. period. No virtual RAM, no per 2 sockets, no multiple options, no starter pack for 2 servers... Simple pricing. 24/7 support.

    Also, as with Oracle Linux, if you purchase an Oracle Server with the Premier support for Systems, it also includes everything I am talking about without the need for extra per server subscriptions.

    Oracle VM support subscriptions for both premier and premier limited comes with the following products and right to use :

  • Oracle VM server
  • Oracle VM server is the product that is installed on each server. It contains the hypervisor and the management console/agent to talk to Oracle VM Manager. This is effectively what you get a support subscription for. A tiny installation of about 200Mb worth of RPMs on a local hard drive or flash drive.

  • Oracle VM Manager
  • Oracle VM Manager is our centralized management console. It is an Oracle WebLogic Server 11g application that uses an Oracle Database as data repository. You typically only need 1 instance to manage a very large set of servers and virtual machines. Oracle VM Manager is the management hub for Oracle VM. It has a web-based admin interface and the management product can be installed on a Linux server, typically outside the Oracle VM server pools. We do not charge extra for this, the use and support of Oracle VM Manager is included in the server subscriptions. More details further down...

  • Oracle Enterprise Manager 12c Cloud Control
  • OEM 12c is our complete management product which can manage from application to disk. It contains management for our various applications, database, middleware, operating systems, virtualization layer, it offers configuration management, lifecycle management, physical and virtual environments, cloud portal and so on. The product consists of many management packs, the components that manage Oracle Linux and Oracle VM (cloud portal, private cloud, self-service portal, VM management, IaaS layer services, cloud API....) are included with Oracle VM subscriptions. These are not extra licensed components, they're part of the package!

  • Oracle Enterprise Manager Ops Center 12c
  • EMOC is a very powerful management product that's focused on infrastructure management, our Oracle systems (X86 and SPARC) (firmware, switches, hardware fault analysis, My Oracle Support integration, bios configuration etc...), our Operating Systems (Oracle Solaris, Oracle Linux), Virtualization products (Solaris Containers, Oracle VM SPARC, Oracle VM x86) and a connector into OEM12c to tie them together. Again, when you have an Oracle VM server subscription, you have the right to use EMOC as part of the stack included with its feature set It also contains an IaaS layer and a cloud API and cli.

    Both EM12c and EMOC connect to Oracle VM Manager instances behind the scenes. They can discover or sync up the information from an existing Oracle VM Manager/Server configuration and you can literally switch back and forth. So you start out installing a bunch of Oracle VM servers, install an Oracle VM manager instance, discover the servers, then install EM12c and/or EMOC, discover Oracle VM Manager and now you have access to their feature sets on top. You can still connect to / log into Oracle VM Manager and so on.

    So when we say "Oracle VM" and when you compare "Oracle VM" without various competitors, we really provide you with all of the above, and our feature checklist and offering is all of the above. Often times people just use Oracle VM Manager/Server without also including the use of EMOC and/or EM12c. Our competitors have many products that are all separately licensed pieces and bits that have to be installed together. Look at the solution bundle. We have no per VM, per 96GB, per socket, per feature pricing... remember, $599 and $1199 subscription.

    Here's a few features that are part of all this :

  • Cloud Infrastructure administrator portal
  • Set up and manage the lowerlevel infrastructure, hypervisor, provisioning of storage...

  • Self service administrator
  • Set up higher-level services, users, roles, access boundaries (assign roles and users to zones)
    Set up charge back plans, publish software components (VMs, full application stacks for self service users)

  • self service user portal
  • end user, developer can log in and see their assigned infrastructure quota and create application setups based on software component from the libraries

  • Consolidation planning physical server assets and P2P or P2V advisories
  • Since EM12c has a view of physical and virtual servers and applications inside, it can provide very detailed planning of which systems are good targets to consolidate, because it has an application view and a complete end to end stack overview, it is much more detailed and capable than simple VM analysis.

  • Zone management (multiple pools of servers into a larger grouping)
  • Depending on administrators needs its possible to configure larger zones (pools) of systems as units

  • Storage management through Oracle VM Storage Connect
  • We can manage iscsi, fibre channel, NFS servers from our management console. It is possible to just provision large chunks of storage and provide credentials to Oracle VM Manager and we can handle the details behind the scenes. Add a new lun? no problem, we will do it. EM12c and EMOC expose this functionality and connect through Oracle VM Manager to the storage.

  • Oracle VM assemblies and templates
  • pre-created virtual machines that include complete Oracle product installations, database, siebel, peoplesoft, ebusiness suite etc... these can eb downloaded from the Oracle website and uploaded into Oracle VM Manager or the EM12c software library for easy deployment of complete solution stacks.

    With assemblies we provide complete multi-tiered environments, including scale-out options. Need more webservers, need more database nodes? We can add/remove them on the fly, based on the application and business need, not just based on what the blackbox VM is doing.

  • Chargeback
  • It is possible to create charge back plans, either your own or using existing models. Whether it's basic charge back for virtualization or depending on management packs and oracle products up the stack for other products as well.

  • build, test, deploy
  • Because of the templates and configuration management for applications support, we can let developers build and create solutions which then get uploaded into the software library for use in product deployments. Since EM12c understands the entire product stack, application to disk, we can automate and reproduce not just simple VM deployments but include application configuration on top as well. Creating a VM with an OS is easy. Deploying a VM with an OS is easy. Deploying and configuring an entire software stack with various applications and database(s) is error prone and can take a lot of time. We can automate it for you, reproduce it, exactly the same steps, every time. Top to bottom.

  • Various abstraction layers
  • Through EM12c, EMOC and Oracle VM Manager you have an easy to use, graphical user interface.
    Through EM12c and EMOC you have access to a complete cloud API.
    EMOC offers an IaaS RESTful interface, a JAVA API library and a complete CLI.
    EM12c offers an IaaS RESTful interface

  • complete monitoring and management
  • EM12c manages and monitors every target in the stack, the applications, the operating systems, the VMs, the hypervisors. You can define jobs, metrics, thresholds, corrective actions. Because we collect information across the entire stack, you can set up auditing rules, you can compete systems top to bottom, you can do change management across the entire infrastructure, physical and virtual. To make it easy, it is possible to create groups of systems to manage as a unit. Configuration management is now more critical than ever. In a cloud infrastructure the number of VMs, and as such, the number of deployed applications, grows exponentially. We keep track of every aspect through a single source/database :) of truth.

  • Roles, LDAP directory services
  • Complete integration of LDAP with EM12c and EMOC. The ability to define fine grained access to all these components. Storage, servers, functions.
  • integration with My Oracle Support
  • Patch advisories, service request management, downloading new templates into the software library

  • standard virtualization features
  • Not to forget the fact that this obviously also includes Live migration, Dynamic Power Management (DPM), Dynamic Resource Management(DRS), storage management, network management(setting up bonding, creating virtual networks, vlan support, built-in HA (VM failover, server failover), cpu pinning, clustered filesystem for storage repositories, direct lun/storage access,...

    Installs on the largest X86 servers available. 4TB RAM, 160 threads, virtual machines up to 2TB of ram and up to 128 virtual CPUs or as many virtual CPUs the guest operating system can handle. Support for Linux, Oracle Solaris, Windows (including Windows logo signed drivers).

    So this is the Oracle VM offering, this is a summary (and I am not even trying to be complete here) of our features. Need an API, need a cli, need a UI, need a cloud portal, need monitoring or management, need roles and responsibilities, need centralized account management, we have it all. So when you compare Oracle VM with our competition out there... compare the included feature set in the subscription.

    A few useful links :

  • EMOC API/CLI reference
  • EM12c API
  • Oracle Virtualization page
  • more Oracle Linux options

    A few days ago I wrote a summary of what you get with Oracle Linux. Because there are so many things, I forgot a few... and this is just a continuation of that previous entry.

    There are 2 more features that I wanted to present :

  • Oracle Database Smart Flash Cache
  • DBSFC is a very cool feature that's available for both Oracle Solaris and Oracle Linux customers with the 11gR2 database. In summary, it allows you to basically extend the Oracle Buffer Cache in memory (SGA) using secondary flash based storage. This flash based storage can be presented to the database through a file on a filesystem on flash storage, a raw disk device (flash-based) or through adding flash storage to Oracle ASM and creating a region inside ASM.

    For the most part this feature is going to help with read-only/read-mostly workloads because DBSFC is a read-only cache extension. It contains clean blocks that got kicked out of the buffercache/sga and now first get placed in this extended cache. A subsequent read can then be from this fast storage instead of from the originating datafiles. When a block gets modified, it's modified in the standard database buffer cache, written to disk and copied over into the flash cache.

    The white paper referenced above provides the details on how to use it and how to configure it in an Oracle Linux environment. You simply specify DB_FLASH_CACHE_FILE and DB_FLASH_CACHE_SIZE and that's it. Any Oracle Database customer using Oracle Linux can make use of this.

  • Assigning a control cgroup (cgroup) in Oracle Linux to an Oracle database instance.
  • Oracle Linux has resource management through a feature called cgroups. cgroups lets you create resource groupings based on cpu, memory or disk parameters (or a combination). cgroups is also the internal set of features that Linux containers (lxc) uses. Basically you put processes (pids/tasks) into a cgroup and then they live within the limits of that cgroup definition. With lxc you basically also get process isolation on top.

    It works like this : as an admin you set up a cgroup, give it a name and set up paremeters around which cpu's to use, how much memory to allocate and so forth. The name is really just a simple mkdir in the cgroups virtual filesystem. You then use a new database init parameter process_group_name and when you start the database it will put its processes and as such its resources into that group. alter system set processor_group_name = 'cgroup_name' scope=spfile;. You can do this for each instance and this way you can isolate cpu and memory resources for each instance on a given OS environment. If you use Oracle Linux 5 with uek/uek2 or Oracle Linux 6 with uek/uek2 you can make use of this.

    cgroups with Oracle Linux 5 and UEK/UEK2 has to happen manually, through mkdir and echo > controlfiles. In Oracle Linux 6 we have cgroup management utilities.

    two more reasons :)...

    Sunday May 20, 2012

    Oracle Linux components

    A few weeks ago I was visiting a customer where we were discussing Oracle Linux and Oracle VM advantages compared to some of our competitors. One of the comments from the customer at some point in the discussion was "And you get -all that- for just -$that-?"... Why, yes, you do, sir. I wanted to take the opportunity to list some of the included advantages of having Oracle Linux support. Next time I will do the same for Oracle VM.

    Oracle Linux is a complete enterprise Linux solution available from Oracle. In fact, it is, we firmly believe, the best enterprise Linux solution available. Both for generic use of Linux and in particular and especially when using any Oracle product on top of it. So it is important to remember and understand that Oracle Linux is not just for running Oracle products on top of Linux. Oracle Linux is a complete generic Linux distribution that's fully supported underneath any Linux application. There are no restrictions from our end as to what you can run on it (as long as they're apps built to run on Linux, of course and any application that runs on Red Hat Enterprise Linux will run the same on the corresponding Oracle Linux version). You can run 3rd party Linux applications just fine. Use it as an NFS server, as a webserver or whatever you want to use the system for. Many customers I talk to somehow believe that it's unique for Oracle products and I have to point out that while we have great, unique advantages and extra benefits running Oracle products on Oracle Linux, it's certainly not in anyway restricted for that. It's the opposite, all you needs is Oracle Linux for any and every Linux server in house.

    Why the best, you ask? Well, we do an incredibly amount of testing on a daily basis (1000's of servers run various types of QA on Oracle Linux amounting to over 100,000 hours of QA happening every single day), we have a huge amount of internal use as consumers (end-user/customer use), we have many years of experience helping enterprise customers in general and we extend that down to Operating System support. We can scale using the greater Oracle infrastructure, etc.

    The support team is a large group of dedicated professionals globally that are trained to handle serious issues, deal with mission critical environments and cases, know their product, have a focus on server use first and foremost, understand, in particular, also the other products around it enough (like, to be able to understand that ASM doesn't stand for -assembler language- (in our customer-base) but stands for automatic storage management (you laugh? I can assure you it's not a joke)) A single phone call: if a database problem is an OS issue, no worries, we deal with it internally, no need to go call someone else that might think a database is a dirty word, or someone that 's an expert in gnome (no offense) but not quite in a complex environment where you have a product use 50GB of RAM and has 2000 processes running and is doing massive amounts of IO and woops something is going wrong. "Reboot?" doesn't cut it.

    When we do Linux development, the focus is basically on two things. 1) find ways to enhance Linux to run our products better (in ways that are generic, and also help other applications). A database does disk/network IO, lots of processes, lots of memory access, if we make our product run better, others also benefit... 2) help make Linux a better OS. improve stability, help add features that an OS needs, unrelated to our own product stack but important for customers to run the operating system, no matter what application. Having access to in-house hardware such as very big x64 servers like the x4800 (4TB ram, 160 threads) and TB and TB of storage allows us to test things at very, very large scale. This allows us to publish benchmark results like the 5M tpc-c result a few months ago. Offering the Unbreakable Enterprise Kernel(UEK) as part of Oracle Linux alongside what we call the Red Hat-Compatible kernel gives us the ability to provide current, modern, tested code to customers without reducing reliable and availability.

    A CTO of another company a little while back implied on twitter that UEK is too current and reduces reliability because the code is new. Can I point something out here? One thing we decided to do, with UEK, was to not backport features but instead just focus on stability. The goal is to stay close (but not too close as we don't ship a development tree) to the "Linus kernel" and focus on testing the heck out of it and fixing any bug that we would find, any regression.. This is good for the Linux community because when we do find a problem, we fix it and it's immediately also relevant to the mainline kernel. This helps us and the kernel community. We spend time testing and fixing, we develop on mainline and we don't waste time backporting new stuff into old versions. It's a symbiotic relationship. The alternative : take a version of Linux that's 3+ years old and take code from a more or less current version, after 100's of 1000's of lines of code changes and architectural changes have happened, and make that new code, somehow work in the old tree. The result : old patched up, totally unique code. Someone spending all their time retrofitting complex features into an old version. Testing is unique, bugs found are mostly only relevant in that version and not so much in newer versions unless they're greater design bugs or one liners. No one else really ever tested it because, well, it's unique, no other company is providing that seriously forked kernel that has 1+M of lines of differences. Somehow, I prefer our approach. At least the folks at SuSE seem to believe the same.

    One advantage we have with our kernel is that we made it possible to install in an existing Oracle Linux 5 environment and you are not required to move to Oracle Linux 6 to get the benefit of new kernel features and new drivers and new performance benefits. Moving from one major version to the next is a very costly effort because you basically always end up doing a re-imaging of the servers. Most customers are driven to new major releases for a hardware refresh. We don't require you to do that, you can run both OL5 and OL6, your choice, your timeframe. You can reboot between 2.6.18 and 2.6.32 or even 2.6.39 (3.0.24) without problem, at any time.

    The complete source code for Oracle Linux is available from oss.oracle.com (OL5 or OL6). The source code for our UEK and UEK2 kernels is available in a public GIT repository (UEK 2.6.32 or UEK2 2.6.39. Don't worry, it's not just a tar file and you don't have to go browse webpages behind a userid/password to figure out what all changed and try to piece it together yourself. It's all there, both the mainline kernel changelog and our changes on top. Transparent.

    The binaries for Oracle Linux are available from edelivery in the form of ISO images of the installable DVDs for the major releases and all the update releases (OL5.1...5.8, OL6, OL6.1, OL6.2 etc). You are not required to first get a support subscription, you can just go and download the software, no activation keys, no trial period, production use OK, distribute to your friends? no problem, no need to remove logos. Errata/bug updates are available through ULN (ULN is our Oracle Linux customer portal where we have extra channels with extra software bits for subscribers, or from public yum (public yum is an open, free, no registration, no subscription site). All Oracle Linux errata/packages are published on both sites at the same time. If you have a set of servers where support is not needed, just a standard Oracle Linux, and then a set of servers where you do need support because they're production systems etc, you can install OL on both and you subscribe for the systems you need to have support for. One distribution installed across the entire company. No need to find a free distribution and a pay to use distribution. If you no longer need support services for a system, don't worry, we won't ask you to de-install.

    But wait, there's more..

    How about managing Oracle Linux... well, included in Oracle Linux support subscriptions, is the use of Oracle Enterprise Manager 12c and Oracle Enterprise Manager OpsCenter 12. This is management software that can handle 1000's of servers from a single managed web-based interface. It offers features like :

  • bare-metal provisioning of Oracle Linux servers
  • configuration management, collecting and analysis and auditing
  • lights-out or ad hoc patching
  • availability and performance monitoring
  • event management and job systems
  • yum repositories
  • grouping of servers, custom jobs, custom action/corrective action scripts
  • this is all included, for no additional fee. Any server with Oracle Linux support subscriptions include the above. No extra pack, no per server additional fees. Or if you don't really care for that and just want a very simple yum based setup, you can do that as well, just grab the rpms, put them on a server, create a repository and have your servers connect to that. So, very basic and simple, to very complete and comprehensive. Same cost.

    And there is even more...

    Oracle Real Application Clusters makes use of the Oracle Clusterware(CRS). CRS is cluster software that is capable of protecting any kind of application in a failover cluster. Not just Oracle. It is the basis for any clustered environment and provides generic availability eliminating the need for other third party cluster solutions. You can protect your own application using this enterprise grade clustering stack. Without additional fees. It's included in Oracle Linux support subscriptions (basic/premier).. have your own nfs server? webserver? want to monitor it and restart it in a cluster no problem? Go for it. Using the exact same software we use for the most mission critical clustered systems. Want to run Oracle Linux virtualized? 1 guest? 10 guests? 20 guests? You only need one subscription per physical server and we don't charge more if you run 1 or more than 1 guest. And how much for all this?

  • Oracle x86 based systems with Premier support for Systems includes all this, both the Oracle Linux subscription and the use of the management features and Oracle CRS
  • Non-Oracle x86 based systems : 2 socket server $499 for all of the above or 4+ socket server $1199 includes all this, both the Oracle Linux subscription and the use of the management features and Oracle CRS
  • This is 24/7 support, not just office hours. Complete 24/7 support and real indemnification not based on subscription fees paid.

    We also have a premier support subscription which adds the ability to use Oracle ksplice technology to apply your kernel security fixes online, while the system is running at full speed, while the applications are running, while the users are working, without downtime. And I mean apply, not just install. The installed fixes are instantly effective at runtime. zero reboots. zero downtime scheduling. zero administrative program/planning management.. Or premier backport support, critical fixes for a specific bug in the specific version of the rpm you run in a mission critical environment. Or lifetime support, now with virtualization more useful than ever before. Virtual hardware can remain the same for much longer than the support lifecycles.

  • Oracle x86 based systems with Premier support for Systems includes all this, both the Oracle Linux subscription and the use of the management features and Oracle ksplice and premier backports and lifetime support
  • Non-Oracle x86 based systems : 2 socket server $1399 for all of the above or 4+ socket server $2299 includes all this, both the Oracle Linux subscription and the use of the management features and Oracle CRS and Oracle ksplice and premier backports and lifetime support
  • See our store for pricing, and our Oracle Linux site for policies and references to the above.

    Ok there you have it. Almost a novel but we just have that much to offer, and wait 'til I get to Oracle VM.. it just gets even better.

    Saturday May 19, 2012

    using ovm_utils

    Now that we released Oracle VM 3.1 and I just added ovm_utils 0.5.2 on ARU, it's time to give a few examples/use cases of the utilities :)

    The utilities are available for download from http://support.oracle.com search for patch name 13602094.

    It's a small 2.5Mb zip file which you should install on the manager node and unzip in /u01/app/oracle/ovm-manager-3. There are manpages for the utilities as well. These are installed in /u01/app/oracle/ovm-manager-3/ovm_utils/man/man8. To make it easy, simply add this to your MANPATH : export MANPATH=/u01/app/oracle/ovm-manager-3/ovm_utils/man.

    To use the utilities you need the admin username and password for the manager instance and in general you can just use localhost as the hostname. If you want to be able to connect to a remote manager instance, you have to first enable SSL on the manager node. By default we listen only on 127.0.0.1. There is an option in the utils to make a secure connection, use -X, but again, you first have to enable SSL for this. You can enable secure connections using ovm_utils and the addkeystore command. We have a support note for this.

    So the format is something like : ./ovm_managercontrol -u admin -p mypassword -h localhost -c [options...]..
    There are quite a few commands in the various utilities at this point. I broke the utilities up into categories :br

  • ovm_managercontrol : generic manager level functions
  • yuminfo : display yum configuration
    setupyum : configure yum
    addkeystore : configure the SSL keystore
    keystoreinfo : display keystore file and path
    createcpugroup : create a CPU Compatibility group
    removecpugroup : delete CPU Compatibility group
    addservertocpugroup : add server to CPU Compatibility group
    removeserverfromcpugroup : remove server from CPU Compatibility group
    listcpugroups : list all CPU Compatibility groups and its servers
    setsessiontimeout : set VM Console session timeout (Oracle VM 3.1+)
    getsessiontimeout : display VM Console session timeout(Oracle VM 3.1+)

  • ovm_repocontrol : generic repository level functions
  • list : list all repositories
    status : query repository status
    info : display repository information
    refresh : send a repository refresh command
    fixrepo : fix a failed repository create
    create : create repository on a storage device

  • ovm_poolcontrol : generic server pool level functions
  • status : display the current status of the server pool
    info : display server pool info such as memory, utilization, servers,...
    list : list all server pools registered with this Oracle VM Manager instance
    events : list all latest event types for this server pool
    addserver: add server to this server pool (-S server name)
    removeserver: remove server to this server pool (-S server name)
    refresh : refresh storage in the server pool

  • ovm_servercontrol : generic server level functions
  • start : use IPMI to power-on the server
    stop : shutdown the server
    kill : use IPMI to power-off the server
    mainton : turn on maintenance mode for this server
    maintoff : turn off maintenance mode for this server
    status : display the current status of the server (Running,Stopped,...)
    info : display server info such as memory, utilization, server-pool,...
    lock : temporarily lock the server disallowing any updates to happen
    upgrade : send an upgrade command to the server
    list : list all servers registered with this Oracle VM Manager instance
    events : list all latest event types for this server
    discover : discover server with provided hostname or IP address and Oracle VM agent password (-P)
    listnfsexports : list NFS exports on this server
    deletenfsexport : delete an NFS export on this server
    createnfsexport : create a new NFS export on this server

  • ovm_vmcontrol : generic Virtual Machine level functions
  • start : start the Virtual Machine
    stop : Power-off the Virtual Machine
    suspend : Suspend a running Virtual Machine
    kill : Kill the Virtual Machine
    resume : Resume a suspended Virtual Machine
    restart : Restart a running Virtual Machine (Power-off, start)
    status : display the current status of the Virtual Machine (Running, Stopped,Suspended)
    info : display Virtual Machine info such as memory, utilization, server-pool,...
    lock : temporarily lock the server disallowing any updates to happen
    events : list all latest events for this Virtual Machine requiring acknowledgement
    list : list all Virtual Machines registered with this Oracle VM Manager instance
    vcpuset : bind virtual cpu’s to physical threads for this Virtual Machine
    vcpuget : list bindings of virtual cpus to physical threads for this Virtual Machine
    settags : store a set of comma separate custom tags for a Virtual Machine
    gettags : retrieve list of tags for this Virtual Machine
    fixcfg : fix Virtual Machine Configuration file
    delete : delete Virtual Machine

  • ovm_vmdisks : a utility to list the virtual disk info for a given Virtual Machine
  • ovm_vmmessage : utility to send/receive message to/from a Virtual Machine using the Oracle VM VM API
  • Here a a few (hopefully) useful examples:

    # ovm_managercontrol -u admin -p Manager -h localhost -c getsessiontimeout
    # ovm_managercontrol -u admin -p Manager -h localhost -c setsessiontimeout -T 60

    Session timeout is the Oracle VM Console session timeout (connect to a Virtual Machine console). When you have a slow network, the default timeout of 30 seconds might be too short and this lets you set this to a longer (or shorter value).

    # ovm_servercontrol -u admin -p Manager -h localhost -c listnfsexports -s server1
    Oracle VM Server Control utility 0.5.1.
    Connected.
    Command : listnfsexports
    NFS Exports :
            id         : 0004fb0000230000978a28e2dc85e06b
            client     : ca-srs1
            options    :
            repository : SATA_ST3500320AS_5QM1EYTX
            path       : /OVS/Repositories/0004fb000003000031fac25b24740cca/
    ---
            id         : 0004fb00002300004b161fd1a0d106b7
            client     : ca-vdi1
            options    : rw,no_root_squash
            repository : SATA_ST3500320AS_5QM1EYTX
            path       : /OVS/Repositories/0004fb000003000031fac25b24740cca/
    ---
    Exit... 

    nfs exports are very useful and new in 3.1. It is now possible to export a local ocfs2 repository on a FC, iscsi or local disk device to be exported from an Oracle VM server to the outside world. You define which repository and which client can mount the repository. listnfsexports lists the exports defined on a specific Oracle VM Server.

    # ovm_servercontrol -u admin -p Manager -h localhost -c createnfsexport -s server1 
    -C ca-vdi1 -o rw,no_root_squash -r SATA_ST3500320AS_5QM1EYTX
    Oracle VM Server Control utility 0.5.1.
    Connected.
    Command : createnfsexport
    Creating nfs export for repository 'SATA_ST3500320AS_5QM1EYTX' on server.
    Created repository export
    Exit... 

    Here I have a repository named SATA_ST3500320AS_5QM1EYTX (this happens to be a local disk on server1) and I am exporting that repo through nfs to external server ca-vdi1. So now, on ca-vdi1, I can do mount server1:/OVS/Repositories/0004fb000003000031fac25b24740cca /mnt and I can backup(or restore) files.

    # /ovm_servercontrol -u admin -p Manager -h localhost -c deletenfsexport -s server1 
    -i 0004fb0000230000978a28e2dc85e06b
    Oracle VM Server Control utility 0.5.1.
    Connected.
    Command : deletenfsexport
    Deleting nfs export with ID '0004fb0000230000978a28e2dc85e06b'
    Deleted repository export
    Exit... 

    Simply deleting a repository. You need to specify the UUID of the nfs export, which you can find using listnfsexports.

    # ovm_repocontrol -u admin -p Manager -h localhost -c create -r foo -s wopr5 
    -i 3600144f057ef8a0000004fb3194b0001
    Oracle VM Repository Control utility 0.5.1.
    Connected.
    Command : create
    Found LUN with id '3600144f057ef8a0000004fb3194b0001'
    Creating Filesystem...
    Creating repository... Please wait...
    Create repository completed.
    Exit... 

    Create a repository on the LUN with a specific UUID. This works around slow storage where we hit a 2 minute timeout.

    # ovm_vmcontrol -u admin -p Manager -h ovmm -c vcpuget -v apitest
    Oracle VM VM Control utility 0.3.9.
    Connected.
    Command : vcpuget
    Current pinning of virtual CPUs to physical threads : 2,3
    

    Pinning virtual CPU's to physical threads.

    # ovm_vmcontrol -u admin -p Manager -h ovmm -c vcpuset -v apitest -s 5,6
    Oracle VM VM Control utility 0.3.9.
    Connected.
    Command : vcpuset
    Pinning virtual CPUs
    Pinning of virtual CPUs to physical threads '5,6' 'apitest' completed.
    A restart of Virtual Machine 'apitest' is required. 
    

    # vm_servercontrol  -u admin -p Manager -h localhost -s server -c events -A all
    Oracle VM Server Control utility 0.3.10.
    Connected.
    Command : events
    Server Events
    Acknowledging Virtual Machine events
            Virtual Machines on this server :
    VM : el5u4_32_pvm_1
    VM : el6u1_64_hvm_1
    VM : el5u7_64_hvm_1
    VM : el5u7_64_pvm_1
    VM : el6u1_64_pvm_1
    VM : el6u2bd5_32_pvhvm_1
    VM : el6u1_32_pvm_1
    VM : uek-boot-el5-64
    Exit..

    Acknowledging events on a given server, including any guest events.

    Friday May 18, 2012

    Oracle VM 3.1 released

    Today we formally announced the release of Oracle VM Server for x86 3.1. It is available for download from http://edelivery.oracle.com/linux. The software consists of 2 ISO images, the Oracle VM 3.1 Manager iso and Oracle VM 3.1 server iso... You can find the documentation library on the documentation website.

    Quite a few enhancements went into this release :

  • Oracle VM Manager UI enhancements to improve ease-of-use.
  • We continue to reduce the number of clicks to do actions and also have very easy to use and very streamlined configuration wizards and setup wizards. For instance for create VM or Create Server pool.

  • Storage enhancements for backup and recovery purposes.
  • It is now possible to do an nfs export on an Oracle VM server of a local/shared SAN/iscsi repository. When you have a filesystem based (ocfs2) repository with Oracle VM 3, only servers that are part of the Server pool can access the filesystem, with this new enhancement, you can define an nfs client hostname/IP address which will be allowed to do a remote nfs mount of this volume for backup purposes (or restore).

  • Using Unbreakable Enterprise Kernel 2 (2.6.39+) as the dom0 kernel
  • We now use the exact same Linux kernel for Oracle Linux with 2.6.39 (uek2) and for Oracle VM 3.1. This is possible because of all the work we have done together with Citrix/Xensource people over the last few years to get the necessary support for Xen and Linux merged into Linux. As I have mentioned in a previous blog entry, Linux mainline contains everything needed to have Linux be a complete dom0 management kernel for Xen without any additional patches. Unlike some of the FUD out there about "Xen" not being part of Linux, we have always made clear that Xen is a hypervisor. And Linux runs on top as the management/control domain kernel giving a great level of separation/isolation. In enterprise environments this is a big deal and helps us create solid, stable, secure solutions.

    The advantage for customers is that we do an incredible amount of testing on uek2 in terms of performance, network, disk io and scalability/stability in general and now Oracle VM automatically benefits from this additional testing. On top of that, for our partners, they do not have to worry about testing 2 different products in terms of drivers, since we share the exact same code base we have the exact same drivers for our hardware/storage vendors.

  • Clone many times
  • Simply the ability to say, clone a template X times in one step.

    You can find more information in the documentation set and in the release notes.

    Wednesday May 16, 2012

    More on templates and first boot configuration

    A little while back I wrote something up about building appliances (physical/virtual) using Oracle Linux. We build a whole set of Oracle VM templates and release them on our edelivery site. But some of what we use inside these templates can be more generically applied... Let me explain..

    A few years ago we created a Linux rpm called oraclevm-template-config. This is a very small package that contains a number of scripts that esssentially do the following:

  • install a service oraclevm-template-config that checks at boot time whether it needs to run or not
  • provides a default first boot configuration service for root password, network setup
  • provides a default cleanup script so that you can re-run this on an existing system, at next boot
  • provides a small set of library calls to use those functions in your own scripts
  • allows you to define your own first boot configuration scripts for your own applications to run
  • regenerates ssh host keys
  • all input was done on the local console or through kernel boot line parameters
  • These things are very straightforward and we wrote it specifically to build Oracle VM templates but there was nothing specific to Oracle VM or virtualization there. The newer version of this, which we released along with Oracle VM 3.x is very much tied into Oracle VM. In the new version (3.x) we integrate the first boot scripts and the configuration scripts with the Oracle VM VM api where it's now possible to send remote VM messages to configure the VM. So in order to automate deployment and have very integrated and fancy assemblies or Oracle VM based templates, this is the way to go. If, however, you want something very simple, with lcoal console (physical or virtual), where it can be on a physical server where you want to reboot and reconfigure and install, whether this is a VirtualBox VM or an Oracle VM VM or, for that matter any other Virtualization technology based VM. the older version is quite usable. I tend to use that one for the VirtualBox templates that I create all the time.

    We just published this rpm on http://public-yum.oracle.com. The rpm can be directly downloaded for Oracle Linux 5, here. Or when you have a system registered with public-yum or with ULN it is also in the add-ons channel. If you have a yum repo config then just enable the addon channel and you can simple type yum install oraclevm-template-config.

    So what are some of the basics you can do :

    initial install By default when you install template config, you have a service that provides the basic, configure root password, setup local network at what's called firstboot. To enable this to happen at the next reboot (which one can then call firstboot), just simple type /usr/sbin/oraclevm-template --enable. There is a simple default cleanup script which un-configures the network and removes ssh keys : /usr/sbin/oraclevm-template --cleanup. Keep in mind that when you run this script, the network will reset to dhcp and you will lose your network connection if you are in a remote ssh session.

    So once you do this and reboot your server or your VM. The next reboot will automatically start /usr/sbin/oraclevm-template --config and go through the questionnaire.

    The base configuration script is /etc/sysconfig/oraclevm-template. If you enable the service, you will see RUN_TEMPLATE_CONFIG=YES. This means next boot it will run. There is also a log file /var/log/oraclevm-template.log.

    If all you need is a simple way of packaging a VM or configure a physical server that deals with network setup, the above is enough. If you have your own software installed and it can handle IP changes without reconfiguration, then you can create an appliance and once firstboot happens, the configuration is set, network is setup, your application comes up and you're done. However, in some cases you want to run your own scripts and integrate with this template configuration. We also handle that case...

    If you write your own shell script (or really any script) that you install on the system, you can add that script to /etc/sysconfig/oraclevm-template and instead of doing the default firstboot run, it will at firstboot (when you call --enable) execute your script instead. Simply by setting TEMPLATE_CONFIG_SCRIPT=/myscript.sh. Inside that script you can do whatever you want... You can also provide a cleanup script for --cleanup, using TEMPLATE_CLEANUPS_SCRIPT=mycleanupscript.sh.

    Keep in mind that we do not automatically run --cleanup, this is a manual step. I would recommend doing the following :

  • /usr/sbin/oraclevm-template --enable
  • /usr/sbin/oraclevm-template --cleanup
  • history -c
  • shutdown -h now
  • We provide a few default function you can include in your own scripts. These functions are defined in /usr/lib/oraclevm-template/functions just source them in your bash script.

    ovm_configure_network [dhcp [hostname]] | [static] | [ip_adddress netmask gateway dns_server hostname]

    You can call this like ovm_configure_network "dhcp" in your script and it will configure your network that way. If you set it to static then there will be a dialog for entering ip, hostname, gateway, netmask, dns, or if you provide that in your function call it will just set the changes without dialog.

    ovm_cleanup_os
    A simple OS cleanup. This resets the network to dhcp and cleans up ssh keys.

    ovm_disable_firewall
    disable iptables

    press_anykey
    Shows user input to enter a key in your script execution flow

    ovm_error_message Displays an error message to the screen and also to /var/log/oraclevm-template.log

    ovm_info_message Displays an informational message to the screen and also to /var/log/oraclevm-template.log

    ovm_warn_message Displays a warning message to the screen and also to /var/log/oraclevm-template.log

    All of the above is very simple, yet very powerful. Like I mentioned in the beginning, I use this all the time when creating both VirtualBox and Oracle VM server templates and it makes my life so much easier. People often write these on their own but I wanted to share how to do this so, when you use Oracle Linux, and/or Oracle VM Server or Oracle VM VirtualBox or anything else, and you're building a cool appliance, physical or virtual, use it and you don't even need a support subscription to download it:)...

    Friday May 04, 2012

    building an appliance? physical ? virtual? production quality? use Oracle Linux

    Many companies are building appliances these days. Whether they are virtual appliance (VMs) or physical appliances, they all have an embedded OS inside. Now, I have looked at many of them, in particular virtual appliances and a few things struck me as interesting :

  • many had a random version of Linux included. Nothing bad about any of the distributions used at all, but just a hodgepodge of debian, fedora, opensuse, centos, etc... Almost all were marked "trial" or "demonstration"
  • a number of these appliances had a known commercial version of Linux, were marked trial edition and had restrictions on use (use for 30 days and so on).
  • So what are some problems with these scenarios?

    Well, first of all, even though there is absolutely nothing wrong at all with using community-based and community-maintained distributions (they do a great job), in a commercial setting, or when you build a product that lives on top of it, you need to have the ability to get formal support. Maybe not everyone needs this, but many companies do not have the knowledge (nor should they necessarily) to support the Linux distribution itself and they would want a reasonable formal service level for when something goes wrong, or when to expect something to get released.

    Secondly, in the case of using a commercial Linux distribution, there are distribution restrictions and even access restrictions. Certain Linux distribution vendors that offer formal support subscriptions will not give you access to the binaries without a subscription for support services and they also do not let you re-distribute without special agreements or making lots of modifications (or even restrict modifications).

    Enter Oracle Linux...

    What do you get?

  • freely downloadable ISO images for all the releases and major updates (Oracle Linux 4(4.4, 4.5, 4.6,4.7,4.8), Oracle Linux 5(5.1, 5.2, 5.3, 5.4, 5.5, 5.6, 5.7,5.8),Oracle Linux 6 (6.1, 6.2)). Both source code and binaries.No support subscription needed to download
  • freely re-distributable Both source code and binaries (no contract or support subscription needed to re-distribute, including logos and trademarks)
  • freely available bugfix errata and security errata, released at same time as we release them to customers with support subscriptions Both source code and binaries. In other words, every RPM released through a free public yum repository
  • the ability to purchase a support subscription for what you build, ship, support to your customers without any change to your appliance. No need to have a trial version or demo version using a community-edition Linux and a production version based on a commercial-edition Linux
  • use it in production, go for it, whatever the use case is, use it, if you want support, get a support subscription, if you don't need support, that's fine.
  • the ability to have a support subscription that offers on-line zero-downtime, rebootless security and bugfix updates as part of Oracle Linux Premier support with Oracle Ksplice technology
  • if you run with Centos or Red Hat Enterprise Linux, just replace the rpm list with the Oracle Linux RPMs and you're done.. so easy to switch
  • heavily tested on the biggest servers and biggest IO intensive, process intensive and network intensive workloads out there
  • do this for installs on physical servers, for Virtual Machines (whether this be Oracle VM Virtual Appliances, VMWare Virtual Appliances or something else, it doesn't matter)
  • One Enterprise Linux distribution, for all of the above. We make it easy for you. Grab the code, binaries and source, use it, distribute it, build your environments with it, freely, no contracts needed. Need our help, get a support subscription. Choice, open. Virtual, physical, cloud. Not just obfuscated tar balls. No license or activation key, good consistent SLAs for releasing security updates, well tested,... Run Oracle Linux in-house in test and development environments, run it in production environments, use it for customer systems, distribute it, any or all of the above. One distribution that you can manage across all the use cases. No need to manage different versions even if they're similar, no need to make different distribution choices based on your use case and pay/not pay.

    http://www.oracle.com/linux
    http://public-yum.oracle.com
    http://edelivery.oracle.com/linux

    Thursday May 03, 2012

    understanding memory allocation in oracle vm / xen

    As a follow up to my previous blog about cpu topology, I wanted to add a little bit about memory topology and memory allocation in the hypervisor. Most systems these days that are multi-socket are considered NUMA. Even though over the years, the NUMA-factor has gone down drastically,there still is a small amount of memory locality involved.

    My test setup is a dual socket server with 36GB memory. You can see this in Oracle VM Manager as part of the server info or directly on the server with xm info :

    # xm info 
    ..
    total_memory           : 36852
    free_memory            : 25742
    ..
    

    I have a few VMs running on this server which is why you see memory be lower than total. The 16GB VM is running with tmem enabled and because of that is not using up all memory but only the base memory needed to be active for the workload it's running.

    # xm list
    Name                                        ID   Mem VCPUs      State   Time(s)
    0004fb00000600001668dac79108cb84             2  4096     4     -b----    129.9
    0004fb0000060000804bac06a5087809             1  4096     4     -b----    129.4
    0004fb0000060000db9c71d539c940ed             3 16000     4     -b----     28.3
    Domain-0                                     0  1244    24     r-----    188.0
    

    Let's start with a clean slate and look at some statistics. The following commands will dump detailed memory information on your server :

    # xm debug-key u ; xm dmesg. Basically debug info for NUMA memory info. xm dmesg will show you the debug output.

    
    (XEN) 'u' pressed -> dumping numa info (now-0xFE:A1CFFF69)
    (XEN) idx0 -> NODE0 start->0 size->4980736
    (XEN) phys_to_nid(0000000000001000) -> 0 should be 0
    (XEN) idx1 -> NODE1 start->4980736 size->4718592
    (XEN) phys_to_nid(00000004c0001000) -> 1 should be 1
    (XEN) CPU0 -> NODE0
    (XEN) CPU1 -> NODE0
    (XEN) CPU2 -> NODE0
    (XEN) CPU3 -> NODE0
    (XEN) CPU4 -> NODE0
    (XEN) CPU5 -> NODE0
    (XEN) CPU6 -> NODE0
    (XEN) CPU7 -> NODE0
    (XEN) CPU8 -> NODE0
    (XEN) CPU9 -> NODE0
    (XEN) CPU10 -> NODE0
    (XEN) CPU11 -> NODE0
    (XEN) CPU12 -> NODE1
    (XEN) CPU13 -> NODE1
    (XEN) CPU14 -> NODE1
    (XEN) CPU15 -> NODE1
    (XEN) CPU16 -> NODE1
    (XEN) CPU17 -> NODE1
    (XEN) CPU18 -> NODE1
    (XEN) CPU19 -> NODE1
    (XEN) CPU20 -> NODE1
    (XEN) CPU21 -> NODE1
    (XEN) CPU22 -> NODE1
    (XEN) CPU23 -> NODE1
    (XEN) Memory location of each domain:
    (XEN) Domain 0 (total: 318627):
    (XEN)     Node 0: 282976
    (XEN)     Node 1: 35651
    
    The above output shows that the first 12 cpu's are bound to memory node 0 and the next 12 to memory node 1. The info shows how many pages of RAM are available on each node NODE0 start->0 size->4980736 and NODE1 start->4980736 size->4718592. the Dom0 domain is about 1.2Gb of RAM and it has some memory allocated on each NODE (it also has all of it's 24 vcpu's allocated across all threads in the box). Now let's start a VM.

    # xm list
    Name                                        ID   Mem VCPUs      State   Time(s)
    0004fb0000060000804bac06a5087809             4  4096     4     r-----      8.8
    Domain-0                                     0  1244    24     r-----    240.9
    
    # xm debug-key u ; xm dmesg
    ...
    (XEN) Memory location of each domain:
    (XEN) Domain 0 (total: 318627):
    (XEN)     Node 0: 282976
    (XEN)     Node 1: 35651
    (XEN) Domain 4 (total: 1048576):
    (XEN)     Node 0: 1048576
    (XEN)     Node 1: 0
    
    You can see that the newly started VM (domain 4) has 4Gb allocated on node 0.
    # xm vcpu-list 4
    Name                                ID  VCPU   CPU State   Time(s) CPU Affinity
    0004fb0000060000804bac06a5087809     4     0     0   -b-       4.8 0-3
    0004fb0000060000804bac06a5087809     4     1     3   -b-      26.1 0-3
    0004fb0000060000804bac06a5087809     4     2     2   -b-       3.5 0-3
    0004fb0000060000804bac06a5087809     4     3     1   -b-       2.4 0-3
    
    The VM also has its virtual CPUs bound to node 0. Let's start another VM.

    # xm vcpu-list 6
    Name                                ID  VCPU   CPU State   Time(s) CPU Affinity
    0004fb00000600001668dac79108cb84     6     0    19   r--       2.2 19-23
    0004fb00000600001668dac79108cb84     6     1    23   r--      24.6 19-23
    0004fb00000600001668dac79108cb84     6     2    20   -b-       1.4 19-23
    0004fb00000600001668dac79108cb84     6     3    22   -b-       1.1 19-23
    
    # xm debug-key u ; xm dmesg
    ...
    (XEN) Memory location of each domain:
    (XEN) Domain 0 (total: 318627):
    (XEN)     Node 0: 282976
    (XEN)     Node 1: 35651
    (XEN) Domain 4 (total: 1048576):
    (XEN)     Node 0: 1048576
    (XEN)     Node 1: 0
    (XEN) Domain 6 (total: 1048576):
    (XEN)     Node 0: 0
    (XEN)     Node 1: 1048576
    
    As you can see, this domain 6 has vCPUs bound to node 1, and Xen automatically also allocates memory from node 1. To ensure memory locality. It tries hard to keep memory and CPU as local as possible. Of course when you run with many VMs with many vCPUs then memory allocation will be spread out across multiple nodes.

    After starting a 16Gb VM on this server (domain 7), now that 8Gb is allocated, you will see that this 16Gb VM's memory allocation is across the 2 memory nodes :

    (XEN) Memory location of each domain:
    (XEN) Domain 0 (total: 318627):
    (XEN)     Node 0: 282976
    (XEN)     Node 1: 35651
    (XEN) Domain 4 (total: 1048576):
    (XEN)     Node 0: 1048576
    (XEN)     Node 1: 0
    (XEN) Domain 6 (total: 1048576):
    (XEN)     Node 0: 0
    (XEN)     Node 1: 1048576
    (XEN) Domain 7 (total: 4097012):
    (XEN)     Node 0: 2524148
    (XEN)     Node 1: 1572864
    

    Wednesday May 02, 2012

    figuring out cpu topology in oracle vm

    I have had a few people in the past ask about CPU topology on Oracle VM and how to figure out which CPUs, cores or threads are used by a VM... Here's a quick overview :

    1) get a summary of your server

    The xm info command prints out a whole bunch of information about your system.

    Look for the lines below in the output for detail on your systems CPUs.

    # xm info
    ...
    nr_cpus                : 8
    nr_nodes               : 1
    cores_per_socket       : 4
    threads_per_core       : 2
    cpu_mhz                : 3200
    ...
    
    This server has a single socket with 4 cores and 2 threads per core. Total of 8 "CPUs". so cpu0..7 really is thread 0..7.

    # xm info
    ...
    nr_cpus                : 12
    nr_nodes               : 1
    cores_per_socket       : 6
    threads_per_core       : 2
    ...
    
    This server has a single socket with 6 cores and 2 threads per core. Total of 12 "CPUs".

    2) get the cpu topology

    The xenpm command lets you print out the thread/core/socket topology on a given server :

    # xenpm get-cpu-topology
    CPU     core    socket  node
    CPU0     0       0       0
    CPU1     0       0       0
    CPU2     1       0       0
    CPU3     1       0       0
    
    In the above example, it's a single socket machine with 2 cores and 2 threads per core. cpu 0 is thread 0 of core 0, cpu 1 is thread 1 of core 0, cpu 2 is thread 0 of core 1, cpu 3 is thread 1 of core 1. If you were to have cpus="0,1" in the VM config file, you would be running on core 0. If you were to have cpus="0-3" in the VM config file, you would actually run a VM on both cores.

    # xenpm get-cpu-topology
    CPU     core    socket  node
    CPU0     0       0       0
    CPU1     0       0       0
    CPU2     1       0       0
    CPU3     1       0       0
    CPU4     2       0       0
    CPU5     2       0       0
    CPU6     3       0       0
    CPU7     3       0       0
    
    In the above example, you have a single socket server with 4 cores and 2 threads per core. cpu 0 -> thread 0 of core 0, cpu 1 -> thread 1 of core 0 etc.If you were to have cpus="4-7" in the VM config file, you would run the VM on cores 2 and 3.

    # xenpm get-cpu-topology
    CPU     core    socket  node
    CPU0     0       0       0
    CPU1     0       0       0
    CPU2     1       0       0
    CPU3     1       0       0
    CPU4     2       0       0
    CPU5     2       0       0
    CPU6     8       0       0
    CPU7     8       0       0
    CPU8     9       0       0
    CPU9     9       0       0
    CPU10    10      0       0
    CPU11    10      0       0
    
    In the above example, you see a single socket server with 6 cores hyperthreaded.

    3) Show the guest/VM vcpu bindings to physical threads : Use the command xm vcpu-list to list a summary of which virtual CPU is running on which physical CPU.

    # xm vcpu-list 1
    Name                                ID  VCPU   CPU State   Time(s) CPU Affinity
    0004fb00000600007c351fa24276c63f     1     0     5   -b-    4673.6 5-6
    0004fb00000600007c351fa24276c63f     1     1     5   -b-    4534.0 5-6
    
    If you add the domain ID to the command xm vcpu-list 1, you get the information for just that guest or VM. In the above example, you have a guest with 2 virtual CPUs both running, at this time, on physical cpu (thread in my case) 5. The column CPU Affinity show's 5-6 which means that both virtual CPUs could be running on either thread 5 or 6. This shows that the guest is pinned on those 2 threads. Combined with the information of xenpm get-cpu-topology I can then see that in my case CPU 5 is thread 1 of core 2 and CPU 6 is thread 1 of core 8. So this 2 vcpu guest is pinned to 2 separate physical cores.

    # xm vcpu-list
    Name                                ID  VCPU   CPU State   Time(s) CPU Affinity
    0004fb00000600007c351fa24276c63f     1     0     5   -b-    4676.8 5-6
    0004fb00000600007c351fa24276c63f     1     1     5   -b-    4537.0 5-6
    Domain-0                             0     0     0   -b-     932.1 any cpu
    Domain-0                             0     1     6   -b-    1168.0 any cpu
    Domain-0                             0     2     7   -b-    1010.8 any cpu
    Domain-0                             0     3    11   -b-     903.0 any cpu
    Domain-0                             0     4     8   -b-     494.2 any cpu
    Domain-0                             0     5     9   r--     773.8 any cpu
    Domain-0                             0     6     1   -b-     522.7 any cpu
    Domain-0                             0     7     2   -b-     785.1 any cpu
    Domain-0                             0     8     4   -b-     473.8 any cpu
    Domain-0                             0     9     3   -b-     728.1 any cpu
    Domain-0                             0    10    10   -b-     490.8 any cpu
    Domain-0                             0    11     0   r--    1219.6 any cpu
    
    This is the same system but xm vcpu-list without argument. It also shows the dom0 guest. As you can see in this example, dom0 can run on any physical thread and the CPU Affinity is any cpu which implies any virtual CPU can be scheduled on any physical thread so no pinning or partitioning.

    4) using ovm_utils ovm_vmcontrol lets a user or admin set and get the cpu/vcpu bindings for a VM through Oracle VM Manager in 3.0.

    # ./ovm_vmcontrol -u admin -p Manager1 -h localhost -v apitest -c vcpuget
    Oracle VM VM Control utility 0.4.2.
    Connected.
    Command : vcpuget
    Current pinning of virtual CPUs to physical threads : 5,6
    
    In the above example, I have a VM named apitest and using the action vcpuget it shows me that virtual cpus of this guest are bound to threads 5 and 6. This happens to be the VM in the previous example which showed that this was core 2 and core 8 (3).

    Now let's bind the vcpu's to core 0, I just run the following command :

    # ./ovm_vmcontrol -u admin -p Manager1 -h localhost -v apitest -c vcpuset  -s 0
    Oracle VM VM Control utility 0.4.2.
    Connected.
    Command : vcpuset
    Pinning virtual CPUs
    Pinning of virtual CPUs to physical threads  '0' 'apitest' completed.
    
    Now, running the xm vcpu-list command, I can see that this immediately took effect (and also will on subsequent startups of the VM) :
    # xm vcpu-list 1
    Name                                ID  VCPU   CPU State   Time(s) CPU Affinity
    0004fb00000600007c351fa24276c63f     1     0     0   -b-    4687.6 0
    0004fb00000600007c351fa24276c63f     1     1     0   -b-    4547.2 0
    
    The VM now has CPU Affinity 0 for both virtual CPUs.

    Friday Apr 13, 2012

    More Oracle VM templates for PeopleSoft and Oracle Enteprise Manager

    Just as I wrote up a blog promoting the Oracle VM Ebusiness suite templates, we also pushed out 2 other products :

  • Oracle Enterprise Manager Cloud Control 12c
  • Oracle PeopleSoft FSCM 9.1 and PeopleTools 8.52.03
  • They can be downloaded from edelivery.

    Same advantage... you download the template, import it and you have a completely pre-installed set of products. That's application deployment, not just VM deployment. That's flexibility across the stack, not just a hypervisor, not just virtualization, but a complete solution stack.

    Oracle Ebusiness Suite 12.1.3 Oracle VM templates

    Steven Chan just published a great blog entry that talks about the release of a new set of Oracle VM templates.

    Oracle Ebusiness Suite 12.1.3.

    You can find the blog post here.

    Templates are available for:

  • E-Business Suite 12.1.3 Vision (64-bit)
  • E-Business Suite 12.1.3 Production (32-bit)
  • E-Business Suite 12.x Sparse Middle Tiers (32-bit and 64-bit)
  • Thanks Steven!

    Why does this stuff matter? Well, in general, virtualization (or cloud) solutions provide an easy way to create Virtual Machines. Whether it's through a "cloud api" or just a virtualization API. But all you end up with, in the end, is still just a Virtual Machine... Maybe with an OS pre-installed/pre-configured. So you have flexibility of moving VMs around and providing a VM but what about the actual applications (anything more than a very basic app)?

    The application administrator then still has to go and install and configure the OS for that application and install the application and its patches and basic configuration so that the application user then can go in. Building gold images for complex software stacks that are not owned by the users/admins is always very difficult.

    With our templates, we provide a number of things :

  • Oracle Linux pre-installed and pre-configured with the minimum required packages for that application to run. (so it's secure)
  • Oracle Linux can be distributed and used for free or with a support subscription. There is no trial license, there is no registration key, no alpha version or community version versus enterprise version. You get what we provide in our engineered systems, what we provide support for, without change. Supported out of the box. No virtual Trial appliances, no prototypes, no POC. What you download is production ready without change.
  • The applications are installed by the developers of the application. The database team builds database templates, the applications engineering team builds applications templates. The first boot/configuration scripts ask for the basic information such as hostname, ip address, user passwords and then go off and set everything up correctly.
  • All tested together -> application -> operating system -> hypervisor. not 3 (or more) products from 3(or more) different companies.
  • Thursday Apr 05, 2012

    Some Oracle VM 3 updates

    Today we did another patch set update for Oracle VM 3 (3.0.3-build 240). This can be downloaded from My Oracle Support as patch ID 13614645. There are quite a few updates in here and I highly recommend any Oracle VM 3 customer or user to install this update.

    This patch can be installed on top of Oracle VM 3.0 versions 3.0.2 and 3.0.3. The patch is cumulative for 3.0.3. So if you already installed patch update 1 (3.0.3-150) then this will just be incremental on top of that and brings you to 3.0.3-build 227. There is a readme file which contains the patchlist in the patch info.

    The following patches are released on ULN for Oracle VM server 3.0 :

  • initscripts-8.45.30-2.100.18.el5.x86_64 The inittab file and the /etc/init.d scripts.
  • kernel-ovs-2.6.32.21-45.6.x86_64 The Linux kernel
  • kernel-ovs-firmware-2.6.32.21-45.6.x86_64 Firmware files used by the Linux kernel
  • osc-oracle-ocfs2-0.1.0-35.el5.noarch Oracle Storage Connect ocfs2 Plugin
  • osc-plugin-manager-1.2.8-9.el5.3.noarch Oracle Storage Connect Plugin Infrastructure
  • osc-plugin-manager-devel-1.2.8-9.el5.3.noarch Oracle Storage Connect Plugin Development
  • ovs-agent-3.0.3-41.6.x86_64 Agent for Oracle VM
  • xen-4.0.0-81.el5.1.x86_64 Xen is a virtual machine monitor
  • xen-devel-4.0.0-81.el5.1.x86_64 Development libraries for Xen tools
  • xen-tools-4.0.0-81.el5.1.x86_64 Various tooling for the manipulation of Xen instances
  • Errata emails will be sent in the next few days with details on the above updates. Or you will find them here.

    I also did an update of my Oracle VM utilities to 0.4.0. They are also available from My Oracle Support, patch ID 14736239.
    These utils can be unzipped and installed on the server running Oracle VM Manager. Typically in /u01/app/oracle/ovm-manager-3/ovm_utils. There is a set of man pages in /u01/app/oracle/ovm-manager-3/ovm_utils/man/man8. There now are 6 commands :

  • ovm_vmcontrol : VM level operations
  • ovm_servercontrol : server level operations
  • ovm_vmdisks : virtual disk/physical location mapping for VM disks
  • ovm_vmmessage : message passing utility between the manager and the VM tools (in the Oracle VM templates)
  • ovm_repocontrol : repository level operations
  • ovm_poolcontrol : pool level operations

    Some of the new changes :

  • at a pool level, acknowledge events and cascade to servers and virtual machines with outstanding events
  • at a pool level, do a rescan of the storage for fibrechannel/iscsi disks if you add new devices (it does this operation then on every running server)
  • at a repository level, fixup a device if it had a failed create repository
  • at a repository level, refresh the repository and this will update the free space in the UI for ocfs2 repositories
  • at a server level, acknowledge server events and cascade to virtual machines if needed
  • at a VM level, acknowledge VM events
  • at a VM level, bind vcpus to cores with vcpuset/vcpuget

    Please see the man pages and remember that these tools are just written As Is - no SRs... (per the documentation) Hopefully they are useful.

  • Thursday Mar 29, 2012

    4.8M wasn't enough so we went for 5.055M tpmc with Unbreakable Enterprise Kernel r2 :-)

    We released a new set of benchmarks today. One is an updated tpc-c from a few months ago where we had just over 4.8M tpmc at $0.98 and we just updated it to go to 5.05M and $0.89. The other one is related to Java Middleware performance. You can find the press release here.

    Now, I don't want to talk about the actual relevance of the benchmark numbers, as I am not in the benchmark team. I want to talk about why these numbers and these efforts, unrelated to what they mean to your workload, matter to customers. The actual benchmark effort is a very big, long, expensive undertaking where many groups work together as a big virtual team. Having the virtual team be within a single company of course helps tremendously... We already start with a very big server setup with tons of storage, many disks, lots of ram, lots of cpu's, cores, threads, large database setups. Getting the whole setup going to start tuning, by itself, is no easy task, but then the real fun starts with tuning the system for optimal performance -and- stability. A benchmark is not just revving an engine at high rpm, it's actually hitting the circuit. The tests require long runs, require surviving availability tests, such as surviving crashes -and- recovery under load.

    In the TPC-C example, the x4800 system had 4TB ram, 160 threads (8 sockets, hyperthreaded, 10 cores/socket), tons of storage attached, tons of luns visible to the OS. flash storage, non flash storage... many things at high scale that all have to be perfectly synchronized.

    During this process, we find bugs, we fix bugs, we find performance issues, we fix performance issues, we find interesting potential features to investigate for the future, we start new development projects for future releases and all this goes back into the products. As more and more customers, for Oracle Linux, are running larger and larger, faster and faster, more mission critical, higher available databases..., these things are just absolutely critical. Unrelated to what anyone's specific opinion is about tpc-c or tpc-h or specjenterprise etc, there is a ton of effort that the customer benefits from. All this work makes Oracle Linux and/or Oracle Solaris better platforms. Whether it's faster, more stable, more scalable, more resilient. It helps.

    Another point that I always like to re-iterate around UEK and UEK2 : we have our kernel source git repository online. Complete changelog of the mainline kernel, and our changes, easy to pull, easy to dissect, easy to know what went in when, why and where. No need to go log into a website and manually click through pages to hopefully discover changes or patches. No need to untar 2 tar balls and run a diff.

    Monday Mar 26, 2012

    Using EPEL repos with Oracle Linux

    There's a Fedora project called EPEL which hosts a set of additional packages that can be installed on top of various distributions such as Red Hat Enterprise Linux, CentOS, Scientific Linux and of course also Oracle Linux. These packages are not distributed by the distribution vendor and as such also not supported by the vendors (including Oracle) however for users that want to pick up some extras that are useful, it's very easy to do this.

    All you need to do is download the EPEL RPM from the website, install it on Oracle Linux 5 or Oracle Linux 6 and run yum install or yum search to get the packages.

    example : 
    # wget http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-5.noarch.rpm
    
    # rpm -ivh epel-release-6-5.noarch.rpm
    
    # yum repolist
    Loaded plugins: refresh-packagekit, rhnplugin
    repo id                repo name                                          status
    epel                   Extra Packages for Enterprise Linux 6 - x86_64      7,124
    
    
    The folks that build these repositories are doing a great job at adding very useful packages. They are free, but also unsupported of course.

    Thursday Mar 22, 2012

    Setting up Oracle Linux 6 with public-yum for all updates

    I just wanted to give you a quick example on how to get started with Oracle Linux 6 and start using the updates we published on http://public-yum.oracle.com.

  • Download Oracle Linux (without the requirement of a support subscription) from http://edelivery.oracle.com/linux.

  • Install Oracle Linux from the ISO or DVD image

  • Log in as user root
  • Download the yum repo file from http://public-yum.oracle.com

    # cd /etc/yum.repos.d
    # wget http://public-yum.oracle.com/public-yum-ol6.repo
    

  • If you want, you can edit the repo file and enable other repositories, I enabled [ol6_UEK_latest] by just setting enabled=1 in the file with a text editor.

  • Run yum repolist to show the registered channels and you see we are including everything including the latest published RPMs.

    Now you can just run yum update and any time we release new security errata or bugfix errata for OL6, they will be posted and you will automatically get them. It's very easy, very convenient and actually very cool. We do a lot more than just build OL RPMs and distribute them, we have a very comprehensive test farm where we test the packages extensively.

  • About

    Wim Coekaerts is the Senior Vice President of Linux and Virtualization Engineering for Oracle. He is responsible for Oracle's complete desktop to data center virtualization product line and the Oracle Linux support program.

    You can follow him on Twitter at @wimcoekaerts

    Search

    Categories
    Archives
    « April 2014
    SunMonTueWedThuFriSat
      
    1
    2
    3
    4
    5
    6
    7
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
       
           
    Today