Tuesday Aug 27, 2013

Single Instance/RAC Oracle VM templates update

Superstar Saar just released a new set of Oracle VM templates. We (Oracle) just released 2 patch sets for the Oracle RDBMS - 11.2.0.4.0 and 11.2.0.2.11 (x86 and x86_64)

Simultaneously, Saar updated his Oracle VM templates to include these latest patchsets as well for both architectures (x86 and x86_64).

  • 11.2.0.4.0 with OL5
  • 11.2.0.4.0 with OL6
  • 11.2.0.2.11 with OL5
  • 11.2.0.2.11 with OL6
  • These templates can be deployed on Oracle VM using the DeployCluster tool, all you need to do is create a very simple textfile with the parameters.

    All templates default to UEK2 2.6.39-400. The templates can be used to create Single Instance, Single Instance with HA (Oracle Restart) and Oracle RAC databases.

    The options vary from ASM, NFS, OCFS2 for db files, local filesystem, no DB, Clusterware only etc.

    Full stack, download, deploy. Production RDBMS code, Production Oracle Linux.

    http://www.oracle.com/technetwork/server-storage/vm/database-templates-12c-11gr2-1972804.html

    Simple Sample script:

    # cat netconfig.ini 
    NODE1=server3
    NODE1IP=10.0.0.4
    PUBADAP=eth0
    PUBMASK=255.255.255.0
    PUBGW=10.0.0.1
    DOMAINNAME=wimmekes.net  # May be blank
    DNSIP=10.0.0.1  # Starting from 2013 Templates allows multi value
    CLONE_SINGLEINSTANCE=yes  # Setup Single Instance
    

    and then # deploycluster -u admin -p mypassword -H localhost -M mydbvm1 -> done

    Thursday Aug 22, 2013

    A little sample snmp module for Oracle VM Server 3.2

    I was looking at snmp for a few days and decided to put together a little snmp module (extension) that would work on Oracle VM Server (3.2 and up). In 3.2 we started to include the net-snmp rpms to allow customers to monitor any given Oracle VM server with standard SNMP tools. Whether that be cacti, snmpwalk, even Oracle Enterprise Manager (snmp fetchlets) or whatever tool. The standard net-snmp installation will expose MIBs and return data pretty much exactly the same as what you would get when installing net-snmp on Oracle Linux and monitoring an Oracle Linux server.

    The little snmp module I added exposes a few extra Oracle VM specific objects. To start with I basically looked at the data you can see on the local console of the server (version, cluster state, management uuid,...). I created a custom MIB (falls in the oracle enterprise oid range ( 1.3.6.1.4.1.111.57.1.1 – 1.3.6.1.4.1.111.57.1.13 )) and packaged it all up in a little RPM (ovs-snmp.rpm) that can be installed in dom0.

    ovs-snmp is an extension to net-snmp. It is a dynamically loadable module that allows extra bits to be monitored in dom0 that are specific to Oracle VM. Once the RPM is installed, snmpd.conf must be updated to load the module at start of snmpd. When you restart the snmpd service, you then have access to an extra MIB.

    This extra MIB is documented in /usr/share/snmp/mibs/OVS-MIB.txt The raw oid range for the OVS extension is from 1.3.6.1.4.1.111.57.1.1 – 1.3.6.1.4.1.111.57.1.13. The module also contains a trap at 1.3.6.1.4.1.111.57.2.0. The trap is defined around ovsAgentState (Running/Stopped) and will allow an admin to monitor the state of the Oracle VM Server agent which is a critical component of every server installed and get a notification from the snmpd.

    If you copy the OVS-MIB.txt file over to another regular server and put the file in the same directory (/usr/share/snmp/mibs) then you can use the text version instead of the raw oid numbers. For instance : 1.3.6.1.4.1.111.57.1.1 is the same as : ORACLE-OVS-MIB::ovsType. This is more humanly readable.

    The following set of attributes are defined in the MIB :

    ovsType           : Oracle VM Server
    ovsVersion        : Version of Oracle VM Server installed
    ovsMaster         : Master node in serverpool?
    ovsClusterState   : Cluster configured / online?
    ovsClusterType    : NFS or Lun based
    ovsClusterStorage : the nfs mount or lun used for the server pool filesystem
    ovsManagerUUID    : UUID of the Oracle VM Manager instance
    ovsServerpoolName : serverpool name this server is a member of (or None)
    ovsServerpoolIP   : Virtual IP address of the serverpool master
    ovsAgentState     : Agent running or stopped
    ovsFreeMemory     : free memory available for Virtual Machines on this server
    ovsHostname       : hostname as known by the Oracle VM Manager instance
    
    vmTable           : table with an index listing all the currently running VMs
                        columns -> vmIndex, vmType
    

    example snmpd.conf file:

    # more /etc/snmp/snmpd.conf
    rocommunity public
    syslocation "hq"
    dlmod ovs /usr/lib64/ovs-snmp/ovs.so
    

    Some examples :

    # snmpwalk -v 1 -c public -O e localhost ORACLE-OVS-MIB::ovsAgentState
    ORACLE-OVS-MIB::ovsAgentState.0 = STRING: Running
    # snmpwalk -v 1 -c public -O e localhost 1.3.6.1.4.1.111.57.1.1
    SNMPv2-SMI::enterprises.111.57.1.1.0 = STRING: "Oracle VM Server
    "
    
    You can download the rpm from MOS, bug number is 17344092. At this point it's provided as-is, tech preview. Once I get some feedback on it we will consider integrating this.

    have fun

    Tuesday Aug 13, 2013

    The life of a Linux RPM (package)

    Another frequently asked question related to Oracle Linux is how versions of specific packages (RPMs) are picked.

    A Linux distribution is basically a collection of a ton of open source projects that make up the Operating System environment, with at its core the Linux kernel. Linux as a development project is about the Linux kernel specifically. There are then many (1000's) of random open source projects out there and a Linux distribution basically is an OS made up of, at its core, the kernel, and tons of those other projects packaged up. Now some packages are more critical than others, there's a small true core of packages that you will find in any Linux distribution, glibc (c library), gcc (compiler), filesystem utils, core utils, binutils, bash,... etc. A good guess would be about 150-200 packages that make up pretty much any more or less usable environment (yes you can do with far fewer but I 'm talking standard OS installs here...)

    All these projects have their own development cycle, their own maintainers/developers, their own project plans, and their own dependencies. They just kind of all move along on their own pace, they're worked on by (usually) different people and so on. So how do these 1000's of packages get into Oracle Linux?

    Oracle Linux is an exact replica of Red Hat Enterprise Linux (same packages, same source code, same versions,...)... so that's what we base our distribution on (similar to CentOS). Now Red Hat Enterprise Linux, in turn, uses the Fedora Linux distribution as its upstream baseline. Fedora is a community distribution (fedoraproject.org) that typically lives far ahead of what any one would install for a stable environment. It's a community driven, cutting edge Linux distribution. Fedora as a project has very frequent releases (like every 6 or so months). The fedora maintainers distribute maintainership of these 1000's of RPMs across a group of people and they gather newer versions of all these projects and build them for a given version of Fedora. They then stabilize this and release it. By stabilizing, I mean, they create packages, test out if all the dependencies work, if there's a build environment that's consistent and if there are bugs, and fix them of course.

    So Fedora evolves rapidly (like every 6 months a new release) and as you go from Fedora 12 to 13 to 14 etc, you see the packages of gcc and glibc and all other stuff evolve version by version, gcc 3.2 to 3.3 to 4.0 to 4.4 etc. Depending on when the fedora project starts "freezing" the package list of the next version, that's what the various versions of those 1000's of packages will be based off of. The maintainers usually will, at some point of the release cycle, take the latest "stable" version of a given project (say gcc) and check that version into the Fedora tree. What happens here is that you typically see Fedora pick up the stable versions of projects pretty regularly. It helps shake out bugs, it reaches a large (end)-userbase and it helps the Linux community that wants to be on the cutting edge by doing a lot of the packaging for them and it helps the downstream use of Fedora because many of the base/generic obvious bugs and build issues have been resolved during the Fedora test, dev and release process.

    Now, because Fedora moves so rapidly, newer versions of RHEL, and as such OL, obviously skip quite a few versions of Fedora. For instance, RHEL6 is based on Fedora 12/13, RHEL5 was based on Fedora Core6, etc... A new version of RHEL is released every few years. So Red Hat decides at a certain point in time to take a snapshot of a given Fedora release they deem stable enough at that point in time and then fork that internally into a separate repository, change the trademarks,logos, add some packages that might not be in Fedora, tests the components for a more commercial use, server use, (most of Fedora is desktop use) and then releases that as the next version of RHEL. And we then similarly follow with OL.

    An important point to make is that within a given release cycle of RHEL or OL, the version of the packages typically doesn't change, at least not the major version (usually not 2 digit versions). For the lifetime of, say, OL5, the version of, say, glibc, will remain pretty static. It will include bugfixes over the lifetime of the distribution version, security fixes and sometimes minor important things that might get backported of a newer version (albeit rare) but that's it. So you have a relatively static vesion of an OS, it improves in stability, quality, security but it doens't improve much in terms of functionality. (Most of the enhancements would probably be in the kernel.).

    This also means that a Linux distribution (RHEL, OL, CentOS,...) can skip package versions, if some external project goes from version 2.1 to 2.2 to 2.3 to 3.0 to 3.1 etc... over the lifecycle of OL5 and OL6, then OL5 might contain 2.1 and OL6 might contain 3.0 or 3.1. You won't see versions in between get picked up. Or, again, in some rare cases, if there's something really important that went into 2.3 that would be really relevant, it might have gotten backported to 2.1 as part of RHEL/OL. You cannot expect that OL5 would go from 2.1 to 2.2 to 3.0 for that given package. That's just not how things work. So if you expect major enhancements or features of some package that's newer than the version that's in the current distribution, you might (likely) have to wait until the next major release. Example : OL5 contains glibc 2.5, OL6 contains glibc 2.12. If there was something really, really important in glibc 2.8, that might have gotten backported into 2.5 and gotten into OL5, but it's unlikely. And OL5 will not start adding 2.6, or 2.7 or so into the distribution. And then the same cycle starts again with OL6, it contains glibc 2.12... but the current version of glibc upstream is 2.18, and Fedora 19 contains glibc 2.17. So the future version of RHEL7/OL7 might end up with 2.16 or 2.17, and it would have skipped over 2.13-2.15. One cannot expect that the commercial distributions backport features of future package versions into prior versions. That doesn't happen for OL, or for RHEL or CentOS or SLES etc.

    What does happen, and it's important to point this out, is the fixing of CVE's/Security vulnerabilities or critical bugfixes. Example, let's say there is a security issue found in glibc 2.17 (upstream), and this is also relevant to glibc 2.5 found in RHEL5/OL5. We obviously will end up fixing that in 2.5 (backport the security fix) and in 2.12 (OL6)... So in terms of critical fixes and security vulnerabilities found in any version of a supported Linux distribution's package, those will be found in various versions, where they matter.

    You can always track this by looking at the changelog of an RPM or look for a CVE number and you will see hits on different versions of an RPM in different versions of OL where it is relevant.

    This is pretty normal stuff, new features go into new versions of a product, like new features go into new versions of the oracle database and we will fix problems and backport changes into older versions but you will not see new features for a new versions of the database pop up in an old version of the database. It's not rocket science. A Linux Distributions is a product based on tons of small subcomponents but in the end the major release is the overarching "feature" release.

    The few exceptions (obviously there are exceptions :-) are : (1) it's possible that new packages for new products or components are introduced during the lifetime of the OS release, a new RPM can be introduced in 5.8 or 5.9 or so... (2) some of the backports of features I talked about earlier can introduce some enhancements, although rare (3) the Linux kernel is probably the most lively component in the OS where the rate of enhancements is the largest compared to anything else.

    I hope this helps.

    Of updates and errata.

    A frequently discussed topic inside Oracle and also outside with customers and partners is Oracle Linux versions and how to treat updates and support and certifications and minimum levels. Here's our take on it, from the Oracle Linux side.

    When talking about Oracle Linux and versions, there really are 3 major components :

    -1- A major new release, such as Oracle Linux 5, Oracle Linux 6,...

    A major new release is an update of the entire OS, kernel, userspace, all the 1000's of packages that make up Oracle Linux. A major release is significant in change compared to the previous version. You will see pretty much every package(RPM) updated to a whole new version, like Apache, MySQL, GCC, glibc, X-windows, gnome, etc etc... In a number of cases, the owners of the packages are not so careful about maintaining backward compatibility and introduce different style config files that could make upgrades difficult or sometimes impossible. This is one of the reasons why it's not easy to 'upgrade' from, say, Oracle Linux 5 to Oracle Linux 6. While it would be ok for a good number of RPMs, it's not guaranteed to work for everything. A config file might get overwritten or the older edited config file is not compatible with the new version...etc. So upgrading here, very often is a new/fresh install of the OS on a server. We see most customers go to new versions at time of a hardware refresh and use that as a good opportunity, and that makes total sense.

    Because a major version has signifcant changes (new kernel, with new features, new glibc,... so core components), unfortunately sometimes changes that affect or really need change not just testing (upstart from OL5 to OL6,... potentially interesting changes due to systemd in future versions), there's typically a good reason to do certification testing of userspace applications on top of the new version. The way we work here is that we build on the lowest common version of the OS we want to support and run certification against newer versions. So we build an Oracle product on Oracle Linux 5, we won't support anything on a version that's older than Oracle Linux 5, and we do extras testing and certification for any new major version, like Oracle Linux 6. This can require us to do some changes to our application (like the database) in order for us to be able to complete that certifcation (or OS bugfixes), this was a big reason as to why it took a long time to be able to consider Oracle Linux 6 certified for the database. (due to, for instance, things like upstart)

    ISV's typically will do the same thing, it takes some time for development teams to add a new major version of the OS, again, because sometimes application changes might be required or OS changes might be required, like bugfixes due to testing.

    A major OS release happens every few years, not more frequently. OL4 -> (for us 2006 since we started with update 4), OL5 -> mid 2006 OL6 -> early 2011

    -2- A minor update, really just a point in time current snapshot of a given major release with bugfixes and security updates applied (Oracle Linux 5 update 9, Oracle Linux 6 update 4...)

    A minor update, released on a regular basis (several months), is really just a snapshot in time of the major OS release. As a version is out, on a regular basis, there are bugfixes, security updates introduced into that release, so out of the 1000's of packages, a number of those will have a minor bugfix update every now and then. These updates really are focused on fixing bugs and fixing security vulnerabilities. New features are normally not introduced into packages as part of a certain major OS release. Sometimes there are some new things added but they are only introduced if it doesn't break compatibility or doesn't change the understood use of that package. Because a major OS release is only every few years and to make it easy to provide a good snapshot of fixes within the release cycle, updates are done on a regular basis. The update is literally a snapshot at a given date and then the latest version of all packages within the release are bundled together and put onto an updated media (iso).

    This makes it convenient for users in a number of ways : 1) As mentioned in the first point, sometimes we need to create OS bugfixes for an application to work on a given version of the OS, these fixes go into the OS at some point or another. It is very convenient to use the update releases to point customers to the minimum version as a starting point. 2) a very important component that gets updated regularly (bugfixes, updated device drivers/hardware support) is the kernel. So if new hardware is released, you typically need a new boot kernel to recognize newer hardware, an update release pretty much always have a newer version of the linux kernel with updated device drivers on the installation media and that's required to be able to install a given OS version on newer hardware. So it might be that you need a minimum level of Oracle Linux 5.9 to install on the latest version of a given server... but it's still "Oracle Linux 5" as a product.

    So, look at update releases as minimum patch levels of an OS release, not as a product version. Oracle Linux 5.9 is not a product version, Oracle Linux 5 is the product, update 9 just implies a recent point in time of the fixes made in the product. Too often a customer (or product team) certifies on very specific update versions, and make the mistake of implying really just that version and not that version and newer. We all stand by the fact that if a minimum version required is, let's say Oracle Linux 5 update 6, then that always implies Oracle Linux 5, starting from the snapshot point update 6 and anything newer since then, as part of Oracle Linux 5.

    There is no point in sticking to a given update version and consider that a release, there are -always- important fixes, whether it be potential crashes or security vulnerabilities released after that update and it really is not a best practice to stick at a certain point in time. We always do a lot of testing on any bugfixes update or security vulnerability and there's no breakage or introduction of incompatibilities within a given OS version. Look at Oracle Linux X update Y as running Oracle Linux X, update Y is just a point in time. ISVs should point out a starting update of what is considered supported or certified but with the understanding that it will imply anything changed from that point on for the same major version X. So if 5.6 is the base certification level, then anything post 5.6 as part of Oracle Linux 5 should be OK and it makes sense to try to remain as current as possible.

    -3- An errata update, an update of a given package, either because of a bugfix or a security vulnerability fix.

    At any point in time during the lifetime of a major OS version (OL5, OL6,...) we obviously fix bugs or address a security issue. These fixes are introduced in a continous stream of updates for each major OS. They always update the minor digits of the package version, for instance the kernel, 2.6.39-300.0.1 to 2.6.39-300.0.2 etc... They do not introduce behavior change or impact how an application runs, they make things more stable and secure. We try to not do this more often than necessary. It would be highly recommended to apply these updates soon after they are released. Most critically the kernel and glibc ones as they are under every application. Of course, with ksplice updates we make kernel updates a breeze since there's no reboot involved. And as mentioned in -2-, these errata are on a regular basis bundled into a newer snapshot of the OS.

    What's the take-away here?

    We recommend you look at an update release as a starting point, not as a product version, and we highly recommend customers and partners to be as current as possible in applying errata packages on their OS. It makes things more stable... it contains fixes... it contains patched security vulnerabilities... those all seem rather important to keep in mind.

    So often, a customer service request comes into the support organization and it's a problem that's known and fixed in an errata, a downtime that could've been prevented by being more current...

    Wednesday Aug 07, 2013

    Oracle VM templates for Database 12c 12.1.0.1.0 both single instance and rac

    Today we made available a few new Oracle VM templates on edelivery. A set of VM templates for database 12c and another set for database 11g 11.2.0.3.7.

    You can find more information on the otn pages here.

    A very important new feature added is the ability to deploy single instance database. In the past the database templates were focused on RAC deployments (Real Application Cluster) but because of popular demand, we also added support for Single Instance. With Single Instance you can really create a new VM with the database up and running in a matter of a few (very few) minutes, and with a very simple config file.

    Example config file for single instance :

    $ cat netconfig.ini 
    NODE1=dbsingle1
    NODE1IP=192.168.1.72
    PUBADAP=eth0
    PUBMASK=255.255.255.0
    PUBGW=192.168.1.1
    DOMAINNAME=wimmekes.net  # May be blank
    DNSIP=8.8.8.8  # Starting from 2013 Templates allows multi value
    CLONE_SINGLEINSTANCE=yes  # Setup Single Instance
    

    That's literally it. You don't need to do anything other than run a few Oracle VM CLI or UI commands and run deploycluster and you're all set. After a few minutes, the VM will be pingable and you can run sqlplus against the database running inside the VM.

    If you use the CLI, here is a sample workflow :

  • import the template
  • - importtemplate repository name=[reponame] url=[http://myurl/template.tbz] server=[servername]
  • create vm from template
  • - clone vm name=[templatename] destType=Vm destName=[vmname] serverpool=[serverpoolname]
  • Create new vnic
  • - create vnic name=[macaddress] network=[network] (list network, will show you the various networks)
  • remove old vnics (you could rename one or alter one but to simplify I just remove the old vnics of the cloned vm and add the newly created
  • - remove vnic name=[macaddr] from vm name=[vmname]
    - show vm name=[vmname] to see the attached vnics

    And that's it, now you can use that netconfig.ini example, edit it for your environment and run deploycluster:

    On top of single instance, the templates also expose or give you the ability to easily configure and enable many of the new rdbms 12c functionality :

    - Oracle Flex Cluster and/or Flex ASM, Hub/Leaf nodes
    - Container Database with x number of pluggable databases
    - Database Express
    - ACFS filesystem
    - Oracle Restart (single instance database with HA)
    - local or shared filesystem installs, including OCFS2 and ACFS
    - Admin Managed or Policy managed database creation with serverpools
    - OS kernel updated to the latest uek 2 version 2.6.39-400

    And all of the above are simple parameters in the config files. This can be 100% automated, 100% reproducible and you don't need to know how to configure them all yourself. As always, high quality work by Saar Maoz.

    Production ready, not trial, not using a random OS, all ready to go. Production-ready virtual appliances.

    About

    Wim Coekaerts is the Senior Vice President of Linux and Virtualization Engineering for Oracle. He is responsible for Oracle's complete desktop to data center virtualization product line and the Oracle Linux support program.

    You can follow him on Twitter at @wimcoekaerts

    Search

    Categories
    • Oracle
    Archives
    « August 2013 »
    SunMonTueWedThuFriSat
        
    1
    2
    3
    4
    5
    6
    8
    9
    10
    11
    12
    14
    15
    16
    17
    18
    19
    20
    21
    23
    24
    25
    26
    28
    29
    30
    31
           
    Today