Monday Sep 02, 2013

Oracle Linux 6 UEK3 beta

Last week we published UEK3 beta on http://public-yum.oracle.com.

It is very easy to get started with this and play around with the new features. Just takes a few steps :

  • Install Oracle Linux 6 (preferrably the latest update) on a system or in a VM
  • Add the beta repository file in /etc/yum.repos.d
  • Enable the beta channel
  • Reboot into the new kernel
  • Add updated packages like lxc tools and dtrace
  • Oracle Linux is freely downloadable from http://edelivery.oracle.com/linux. Oracle Linux is free to use on as many systems as you want, is freely re-distributable without changing the CD/ISO content (so including our cute penguin), provides free security errata and bugfix errata updates. You only need to pay for a support subscription for those systems that you want/need support for, not for other systems. This allows our customers/users to run the exact same software on test and dev systems as well as production systems without having to maintain potentially two kinds of repositories. All systems can run the exact same software all the time.

    The free yum repository for security and bugfix errata is at http://public-yum.oracle.com. This site also contains a few other repositories :

  • Playground channel (a yum repository where we publish the latest kernels as released on kernel.org. We take the mainline tree and build it into RPMs that can easily be installed on Oracle Linux (Oracle Linux 6 and x86_64 specifically).
  • Beta channel (a yum repository where we publish new early versions of UEK along with corresponding packages that need to be updated along with it.
  • Now, back to UEK3 beta. Just a few steps are needed to get started.

    I will assume you have already installed Oracle Linux 6 (update 4) on a system and it is configured to use public-yum as the repository.

    First download and enable the beta repository.

    # cd /etc/yum.repos.d/
    
    # wget http://public-yum.oracle.com/beta/public-yum-ol6-beta.repo
    
    # sed -i s/enabled=0/enabled=1/g public-yum-ol6-beta.repo 
    

    You don't have to do sed you can just edit (vi/emacs) the repo file and manually set it to 1 (enable). Now you can just run yum update

    # yum update
    

    This will install UEK3 (3.8.13-13) and it will update any relevant packages that are required to be on a later version as well. At this point you should reboot into UEK3.

    New features introduced in UEK3 are listed in our release notes. There are tons of detailed improvements in the kernel since UEK2 (3.0 based). Kernelnewbies is an awesome site that keeps a nice list of changes for each version. We will add more detail to our release notes over time but for those that want to browse through all the changes, check it out.

  • http://kernelnewbies.org/Linux_3.1
  • http://kernelnewbies.org/Linux_3.2
  • http://kernelnewbies.org/Linux_3.3
  • http://kernelnewbies.org/Linux_3.4
  • http://kernelnewbies.org/Linux_3.5
  • http://kernelnewbies.org/Linux_3.6
  • http://kernelnewbies.org/Linux_3.7
  • http://kernelnewbies.org/Linux_3.8
  • To try out dtrace, you need to install the dtrace packages. We introduced USDT in UEK3's version of dtrace, there is some information in the release notes about the changes.

    # yum install dtrace-utils
    

    To try out lxc, you need to install the lxc packages. lxc is capable of using Oracle VM Oracle Linux templates as a base image to create a container.

    # yum install lxc
    

    Enjoy.

    Tuesday Nov 27, 2012

    Introducing the Oracle Linux Playground yum repo

    We just introduced a new yum repository/channel on http://public-yum.oracle.com called the playground channel. What we started doing is the following:

    When a new stable mainline kernel is released by Linus or GregKH, we internally build RPMs to test it and do some QA work around it to keep track of what's going on with the latest development kernels. It helps us understand how performance moves up or down and if there are issues, we try to help look into them and of course send that stuff back upstream. Many Linux users out there are interested in trying out the latest features but there are some potential barriers to do this.

    (1) in general, you are looking at an upstream development distribution, which means that everything changes both in userspace(random applications) and kernel. Projects like Fedora are very useful and someone that wants to just see how the entire distribution evolves with all the changes, this is a great way to be current. A drawback here, though, is that if you have applications that are not part of the distribution, there's a lot of manual work involved or they might just not work because the changes are too drastic. The introduction of systemd is a good example.

    (2) when you look at many of our customers, that are interested in our database products or applications, the starting point of having a supported/certified userspace/distribution, like Oracle Linux, is a much easier way to get your feet wet in seeing what new/future Linux kernel enhancements could do.

    This is where the playground channel comes into play. When you install Oracle Linux 6 (which anyone can download and use from http://edelivery.oracle.com/linux), grab the latest public yum repository file http://public-yum.oracle.com/public-yum-ol6.repo, put it in /etc/yum.repos.d and enable the playground repo :

    [ol6_playground_latest]
    name=Latest mainline stable kernel for Oracle Linux 6 ($basearch) - Unsupported 
    baseurl=http://public-yum.oracle.com/repo/OracleLinux/OL6/playground/latest/$basearch/
    gpgkey=http://public-yum.oracle.com/RPM-GPG-KEY-oracle-ol6
    gpgcheck=1
    enabled=1
    
    Now, all you need to do : type yum update and you will be downloading the latest stable kernel which will install cleanly on Oracle Linux 6. Thus you end up with a stable Linux distribution where you can install all your software, and then download the latest stable kernel (at time of writing this is 3.6.7) without having to recompile a kernel, without having to jump through hoops.

    There is of course a big, very important disclaimer this is NOT for PRODUCTION use.

    We want to try and help make it easy for people that are interested, from a user perspective, where the Linux kernel is going and make it easy to install and use it and play around with new features. Without having to learn how to compile a kernel and without necessarily having to install a complete new distribution with all the changes top to bottom.

    So we don't or won't introduce any new userspace changes, this project really is around making it easy to try out the latest upstream Linux kernels in a very easy way on an environment that's stable and you can keep current, since all the latest errata for Oracle Linux 6 are published on the public yum repo as well. So one repository location for all your current changes and the upstream kernels. We hope that this will get more users to try out the latest kernel and report their findings. We are always interested in understanding stability and performance characteristics.

    As new features are going into the mainline kernel, that could potentially be interesting or useful for various products, we will try to point them out on our blogs and give an example on how something can be used so you can try it out for yourselves.

    Anyway, I hope people will find this useful and that it will help increase interested in upstream development beyond reading lkml by some of the more non-kernel-developer types.

    Sunday May 20, 2012

    Oracle Linux components

    A few weeks ago I was visiting a customer where we were discussing Oracle Linux and Oracle VM advantages compared to some of our competitors. One of the comments from the customer at some point in the discussion was "And you get -all that- for just -$that-?"... Why, yes, you do, sir. I wanted to take the opportunity to list some of the included advantages of having Oracle Linux support. Next time I will do the same for Oracle VM.

    Oracle Linux is a complete enterprise Linux solution available from Oracle. In fact, it is, we firmly believe, the best enterprise Linux solution available. Both for generic use of Linux and in particular and especially when using any Oracle product on top of it. So it is important to remember and understand that Oracle Linux is not just for running Oracle products on top of Linux. Oracle Linux is a complete generic Linux distribution that's fully supported underneath any Linux application. There are no restrictions from our end as to what you can run on it (as long as they're apps built to run on Linux, of course and any application that runs on Red Hat Enterprise Linux will run the same on the corresponding Oracle Linux version). You can run 3rd party Linux applications just fine. Use it as an NFS server, as a webserver or whatever you want to use the system for. Many customers I talk to somehow believe that it's unique for Oracle products and I have to point out that while we have great, unique advantages and extra benefits running Oracle products on Oracle Linux, it's certainly not in anyway restricted for that. It's the opposite, all you needs is Oracle Linux for any and every Linux server in house.

    Why the best, you ask? Well, we do an incredibly amount of testing on a daily basis (1000's of servers run various types of QA on Oracle Linux amounting to over 100,000 hours of QA happening every single day), we have a huge amount of internal use as consumers (end-user/customer use), we have many years of experience helping enterprise customers in general and we extend that down to Operating System support. We can scale using the greater Oracle infrastructure, etc.

    The support team is a large group of dedicated professionals globally that are trained to handle serious issues, deal with mission critical environments and cases, know their product, have a focus on server use first and foremost, understand, in particular, also the other products around it enough (like, to be able to understand that ASM doesn't stand for -assembler language- (in our customer-base) but stands for automatic storage management (you laugh? I can assure you it's not a joke)) A single phone call: if a database problem is an OS issue, no worries, we deal with it internally, no need to go call someone else that might think a database is a dirty word, or someone that 's an expert in gnome (no offense) but not quite in a complex environment where you have a product use 50GB of RAM and has 2000 processes running and is doing massive amounts of IO and woops something is going wrong. "Reboot?" doesn't cut it.

    When we do Linux development, the focus is basically on two things. 1) find ways to enhance Linux to run our products better (in ways that are generic, and also help other applications). A database does disk/network IO, lots of processes, lots of memory access, if we make our product run better, others also benefit... 2) help make Linux a better OS. improve stability, help add features that an OS needs, unrelated to our own product stack but important for customers to run the operating system, no matter what application. Having access to in-house hardware such as very big x64 servers like the x4800 (4TB ram, 160 threads) and TB and TB of storage allows us to test things at very, very large scale. This allows us to publish benchmark results like the 5M tpc-c result a few months ago. Offering the Unbreakable Enterprise Kernel(UEK) as part of Oracle Linux alongside what we call the Red Hat-Compatible kernel gives us the ability to provide current, modern, tested code to customers without reducing reliable and availability.

    A CTO of another company a little while back implied on twitter that UEK is too current and reduces reliability because the code is new. Can I point something out here? One thing we decided to do, with UEK, was to not backport features but instead just focus on stability. The goal is to stay close (but not too close as we don't ship a development tree) to the "Linus kernel" and focus on testing the heck out of it and fixing any bug that we would find, any regression.. This is good for the Linux community because when we do find a problem, we fix it and it's immediately also relevant to the mainline kernel. This helps us and the kernel community. We spend time testing and fixing, we develop on mainline and we don't waste time backporting new stuff into old versions. It's a symbiotic relationship. The alternative : take a version of Linux that's 3+ years old and take code from a more or less current version, after 100's of 1000's of lines of code changes and architectural changes have happened, and make that new code, somehow work in the old tree. The result : old patched up, totally unique code. Someone spending all their time retrofitting complex features into an old version. Testing is unique, bugs found are mostly only relevant in that version and not so much in newer versions unless they're greater design bugs or one liners. No one else really ever tested it because, well, it's unique, no other company is providing that seriously forked kernel that has 1+M of lines of differences. Somehow, I prefer our approach. At least the folks at SuSE seem to believe the same.

    One advantage we have with our kernel is that we made it possible to install in an existing Oracle Linux 5 environment and you are not required to move to Oracle Linux 6 to get the benefit of new kernel features and new drivers and new performance benefits. Moving from one major version to the next is a very costly effort because you basically always end up doing a re-imaging of the servers. Most customers are driven to new major releases for a hardware refresh. We don't require you to do that, you can run both OL5 and OL6, your choice, your timeframe. You can reboot between 2.6.18 and 2.6.32 or even 2.6.39 (3.0.24) without problem, at any time.

    The complete source code for Oracle Linux is available from oss.oracle.com (OL5 or OL6). The source code for our UEK and UEK2 kernels is available in a public GIT repository (UEK 2.6.32 or UEK2 2.6.39. Don't worry, it's not just a tar file and you don't have to go browse webpages behind a userid/password to figure out what all changed and try to piece it together yourself. It's all there, both the mainline kernel changelog and our changes on top. Transparent.

    The binaries for Oracle Linux are available from edelivery in the form of ISO images of the installable DVDs for the major releases and all the update releases (OL5.1...5.8, OL6, OL6.1, OL6.2 etc). You are not required to first get a support subscription, you can just go and download the software, no activation keys, no trial period, production use OK, distribute to your friends? no problem, no need to remove logos. Errata/bug updates are available through ULN (ULN is our Oracle Linux customer portal where we have extra channels with extra software bits for subscribers, or from public yum (public yum is an open, free, no registration, no subscription site). All Oracle Linux errata/packages are published on both sites at the same time. If you have a set of servers where support is not needed, just a standard Oracle Linux, and then a set of servers where you do need support because they're production systems etc, you can install OL on both and you subscribe for the systems you need to have support for. One distribution installed across the entire company. No need to find a free distribution and a pay to use distribution. If you no longer need support services for a system, don't worry, we won't ask you to de-install.

    But wait, there's more..

    How about managing Oracle Linux... well, included in Oracle Linux support subscriptions, is the use of Oracle Enterprise Manager 12c and Oracle Enterprise Manager OpsCenter 12. This is management software that can handle 1000's of servers from a single managed web-based interface. It offers features like :

  • bare-metal provisioning of Oracle Linux servers
  • configuration management, collecting and analysis and auditing
  • lights-out or ad hoc patching
  • availability and performance monitoring
  • event management and job systems
  • yum repositories
  • grouping of servers, custom jobs, custom action/corrective action scripts
  • this is all included, for no additional fee. Any server with Oracle Linux support subscriptions include the above. No extra pack, no per server additional fees. Or if you don't really care for that and just want a very simple yum based setup, you can do that as well, just grab the rpms, put them on a server, create a repository and have your servers connect to that. So, very basic and simple, to very complete and comprehensive. Same cost.

    And there is even more...

    Oracle Real Application Clusters makes use of the Oracle Clusterware(CRS). CRS is cluster software that is capable of protecting any kind of application in a failover cluster. Not just Oracle. It is the basis for any clustered environment and provides generic availability eliminating the need for other third party cluster solutions. You can protect your own application using this enterprise grade clustering stack. Without additional fees. It's included in Oracle Linux support subscriptions (basic/premier).. have your own nfs server? webserver? want to monitor it and restart it in a cluster no problem? Go for it. Using the exact same software we use for the most mission critical clustered systems. Want to run Oracle Linux virtualized? 1 guest? 10 guests? 20 guests? You only need one subscription per physical server and we don't charge more if you run 1 or more than 1 guest. And how much for all this?

  • Oracle x86 based systems with Premier support for Systems includes all this, both the Oracle Linux subscription and the use of the management features and Oracle CRS
  • Non-Oracle x86 based systems : 2 socket server $499 for all of the above or 4+ socket server $1199 includes all this, both the Oracle Linux subscription and the use of the management features and Oracle CRS
  • This is 24/7 support, not just office hours. Complete 24/7 support and real indemnification not based on subscription fees paid.

    We also have a premier support subscription which adds the ability to use Oracle ksplice technology to apply your kernel security fixes online, while the system is running at full speed, while the applications are running, while the users are working, without downtime. And I mean apply, not just install. The installed fixes are instantly effective at runtime. zero reboots. zero downtime scheduling. zero administrative program/planning management.. Or premier backport support, critical fixes for a specific bug in the specific version of the rpm you run in a mission critical environment. Or lifetime support, now with virtualization more useful than ever before. Virtual hardware can remain the same for much longer than the support lifecycles.

  • Oracle x86 based systems with Premier support for Systems includes all this, both the Oracle Linux subscription and the use of the management features and Oracle ksplice and premier backports and lifetime support
  • Non-Oracle x86 based systems : 2 socket server $1399 for all of the above or 4+ socket server $2299 includes all this, both the Oracle Linux subscription and the use of the management features and Oracle CRS and Oracle ksplice and premier backports and lifetime support
  • See our store for pricing, and our Oracle Linux site for policies and references to the above.

    Ok there you have it. Almost a novel but we just have that much to offer, and wait 'til I get to Oracle VM.. it just gets even better.

    Sunday Oct 09, 2011

    trying uek2

    Another Oracle Openworld revelation was the availability of version 2 of the Unbreakable Enterprise Kernel, in short uek2. Let me put together a few notes on this as well.

    UEK2 is based on Linux 2.6.39 with applied changesets (not backports) up to Linux 3.0.3. The change from 2.6 to 3.0, by Linus, was pretty much arbitrary this time around. There were no major changes in design that went into the kernel post 2.6.39. However, versions changes like that do have an effect on userspace programs, so in order to maintain our true compatibility and refrain from code changes, sticking to 2.6.39 was the best solution.

    UEK2 beta is freely available to everyone. All you need is Oracle Linux 6 installed (also freely available from edelivery) and connect to the Unbreakable Linux Network (ULN) if you are an existing customer or if you just want to play with cool new stuff, use our public yum repository.

    The source code for our git repo is also publicly available on oss.oracle.com. http://oss.oracle.com/git/?p=linux-2.6-unbreakable-beta.git;a=summary. Indeed, not just a tarball of junk but the entire changelog, all the checkins, all the history, nicely linked up with the original git repo from kernel.org.

    There is a huge list of changes in the mainline kernel since 2.6.32, too long to sum up here. Some of the things we discussed as new features are the following :

    - btrfs production.
    btrfs is an awesome filesystem which is going through rigorous testing in order to be able to announce it production quality with uek2. The feature list of the filesystem is great, you can read some of my earlier blog entries to get a feel of it. Use it for root filesystem and do snapshots while upgrading packages is definitely an interesting enhancement to deliver.The existing Oracle Linux 6 btrfs progs work just fine but the UEK2 channel will soon include an updated version of the btrfs userspace programs to give you full access to all features in uek2.

    - Linux containers
    This is not a port of Solaris Zones. Linux containers is based on lxc and all the cgroups work that has gone into the kernel over the last several years. All the kernel features required to handle container functionality is in uek2 and as with btrfs progs, we are going to update the beta channel with the lxc tools in userspace to enable easy creation of Oracle Linux containers. Oracle VM templates will be easily converted to running as a container as well as a VM in Oracle VM.

    - OpenVswitch
    Openvswitch is a very interesting virtual switch product that can help a lot in particular in a virtual world but is a good Linux bridge replacement. These packages will also be made available with uek2 in the beta channel. More information about openvswitch can be found here.

    - many new enhancements including even better performance than uek today.
    More details will be posted in upcoming blogs with detailed data and use cases to show how this will help running Oracle software (and non-Oracle software) better than on any other Linux distribution out there.

    Getting started from ULN :

    Register your Oracle Linux 6 server with ULN using uln_register. Click on your server and Manage subscriptions and add the ol6_x86_64_UEK_BETA channel to the list of channels for this server. Then on the server just run yum update kernel-uek, reboot and you are done.

    Getting started from the public-yum repository :

    # cd /etc/yum.repos.d
    # wget http://public-yum.oracle.com/beta/public-yum-ol6-beta.repo
    
    Edit the repo file and change enabled=0 to enabled=1
    # yum update kernel-uek
    Loaded plugins: refresh-packagekit, rhnplugin
    This system is not registered with ULN.
    ULN support will be disabled.
    uek2_beta                                                |  951 B     00:00
    uek2_beta/primary                                        | 339 kB     00:01
    uek2_beta                                                                   8/8
    Setting up Update Process
    Resolving Dependencies
    --> Running transaction check
    ---> Package kernel-uek.x86_64 0:2.6.39-100.0.12.el6uek set to be updated
    --> Processing Dependency: kernel-uek-firmware >= 2.6.39-100.0.12.el6uek for pac
    kage: kernel-uek-2.6.39-100.0.12.el6uek.x86_64
    --> Running transaction check
    ---> Package kernel-uek-firmware.noarch 0:2.6.39-100.0.12.el6uek set to be updat
    ed
    --> Finished Dependency Resolution
    
    Dependencies Resolved
    
    ================================================================================
     Package                Arch      Version                    Repository    Size
    ================================================================================
    Installing:
     kernel-uek-firmware    noarch    2.6.39-100.0.12.el6uek     uek2_beta    1.1 M
         replacing  kernel-firmware.noarch 2.6.32-71.el6
         replacing  kernel-uek-firmware.noarch 2.6.32-100.28.5.el6
    Updating:
     kernel-uek             x86_64    2.6.39-100.0.12.el6uek     uek2_beta     25 M
    
    Transaction Summary
    ================================================================================
    Install       1 Package(s)
    Upgrade       1 Package(s)
    
    Total download size: 26 M
    Is this ok [y/N]:
    Downloading Packages:
    (1/2): kernel-uek-2.6.39-100.0.12.el6uek.x86_64.rpm      |  25 MB     00:51
    (2/2): kernel-uek-firmware-2.6.39-100.0.12.el6uek.noarch | 1.1 MB     00:05
    --------------------------------------------------------------------------------
    Total                                           452 kB/s |  26 MB     00:58
    Running rpm_check_debug
    Running Transaction Test
    Transaction Test Succeeded
    Running Transaction
    Warning: RPMDB altered outside of yum.
      Installing     : kernel-uek-firmware-2.6.39-100.0.12.el6uek.noarch        1/5
      Updating       : kernel-uek-2.6.39-100.0.12.el6uek.x86_64                 2/5
      Cleanup        : kernel-uek-2.6.32-100.28.5.el6.x86_64                    3/5
      Cleanup        : kernel-uek-firmware-2.6.32-100.28.5.el6.noarch           4/5
      Erasing        : kernel-firmware-2.6.32-71.el6.noarch                     5/5
    
    Installed:
      kernel-uek-firmware.noarch 0:2.6.39-100.0.12.el6uek
    
    Updated:
      kernel-uek.x86_64 0:2.6.39-100.0.12.el6uek
    
    Replaced:
      kernel-firmware.noarch 0:2.6.32-71.el6
      kernel-uek-firmware.noarch 0:2.6.32-100.28.5.el6
    
    Complete!
    

    At this point, your new kernel is installed and ready to be used. /etc/grub.conf contains an entry for this beta kernel. Just a simple reboot and you are ready to go.

    # reboot
    
    About

    Wim Coekaerts is the Senior Vice President of Linux and Virtualization Engineering for Oracle. He is responsible for Oracle's complete desktop to data center virtualization product line and the Oracle Linux support program.

    You can follow him on Twitter at @wimcoekaerts

    Search

    Categories
    Archives
    « April 2014
    SunMonTueWedThuFriSat
      
    1
    2
    3
    4
    5
    6
    7
    9
    10
    11
    12
    13
    14
    15
    16
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
       
           
    Today