Friday Mar 15, 2013

Exalogic Virtual - Creating a vServer using LVM2 for the root file-system

For a customer Proof of Concept - demonstrating the benefits of Oracle Fusion Middleware on Exalogic Virtual - we decided to build a custom base template. All vServers created by using this custom base template enable some customer specific configuration settings, activation of additional services and yields flexibility for the file-system layout.

We decided to use Logical Volume Management LVM2 to achieve the goal of file-system flexibility. This can easily be implemented, as the Base vServer Template, delivered by Oracle, already has all the necessary rpm's installed. The Default Gemini and Navstar vServer Templates come with a fixed partition and file-system layout. To keep a simple layout we went for one logical volume for the root file-system and another volume for swap. By using an ext3 file-system on top of the logical volume for root we are now able to re-size it as needed during normal operation. So whenever logs are filling up /var, or pre-req checks fail due to a lack of space in /tmp we can easily correct this.
In this first blog I will describe how to build a vServer using LVM2. As I said before, for the PoC we planned to use this vServer as a new custom base template. In a second blog post I will show the steps needed to create a vServer template out of this vServer.
This is very similar to what my colleague Andrew Hopkinson has already blogged about here, but needs a few extra steps because of the LVM2 usage.

[Read More]

Tuesday Feb 26, 2013

Solaris on Exalogic - Transitive Probe-based Failure Detection

On Exalogic, no matter which supported Operating Systems that a compute node is running, it relies on the IB gateways (NM2-GW) to provide both internal (IPoIB) and external (EoIB) network connectivity. Each compute node is physically connected to two IB gateways by copper cables, the IB gateway in turn is connected to customer's 10GbE infrastructure, typically a level 2 switch.

By default, only link-based failure detection for IPMP group is enabled on compute node running Solaris. This default setting remains the same even if a compute node has been upgraded from Solaris 11 Express to Solaris 11.1 on X2-2 hardware or on X3-2 hardware where Solaris 11.1 can be installed directly.

The limitation of link-based failure detection is that it cannot detect failure over the link between IB gateway and customer's infrastructure, that means even if that link goes down, bond1 will not fail over and therefore 10GbE connectivity to the compute node is lost.

In fact, there exists a scenario where even the link between compute node and IB gateway failed, bond1 will not fail over but that's a topic for another blog entry.

For customers running Solaris 11 Express, the solution is enable Probe-based Failure Detection, the downside of this solution is we need 2 additional IP addresses for each IPMP group. It could be a challenge for customers running tight on IP addresses.

On Solaris 11.1, we have a better solution called Transitive Probe-based Failure Detection and it does not require additional IP addresses to be assigned to the IPMP group members.

To enable Transitive Probe-based Failure Detection, run the following commands on a compute node:

#svccfg -s svc:/network/ipmp setprop config/transitive-probing=true
#svcadm refresh svc:/network/ipmp:default

If default gateway is already configured for bond1, it will be used as the target system, otherwise you will need to create a host route to a particular system that you would like to probe.

To check if Transitive  Probe-based Failure Detection is working, run the following command:

root@el01cn01:~# ipmpstat -t
INTERFACE   MODE       TESTADDR            TARGETS
eoib1       transitive <eoib1>             <eoib0>
eoib0       routes     el01cn01-pub   192.168.123.254
bond0_1     transitive <bond0_1>           <bond0_0>
bond0_0     multicast  el01cn01-priv  el01cn05-priv el01cn04-priv el01cn02-priv el01cn06-priv el01sn-priv

See the Solaris official documentation here on how to specify a target system for probe-based failure detection.



Solaris on Exalogic - Reverse the active/pasive interface of an IPMP group

In a customer engagement, I found that that their bond0 and bond1 configurations look like this:

root@el01cn01:~# ipmpstat -i
INTERFACE   ACTIVE  GROUP       FLAGS     LINK      PROBE     STATE
eoib0       no      bond1       is-----   up        disabled  ok
eoib1       yes     bond1       --mb---   up        disabled  ok
bond0_0     yes     bond0       --mb---   up        disabled  ok
bond0_1     no      bond0       is-----   up        disabled  ok

notice that the active interface for bond1 is eoib1 while the active interface for bond0 is bond0_0

Although it is perfectly fine for the EoIB traffic and IPoIB traffic to go over different IB gateway, customer want like to re-configure it so that both type of traffic will go through the same IB gateway.

First of all, we turn on "standby" property for eoib1 and turn off "standby" property for eoib0 with the following commands:

root@el01cn01:~# ipadm set-ifprop -p standby=on -m ip eoib1
root@el01cn01:~# ipadm set-ifprop -p standby=off -m ip eoib0

The status of the IPMP groups look like this now:

root@el01cn01:~# ipmpstat -i
INTERFACE   ACTIVE  GROUP       FLAGS     LINK      PROBE     STATE
eoib0       yes     bond1       -------   up        disabled  ok
eoib1       yes     bond1       -smb---   up        disabled  ok
bond0_0     yes     bond0       --mb---   up        disabled  ok
bond0_1     no      bond0       is-----   up        disabled  ok

Then we forced a failover by detaching eoib1 from bond1 with the following command:

root@el01cn01:~# if_mpadm -d eoib1

The status of the IPMP groups look like this now:

root@el01cn01:~# ipmpstat -i
INTERFACE   ACTIVE  GROUP       FLAGS     LINK      PROBE     STATE
eoib0       yes     bond1       --mb---   up        disabled  ok
eoib1       no      bond1       -s---d-   up        disabled  offline
bond0_0     yes     bond0       --mb---   up        disabled  ok
bond0_1     no      bond0       is-----   up        disabled  ok

Then we re-attach eoib1 back to bond1, because it is a standby interface, failback will not happen:

root@el01cn01:~# if_mpadm -r eoib1

That's how it looks like after the re-configuration:

root@el01cn01:~# ipmpstat -i
INTERFACE   ACTIVE  GROUP       FLAGS     LINK      PROBE     STATE
eoib0       yes     bond1       --mb---   up        disabled  ok
eoib1       no      bond1       is-----   up        disabled  ok
bond0_0     yes     bond0       --mb---   up        disabled  ok
bond0_1     no      bond0       is-----   up        disabled  ok

Exalogic Virtual Tea Break Snippets - Wrapping the Exalogic iaas cli

Having worked with the Exalogic Command Line for a while I decided to wrap some of the common functions in a simplified bash script. This saves me creating the keys, connecting and identifying the appropriate Ids. Instead I can simply specify the Name and the script will do the rest of the work. This initial version has just a few commands in it but as I add more the blog entry will expand, as will the script, and document the new functionality.

[Read More]

Exalogic Virtual Tea Break Snippets - Cloning an existing vServer

Following on from my Blog entry "Scripted Template Generation from an existing vServer" I have built a wrapper script that can be used to execute the Template Generation script or Clone a specific vServer. This script was not incorporated into the original script because it must be executed on a Compute Node with access to the /OVS/Repositories directory and the Compute Node are a minimal install and hence do not have all the required software available. As part of the Cloning process this new script will create an Assets input file that can be used with the CreateAssets.sh describe in the blog "Scripting Asset Creation" and optionally execute the result to create the clone.

To successfully run the script you will need the following 3 scripts located in the same directory on a machine with the EMOC cli/api rms installed.

[Read More]

Friday Feb 22, 2013

Exalogic Virtual Tea Break Snippets - Scripted Template Generation from an existing vServer

Note: This has been superseded  by the Script described in the "Exalogic Virtual Tea Break Snippets – Simplified Exalogic IaaS Cli".

As part of your Exalogic Virtual environment you may want to build vServer that will be used, going forwards, as a template for future vServers. Currently the "Exalogic Elastic Cloud Administrator's Guide" has an appendix describing how this can be achieved using the OVMM interface. Based on internal A-Team work it is now possible to achieve this directly from  a compute nodes command-line without accessing OVMM.

As a result of this I have built the script below that will take the files associated with a "Stopped" vServer and converts them to a template.

For this templating process to work the script will need to be executed on a machine with access to the /OVS/Repositories/* directories and this means running directly on one of the Compute Nodes (I generally run it on Compute Node 1).

Because of the space and resource limitations of the Compute Node (minimal OS) we will need to create a and mount a Share from the internal ZFS to save the working files and ultimately the Template. To this end the script will take a number of parameters that will specification of these directories. If these are not specified the script assumes we have the ZFS /export/common/images mounted on /u01/common/images.

As can been seen from the Usage section below the script only mandates the Name of the vServer to be copied but assumes that the user has stopped the vServer previously. Once the template has been created, or post copy, the vServer can be restarted.

[Read More]

Wednesday Jan 23, 2013

Exalogic 2.0.1 Tea Break Snippets - Some Simple ZFS Scripts

Whilst working on an Exalogic Upgrade I was working with the ZFS storage and having executed the same commands a number of times I decided to script them. This short blog entry, although it will grow over time, contains the scripts I find useful. For each of the scripts I will simply provide a brief description and the source of the script and occasionally add the output assuming it is not too long. Where I need to pass the Hostname / IP Address of the storage heads the scripts will use the flags (unless otherwise specified):

  • -p <Primary - first storage head>
  • -s <Secondary - Second storage head>

I will be using a combinations of simple bash scripts and the more function ZFS scripting language.

[Read More]

Thursday Dec 06, 2012

Exalogic 2.0.1 Tea Break Snippets - Creating a ModifyJeOS VirtualBox

Following on from my previous blog entry "Modifying the Base Template" I decided to put together a quick blog to show how to create a small VirtualBox, guest, that can be used to execute the ModifyJeOS and hence edit you templates. One of the main advantages of this is that Templates can be created away from the Exalogic Environment. For the Guest OS I chose Oracle Linux 6u3 and decided to create it as a basic server because I did not require a graphical interface but it's a simple change to create it with a GUI.

Required Software

[Read More]

Monday Dec 03, 2012

Exalogic 2.0.1 Tea Break Snippets - Creating and using Distribution Groups

By default running your Exalogic in a Virtual provides you with, what to Cloud Users, is a single large resource and they can just create vServers and not care about how they are laid down on the the underlying infrastructure. All the Cloud Users will know is that they can create vServers. For example if we have a Quarter Rack (8 Nodes) and our Cloud User creates 8 vServers those 8 vServers may run on 8 distinct nodes or may all run on the same node.

Although in many cases we, as Cloud Users, may not be to worried how the Virtualisation Algorithm decides where to place our vServers there are cases where it is extremely important that vServers run on distinct physical compute nodes. For example if we have a Weblogic Cluster we will want the Servers with in the cluster to run on distinct physical node to cover for the situation where one physical node is lost.

To achieve this the Exalogic Virtualised implementation provides Distribution Groups that define and anti-aliasing policy that the underlying Virtualisation Algorithm will take into account when placing vServers.

It should be noted that Distribution Groups must be created before you create vServers because a vServer can only be added to a Distribution Group at creation time.

[Read More]

Wednesday Nov 21, 2012

Exalogic 2.0.1 Tea Break Snippets - Modifying the Default Shipped Template

Having installed your Exalogic Virtual environment by default you have a single template which can be used to create your vServers. Although this template is suitable for creating simple test or development vServers it is recommended that you look at creating your own custom vServers that match the environment you wish to build and deploy. Therefore this Tea Time Snippet will take you through the simple process of modifying the standard template.

[Read More]
About

The primary contributors to this blog are comprised of the Exalogic and Cloud Application Foundation contingent of Oracle's Fusion Middleware Architecture Team, fondly known as the A-Team. As part of the Oracle development organization, The A-Team supports some of Oracle's largest and most strategic customers worldwide. Our mission is to provide deep technical expertise to support various Oracle field organizations and customers deploying Oracle Fusion Middleware related products. And to collect real world feedback to continuously improve the products we support. In this blog, our experts and guest experts will focus on Exalogic, WebLogic, Coherence, Tuxedo/mainframe migration, Enterprise Manager and JDK/JRockIT performance tuning. It is our way to share some of our experiences with Oracle community. We hope our followers took away something of value from our experiences. Thank you for visiting and please come back soon.

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today