Tuesday Aug 23, 2011

Moving Address

Managing IP Addresses with Node Manager

Moving house and changing address is always a problem, Auntie Matilda and the Mega Credit card company continue to send letters to the old address for years, which are dutifully forwarded by the new occupants.  Every few months the dear folks at the Bristol England Oracle office start to feel guilty about the amount of mail addressed to me, so they stick it in a FedEx envelope and send it out to the Colorado Springs Oracle office, where I open it and throw it all in the recycling bin.  So it is with some relief that I can reveal how easy it is to have node manager take care of all moving address requirements for your managed WebLogic servers.

My colleague James Bayer pointed out to me last week that there have been some enhancements to Node Manager in the way it handled IP addresses.

Justification

Some WebLogic managed servers need to be able to move from one machine to another, and when they move, so that everyone else can find them, we need to move their IP listening addresses at the same time.  This “Whole Server Migration”, sometimes referred to as “WSM” to deliberately cause confusion with “Web Services Manager”, can occur for a number of reasons:

  • Allow recovery of XA transactions by the transaction manager in that WebLogic instance
  • Allow recovery of messages in a JMS Queue managed by a JMS server in that WebLogic instance
  • Allow migration of singleton services, for instance the BAM server in SOA Suite

Early History

We used to enable whole server migration in the following way

  • Grant sudo privileges to the “oracle” user on the ifconfig and arping commands (which are used by the wlsifcfg script called by Node Manager) by adding the following to the /etc/sudoers file:
    • oracle ALL=NOPASSWD: /sbin/ifconfig, /sbin/arping
  • Tell Node Manager which interface (in the example below the eth1 network interface) it is managing and what the netmask is for that interface (in our example an 8-bit sub-net) by adding the following to the nodemanager.properties file:
    • Interface=eth1
    • NetMask=255.255.255.0
    • UseMACBroadcast=true
  • Identify target machines for the cluster in WebLogic console Environment->Clusters->cluster_name->Migration
  • Identify a target machine for the managed server in WebLogic console Environment->Servers->server_name
  • Identify a failover machine or machines for the managed server in WebLogic console Environment->Servers->server_name->Migration

Once configured like this when a managed server is started the Node Manager will first check that the interface is not in use and then bring up the IP address on the given interface before starting the Managed Server, the IP address is brought up on a sub-interface of the given interface such as eth0:1.  Similarly when the Managed Server is shutdown or fails the Node Manager will release the IP address if it allocated it.  When failover occurs the Node Manager again checks for IP usage before starting the managed server on the failover machine.

A Problem

The problem with this approach is what happens when you have a managed server listening on more than one address!  We can only provide one Interface and even if multiple interface properties were allowed (they are not) we would not know which NetMask to apply.

So for many years we have labored under the inability to have a server support both multiple listening addresses (channels in WebLogic parlance) and whole server migration – until now.

Solution

The latest Node Manager comes with a different syntax for managing IP addresses.  Instead of an Interface and NetMask property we now have a new property corresponding to the name of an interface on our computer.  For this interface we identify the range of IP addresses we manage on that interface and the NetMask associated with that address range.  This allows us to have multiple listening addresses in our managed server listening on different interfaces.  An example of adding support for multiple listening addresses on two interfaces (bond0 and eth0) is shown below:

bond0=10.1.1.128-10.1.1.254,NetMask=255.255.255.0

etho=10.2.3.10-10.2.3.127,NetMask=255.255.240.0

UseMACBroadcast=true

So now when a server has multiple listen channels then the Node Manager will check what address range each listen channel falls into and start the appropriate interface.

A Practical Application

A practical example of where this is useful is when we have an ExaLogic machine with an external Web Server or load balancer.  We want the managed server to talk to each other using the internal Infiband network, but we want to access the managed servers externally using the 10GigaBit ethernet network.  With the enhancements to the Node Manager we can do just this.

Acknowledgements

Thanks to James Bayer, Will Howery and Teju for making me aware of this functionality on a recent ExaLogic training course.  Thanks to them my managed servers can now move multiple addresses without the constant forwarding of mail that I get from the Bristol office.  Now if they can just get the Bristol office to stop forwarding my mail…

ExaLogic Documentation on Server Migration

About

Musings on Fusion Middleware and SOA Picture of Antony Antony works with customers across the US and Canada in implementing SOA and other Fusion Middleware solutions. Antony is the co-author of the SOA Suite 11g Developers Cookbook, the SOA Suite 11g Developers Guide and the SOA Suite Developers Guide.

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today