Friday Nov 15, 2013

Postscript on Scripts

More Scripts for SOA Suite

Over time I have evolved my startup scripts and thought it would be a good time to share them.  They are available for download here.  I have finally converted to using WLST, which has a number of advantages.  To me the biggest advantage is that the output and log files are automatically written to a consistent location in the domain directory or node manager directory.  In addition the WLST scripts wait for the component to start and then return, this lets us string commands together without worrying about the dependencies.

The following are the key scripts (available for download here):

Script Description Pre-Reqs Stops when
Task Complete Starts Node Manager using WLST None Yes Starts Node Manager None Yes Stops Node Manager using WLST Node Manager running Yes Starts Admin Server using WLST Node Manager running Yes Starts Admin Server None No Stops Admin Server Admin Server running Yes Starts Managed Server using WLST Node Manager running Yes Starts Managed Server None No Stops Managed Server Admin Server running Yes


To start Node Manager and Admin Server ;

To start Node Manager, Admin Server and SOA Server ; ; startWlstManagedServer soa_server1

Note that the Admin server is not started until the Node Manager is running, similarly the SOA server is not started until the Admin server is running.

Node Manager Scripts

Uses WLST to start the Node Manager.  When the script completes the Node manager will be running.

The Node Manager is started in the background and the output is piped to the screen. This causes the Node Manager to continue running in the background if the terminal is closed. Log files, including a .out capturing standard output and standard error, are placed in the <WL_HOME>/common/nodemanager directory, making them easy to find. This script pipes the output of the log file to the screen and keeps doing this until terminated, Terminating the script does not terminate the Node Manager.

Uses WLST to stop the Node Manager.  When the script completes the Node manager will be stopped.

Admin Server Scripts

Uses WLST to start the Admin Server.  The Node Manager must be running before executing this command.  When the script completes the Admin Server will be running.

The Admin Server is started in the background and the output is piped to the screen. This causes the Admin Server to continue running in the background if the terminal is closed.  Log files, including the .out capturing standard output and standard error, are placed in the same location as if the server had been started by Node Manager, making them easy to find.  This script pipes the output of the log file to the screen and keeps doing this until terminated,  Terminating the script does not terminate the server.

Stops the Admin Server.  When the script completes the Admin Server will no longer be running.

Managed Server Scripts <MANAGED_SERVER_NAME>

Uses WLST to start the given Managed Server. The Node Manager must be running before executing this command. When the script completes the given Managed Server will be running. <MANAGED_SERVER_NAME>

The given Managed Server is started in the background and the output is piped to the screen. This causes the given Managed Server to continue running in the background if the terminal is closed. Log files, including the .out capturing standard output and standard error, are placed in the same location as if the server had been started by Node Manager, making them easy to find. This script pipes the output of the log file to the screen and keeps doing this until terminated, Terminating the script does not terminate the server. <MANAGED_SERVER_NAME>

Stops the given Managed Server. When the script completes the given Managed Server will no longer be running.

Utility Scripts

The following scripts are not called directly but are used by the previous scripts.

This script is used to provide information about the Node Manager and WebLogic Domain and must be edited to reflect the installed FMW environment, in particular the following values must be set:

  • DOMAIN_NAME – the WebLogic domain name.
  • NM_USERNAME – the Node Manager username.
  • NM_PASSWORD – the Node Manager password.
  • MW_HOME – the location where WebLogic and other FMW components are installed.
  • WEBLOGIC_USERNAME – the WebLogic Administrator username.
  • WEBLOGIC_PASSWORD - the WebLogic Administrator password.

The following values may also need changing:

  • ADMIN_HOSTNAME – the server where AdminServer is running.
  • ADMIN_PORT – the port number of the AdminServer.
  • DOMAIN_HOME – the location of the WebLogic domain directory, defaults to ${MW_HOME}/user_projects/domains/${DOMAIN_NAME}
  • NM_LISTEN_HOST – the Node Manager listening hostname, defaults to the hostname of the machine it is running on.
  • NM_LISTEN_PORT – the Node Manager listening port.

This script runs the WLST script passed in environment variable ${SCRIPT} and takes its configuration from  It dynamically builds a WLST properties file in the /tmp directory to pass parameters into the scripts.  The properties filename is of the form <DOMAIN_NAME>.<PID>.properties.

This script runs the command passed in as an argument, writing standard out and standard error to a log file.  The log file is rotated between invocations to avoid losing the previous log files.  The log file is then tailed and output to the screen.  This means that this script will never finish by itself.

WLST Scripts

The following WLST scripts are used by the scripts above, taking their properties from /tmp/<DOMAIN_NAME>.<PID>.properties:



The dependencies and relationships between my scripts and the built in scripts are shown in the diagram below.

Thanks for the Memory

Controlling Memory in Oracle SOA Suite

Within WebLogic you can specify the memory to be used by each managed server in the WebLogic console.  Unfortunately if you create a domain with Oracle SOA Suite it adds a new config script,, that overwrites any USER_MEM_ARGS passed in to the start scripts. only sets a single set of memory arguments that are used by all servers, so an admin server gets the same memory parameters as a BAM server.  This means some servers will have more memory than they need and others will be too tightly constrained.  This is a bad thing.

A Solution

To overcome this I wrote a small script,, that checks to see which server is being started and then sets the memory accordingly.  It supports up to 9 different server types, each identified by a prefix.  If the prefix matches then the max and min heap and max and min perm gen are set.  Settings are controlled through variables set in the script,  If using JRockit then leave the PERM settings empty and they will not be set.


The above settings match any server whose name starts Admin.  The above will result in the following USER_MEM_ARGS value

USER_MEM_ARGS=-Xms768m -Xmx1280m -XX:PermSize=256m -XX:MaxPermSize=512m

If the prefix were soa then it would match soa_server1, soa_server2 etc.

There is a set of DEFAULT_ values that allow you to default the memory settings and only set them explicitly for servers that have settings different from your default.  Note that there must still be a matching prefix, otherwise the USER_MEM_ARGS will be the ones from

This script needs to be called from the  Add the call immediately before the following lines:

if [ "${USER_MEM_ARGS}" != "" ] ; then
        export MEM_ARGS

The script can be downloaded here.

Thoughts on Memory Setting

I know conventional wisdom is that Xms (initial heap size) and Xmx (max heap size) should be set the same to avoid needing to allocate memory after startup.  However I don’t agree with this.  Setting Xms==Xmx works great if you are omniscient, I don’t claim to be omniscient, my omni is a little limited, so I prefer to set Xms to what I think the server needs, this is what I would set it to if I was setting the parameters to be Xms=Xmx.  However I like to then set Xmx to be a little higher than I think the server needs.  This allows me to get the memory requirement wrong and not have my server fail due to out of memory.  In addition I can now monitor the server and if the heap memory usage goes above my Xms setting I know that I calculated wrong and I can change the Xms setting and restart the servers at a suitable time.  Setting them not equal buys be time to fix increased memory needs, for exampl edue to change in usage patterns or new functionality, it also allows me to be less than omniscient.

ps Please don’t tell my children I am not omniscient!

Wednesday Apr 25, 2012

Scripting WebLogic Admin Server Startup

How to Script WebLogic Admin Server Startup

My first car was a 14 year old Vauxhall Viva.  It is the only one of my cars that has ever been stolen, and to this day how they stole it is a mystery to me as I could never get it to start.  I always parked it pointing down a steep hill so that I was ready to jump start it!  Of course its ability to start was dramatically improved when I replaced the carburetor butterfly valve!

Getting SOA Suite or other WebLogic based systems to start can sometimes be a problem because the default WebLogic start scripts require you to stay logged on to the computer where you started the script.  Obviously this is awkward and a better approach is to run the script in the background.  This problem can be avoided by using a WLST script to start the AdminServer but that is more work, so I never bother with it.

If you just run the startup script in the background the standard output and standard error still go to the session where you started the script, not helpful if you log off and later want to see what is happening.  So the next thing to do is to redirect standard out and standard error from the script.

Finally it would be nice to have a record of the output of the last few runs of the Admin Server, but these should be purged to avoid filling up the directory.

Doing the above three tasks is the job of the script I use to start WebLogic.  The script is shown below:

Startup Script



SCRIPT_HOME=`dirname $0`





logrotate -f -s $SCRIPT_HOME/logrotate.status $SCRIPT_HOME/AdminServerLogRotation.cfg


touch $LOG_FILE

nohup $DOMAIN_HOME/ &> $LOG_FILE &

tail -f $LOG_FILE


Lets walk through each section of the script.


The first few lines of the script just set the environment.  Note that I put the output of the start script into the same location and same filename that it would go to if I used the Node Manager to start the server.  This keeps it consistent with other servers that are started by the node manager.


The next section keeps a copy of the previous output file by using the logrotate command.  This reads its configuration from the “AdminServerLogRotation.cfg” file shown below:

/home/oracle/app/Middleware/user_projects/domains/dev_domain/servers/AdminServer/logs/AdminServer.out {
  rotate 10

This tells the logrotate command to keep 10 copies (rotate 10) of the log file and if there is no previous copy of the log file that is not an error condition (missingok).

The logrotate.status file is used by logrotate to keep track of what it has done.  It is ignored when the –f flag is used, causing the log file to be rotated every time the command is invoked.


UPDATE: Sometimes the tail command starts before the shell has created the log file for the command. To avoid an error in the tail command I "touch" the log file to make sure that it is there.

The final section actually invokes the standard command to start an admin server ( and redirects the standard out and standard error to the log file.  Note that I run the command in the background and set it to ignore the death of the parent shell.

Finally I tail the log file so that the user experience is the same as running the start command directly.  However in this case if I Ctrl-C the command only the tail will be terminated, the Admin Server will continue to run as a background process.

This approach allows me to watch the output of the AdminServer but not to shut it down if I accidently hit Ctrl-C or close the shell window.

Restart Script

I also have a restart script shown below:

SCRIPT_HOME=`dirname $0`



This is just like the start script except that it runs the stop weblogic command followed by my start script command.


The above scripts are quick and easy to put in place for the Admin Server and make the stdout and stderr logging consistent with other servers that are started from the node manager.  Now can someone help me push start my car!

Thursday Dec 22, 2011

SOA in a Windows World

Installing BPM Suite on Windows Server 2008 Domain Controller under VirtualBox

It seems I am working with a number of customers for whom Windows is an important part of their infrastructure.  Security is tied in with Windows Active Directory and many services are hosted using Windows Communication Framework.  To better understand these customers environment I got myself a Windows 2008 server license and decided to install BPM Suite on a Windows 2008 Server running as a domain controller.  This entry outlines the process I used to get it to work.

The Environment

I didn’t want to dedicate a physical server to running Windows Server so I installed it under Oracle Virtual Box.

My target environment was Windows 2008 Server with Active Directory and DNS roles.  This would give me access to the Microsoft security infrastructure and I could use this to make sure I understood how to properly integrate WebLogic and SOA Suite security with Windows security.  I wanted to run Oracle under a non-Administrator account, as this is often the way I have to operate on customer sites.  This was my first challenge.  For very good security reasons the only accounts allowed to log on to a Windows Domain controller are domain administrator accounts.  Now I only had resources (and licenses) for a single Windows server so I had to persuade Windows to let me log on with a non-Domain Admin account.

Logging On with  a non-Domain Admin Account

I found this very helpful blog entry on how to log on using a non-domain account - Allow Interactive Logon to Domain Controllers in Windows Server 2008.  The key steps from this post are as follows:

  • Create a non-Admin user – I created one called “oracle”.
  • Edit the “Default Domain Controllers” group policy to add the user “oracle” to the “Allow log on locally” policy in Computer Configuration > Policies > Windows Settings >Security Settings > Local Policies > User Rights Assignment:
  • Force a policy update.

If you didn’t get it right then you will get the following error when trying to logon "You cannot log on because the logon method you are using is not allowed on this computer".  This means that you correctly created the user but the policy has not been modified correctly.

Acquiring Software

The best way to acquire the software needed is to go to the BPM download page.  If you choose the Microsoft Windows 32bit JVM option you can get a list of all the required components and a link to download them directly from OTN.  The only download link I didn’t use was the database download because I opted for an 11.2 database rather than the XE link that is given.  The only additional software I added was the BPM feature pack (obtain from Oracle Support as patch #12413651: BPM FEATURES PACK) and the OSB software.  The BPM feature pack patch is applied with OPatch so I also downloaded the latest OPatch from Oracle support (patch 6880880 for 11.1.0.x releases on Windows 32-bit).

Installing Oracle Database

I began by setting the system environment variable ORACLE_HOSTNAME to be the hostname of the my Windows machine.  I also added this hostname to the hosts file, mapping it to  When launching the installer as a non-Administrator account you will be asked for Administrator credentials in order to install.

Gotcha with Virtual Box Paths

I mounted the install software as a VirtualBox shared Folder and told it to auto-mount.  Unfortunately this auto-mount in Windows only applied to the current user, so when the software tried to run as administrator it couldn’t find the path.  The solution to this was to launch the installer using a UNC path “\\vboxsrv\<SHARE_NAME>\<PATH_TO_INSTALL_FILES>” because the mount point is available to all users, but the auto-mapping is only done at login time for the current user.

Database Install Options

When installing the database I made the following choices to make life easier later, in particular I made sure that I had a UTF-8 character set as recommended for SOA Suite.

  • Declined Security Updates (this is not a production machine)
  • Created and Configured a Database during install
  • Chose Server Class database to get character set options later
  • Chose single instance database installation
  • Chose advanced install to get character set options later
  • Chose English as my language
  • Chose Enterprise Edition
    • Included Oracle Data Extensions for .NET
  • Set Oracle Base as C:\app\oracle
  • Selected General Purpose/Transaction Processing
  • Changed default database name
  • Configuration Options
    • Accepted default memory management settings
    • Change character set to be AL32UTF8 as required by SOA Suite
    • Unselected “Assert all new security settings” to relax security as this is not a production system
    • Chose to create sample schemas
  • Used database control for database management
  • Use file system for database storage
  • Didn’t enable automated backups (that is what Virtual Box snapshots are for)
  • Used same non-compliant password for all accounts

I set up the environment variable ORACLE_UNQNAME to be the database name, this is provided on the last screen of the Oracle database Configuration Assistant.

Configuring for Virtual Box

Because Virtual Box port forwarding settings are global I changed the DB console listen port (from 1158 using emca) and the database listener port (from 1521 using EM console) before setting up port forwarding for virtual box to the new ports.  This required me to re-register the database with the listener and to reconfigure EM.

Firewall Restrictions

After changing my ports I had a final task to do before snapshotting my image, I had add a new Windows Firewall rule to open up database ports (EM & listener).

Installing WebLogic Server

With a working database I was now able to install WebLogic Server.  I decided to do a 32-bit install to simplify the process (no need for a separate JDK install).  As this was intended to be an all in one machine (developer and server) I accepted the Coherence (needed for SOA Suite) and OEPE (needed for OSB design time tooling) options.  After installing I set the oracle user to have full access permissions on the Middleware home I created in C:\app\oracle\product\FMW.

Installing SOA/BPM Suite

Because I was using a 32-bit JVM I had to provide the “–jreLoc” option to the setup.exe command in order to run the SOA Suite installer (see release notes).  The installer correctly found my Middleware Home and installed the SOA/BPM Suite.  After installing I set the oracle user to have full access to the new SOA home created in C:\app\oracle\product\FMW\Oracle_SOA and the Oracle common directory (C:\app\oracle\product\FMW\oracle_common).

Running Repository Creation Utility

I ran the RCU from my host OS rather than from within the Windows guest OS.  This helps avoid any unnecessary temporary files being created in the virtual machine.  I selected the SOA and BPM Infrastructure component and left the prefix at the default DEV.  Using DEV makes life easier when you come to create a SOA/BPM doamin because you don’t need to change the username in the domain config wizard.  Because this isn’t a production environment I also set all the passwords to be the same, again this will simplify things in the config wizard.

Adding BPM Feature Pack

With SOA installed I updated it to include the BPM feature pack.

Installing OPatch Update

First I needed to apply patch 6880880 to get the latest OPatch.  The patch can be applied to any Oracle home and I chose to apply it to the oracle_common home, it seemed to make more sense there rather than the Oracle_SOA home.  To apply the patch I moved the original OPatch directory to OPatch.orig and then unzipped the patch in the oracle_common directory which created a new OPatch directory for me.  Before applying the feature set patch I opened a command prompt and set the ORACLE_HOME environment variable to the Oracle_SOA home and added the new OPatch directory to the path.  I then tested the new OPatch by running the command “opatch lsinventory” which showed me the SOA Suite install version.

Fixing a Path Problem

OPatch uses  setupCCR.exe which has a dependency on msvc71.dll.  Unfortunately this DLL is not on the path so by default the call to setupCCR fails with an error “This application failed to start because MSVCR71.dll was not found”.  To fix this I found a helpful blog entry that led me to create a new key in the registry at “HKEY_LOCAL_MACHINE\SOFTWARE\Microsfot\Windows\CurrentVersion\App Paths\setupCCR.exe” with the default value set to “<MW_HOME>\utils\ccr\bin\setupCCR.exe”.  I added a String value to this key with a name of “Path” and a value of “<Oracle_Common_Home>\oui\lib\win32”.  This registers the setupCCR application with Windows and adds a custom path entry for this application so that it can find the MSVCR71 DLL.

Patching oracle_common Home

I then applied the BPM feature pack patch to oracle_common by

  • Setting ORACLE_HOME environment variable to the oracle_common directory
  • Creating a temporary directory “PATCH_TOP”
  • Unzipping the following files from the patch into PATCH_TOP
  • From the PATCH_TOP directory run the command “<Oracle_Common_Home>\OPatch\opatch napply”
    • Note I didn’t provide the inventory pointer parameter (invPtrLoc) because I had a global inventory that was found just fine by OPatch and I didn’t have a local inventory as the patch readme seems to expect.
  • Deleting the PATCH_TOP directory

After successful completion of this “opatch lsinventory” showed that 3 patches had been applied to the oracle_common home.

Patching Oracle_SOA Home

I applied the BPM feature pack patch to Oracle_SOA by

  • Setting ORACLE_HOME environment variable to the Oracle_SOA directory
  • Creating a temporary directory “PATCH_TOP”
  • Unzipping the following file from the patch into PATCH_TOP
  • From the PATCH_TOP directory run the command “<Oracle_Common_Home>\OPatch\opatch napply”
    • Note again I didn’t provide the inventory pointer parameter (invPtrLoc) because I had a global inventory that was found just fine by OPatch and I didn’t have a local inventory as the patch readme seems to expect.
  • Deleting the PATCH_TOP directory

After successful completion of this “opatch lsinventory” showed that 1 patch had been applied to the Oracle_SOA home.

Updating the Database Schemas

    Having updated the software I needed to update the database schemas which I did as follows:

    • Setting ORACLE_HOME environment variable to the Oracle_SOA directory
    • Setting Java_HOME to <MW_HOME>\jdk160_24
    • Running “psa -dbType Oracle -dbConnectString  //<DBHostname>:<ListenerPort>/<DBServiceName> -dbaUserName sys –schemaUserName DEV_SOAINFRA
      • Note that again I elided the invLocPtr parameter

    Because I had not yet created a domain I didn’t have to follow the post installation steps outlined in the Post-Installation Instructions.

    Creating a BPM Development Domain

    I wanted to create a development domain.  So I ran config from <Oracle_Common_Home>\common\bin selecting the following:

    • Create a New Domain
    • Domain Sources
      • Oracle BPM Suite for developers –
        • This will give me an Admin server with BPM deployed in it.
      • Oracle Enterprise Manager –
      • Oracle Business Activity Monitoring –
        • Adds a managed BAM server.
    • I changed the domain name and set the location of the domains and applications directories to be under C:\app\oracle\MWConfig
      • This removes the domain config from the user_projects directory and keeps it separate from the installed software.
    • Chose Development Mode and Sun JDK rather than JRockit
    • Selected all Schema and set password, service name, host name and port.
      • Note when testing the test for SOA Infra will fail because it is looking for version but the BPM feature pack incremented it to  If the reason for the failure is “no rows were returned from the test SQL statement” then you can continue and select OK when warned that “The JDBC configuration test did not fully complete”.  This is covered in Oracle Support note 1386179.1.
    • Selected Optional Configuration for Administration Server and Managed Servers so that I could change the listening ports.
      • Set Admin Server Listen port to 7011 to avoid clashes with other Admin Servers in other guest OS.
      • Set bam_server Listen port to 9011 to avoid clashes with other managed servers in other guest OS.
      • Changed the name of the LocalMachine to reflect hostname of machine I was installing on.
      • Changed the node manager listen port to 5566 to avoid clashes with other Node Managers in other guest OS.

    Having created my domain I then created a file for the bam_server.

    Configuring Node Manager

    With the domain created I set up Node Manager to use start scripts by running setNMProps.cmd from <oracle_common>\common\bin.

    I then edited the <MW_Home>\wlserver_10.3\common\nodemanager\ file and added the following property:

    • ListenPort=5566

    Firewall Policy Updates

    I had to add the Admin Server, BAM Server and Node Manager ports to the Windows firewall policy to allow access to those ports from outside the Windows server.

    Set Node Manager to Start as a Windows Service

    I wanted node manager to automatically run on the machine as a Windows service so I first edited the <MW_HOME>\wlserver_10.3\server\bin\installNodeMgrSvc.cmd and changed the port to 5566.  Then I ran the command as Administrator to register the service.  The service is automatically registered for automatic startup.

    Set Admin Server to Start as a Windows Service

    I also wanted the Admin Server to run as a Windows service.  There is a blog entry about how to do this using the installSvc command but I found it much easier to use NSSM. To use this I did the following:

    • Downloaded NSSM and put the 64-bit version in my MWConfig directory.
      • Once you start using NSSM the Services you create will point to the location from which you ran NSSM so don’t move it after installing a service!
    • Created a simple script to start the admin server and redirect its standard out and standard error to a log file (I redirected to the “%DOMAIN_HOME%\servers\AdminServer\logs\AdminServer.out” because this is the location that would be used if the AdminServer were started by the node manager.

        @REM Point to Domain Directory
        set DOMAIN_HOME=C:\app\oracle\MWConfig\domains\bp_domain
        @REM Point to Admin Server logs directory
        set LOGS_DIR=%DOMAIN_HOME%\servers\AdminServer\logs
        @REM Redirect WebLogic stdout and stderr
        set JAVA_OPTIONS=-Dweblogic.Stdout="%LOGS_DIR%\AdminServer.out" -Dweblogic.Stderr="%LOGS_DIR%\AdminServer.out"
        @REM Start Admin Server
        call %DOMAIN_HOME%\startWebLogic.cmd

    • Registered the script as a Windows service using NSSM
      • nssm install “Oracle WebLogic AdminServer” “C:\app\oracle\MWConfig\startAdminServer.cmd”

    Note that when you redirect WebLogic stdout and stderr as I have done it does not get the first few lines of output, so test your script from the command line before registering it as a service.

    By default the AdminServer will be restarted if it fails, allowing you to bounce the Admin Server without having to log on to the Windows machine.

    Configuring for Virtual Box

    Having created the domain and configured Node Manager I enabled port forwarding in VirtualBox to expose the Admin Server (port 7011), BAM Server (port 9011) and the Node Manager (port 5566).

    Testing It

    All that is left is to start the node manager as a service, start the Admin server as a service, start the BAM server from the WebLogic console and make sure that things work as expected.  In this case all seemed fine.  When I shut down the machine and then restarted everything came up as expected!


    The steps above create a SOA/BPM installation running under Windows Server 2008 that is automatically started when Windows Server starts.  The log files can be accessed and read by a non-admin user so the status of the environment can be checked.  Additional managed servers can be started from the Admin console because we have node manager installed.  The database, database listener, database control, node manager and Admin Server all start up as Windows services when the server is started avoiding the need for an Administrator to start them.

    Friday Sep 23, 2011

    Oracle & JBoss Comparison

    It’s All About TCO!

    CrimsonConsultingLogoCrimson Research has just published a paper comparing total cost of ownership (TCO) of WebLogic versus JBoss.

    You can download the paper here.  Key point it makes is that acquisition of an application server platform is only a small part of the total cost of ownership over a 5 year period.  What I found surprising was the speed with which the report suggests the lower TCO of WebLogic begins to be noticable, it indicates that the break even point is about 18 months into the deployment.

    The study was sponsored by Oracle but Crimson conducted it using their own methodology and it certainly sets out a good case for the benefits of “-ility” features in bringing down the implementation costs of an applications server infrastructure.  This gels with my own experience where the more I work with operations staff the more I learn how important things like script recording and WLST are to them.

    Tuesday Aug 23, 2011

    Moving Address

    Managing IP Addresses with Node Manager

    Moving house and changing address is always a problem, Auntie Matilda and the Mega Credit card company continue to send letters to the old address for years, which are dutifully forwarded by the new occupants.  Every few months the dear folks at the Bristol England Oracle office start to feel guilty about the amount of mail addressed to me, so they stick it in a FedEx envelope and send it out to the Colorado Springs Oracle office, where I open it and throw it all in the recycling bin.  So it is with some relief that I can reveal how easy it is to have node manager take care of all moving address requirements for your managed WebLogic servers.

    My colleague James Bayer pointed out to me last week that there have been some enhancements to Node Manager in the way it handled IP addresses.


    Some WebLogic managed servers need to be able to move from one machine to another, and when they move, so that everyone else can find them, we need to move their IP listening addresses at the same time.  This “Whole Server Migration”, sometimes referred to as “WSM” to deliberately cause confusion with “Web Services Manager”, can occur for a number of reasons:

    • Allow recovery of XA transactions by the transaction manager in that WebLogic instance
    • Allow recovery of messages in a JMS Queue managed by a JMS server in that WebLogic instance
    • Allow migration of singleton services, for instance the BAM server in SOA Suite

    Early History

    We used to enable whole server migration in the following way

    • Grant sudo privileges to the “oracle” user on the ifconfig and arping commands (which are used by the wlsifcfg script called by Node Manager) by adding the following to the /etc/sudoers file:
      • oracle ALL=NOPASSWD: /sbin/ifconfig, /sbin/arping
    • Tell Node Manager which interface (in the example below the eth1 network interface) it is managing and what the netmask is for that interface (in our example an 8-bit sub-net) by adding the following to the file:
      • Interface=eth1
      • NetMask=
      • UseMACBroadcast=true
    • Identify target machines for the cluster in WebLogic console Environment->Clusters->cluster_name->Migration
    • Identify a target machine for the managed server in WebLogic console Environment->Servers->server_name
    • Identify a failover machine or machines for the managed server in WebLogic console Environment->Servers->server_name->Migration

    Once configured like this when a managed server is started the Node Manager will first check that the interface is not in use and then bring up the IP address on the given interface before starting the Managed Server, the IP address is brought up on a sub-interface of the given interface such as eth0:1.  Similarly when the Managed Server is shutdown or fails the Node Manager will release the IP address if it allocated it.  When failover occurs the Node Manager again checks for IP usage before starting the managed server on the failover machine.

    A Problem

    The problem with this approach is what happens when you have a managed server listening on more than one address!  We can only provide one Interface and even if multiple interface properties were allowed (they are not) we would not know which NetMask to apply.

    So for many years we have labored under the inability to have a server support both multiple listening addresses (channels in WebLogic parlance) and whole server migration – until now.


    The latest Node Manager comes with a different syntax for managing IP addresses.  Instead of an Interface and NetMask property we now have a new property corresponding to the name of an interface on our computer.  For this interface we identify the range of IP addresses we manage on that interface and the NetMask associated with that address range.  This allows us to have multiple listening addresses in our managed server listening on different interfaces.  An example of adding support for multiple listening addresses on two interfaces (bond0 and eth0) is shown below:




    So now when a server has multiple listen channels then the Node Manager will check what address range each listen channel falls into and start the appropriate interface.

    A Practical Application

    A practical example of where this is useful is when we have an ExaLogic machine with an external Web Server or load balancer.  We want the managed server to talk to each other using the internal Infiband network, but we want to access the managed servers externally using the 10GigaBit ethernet network.  With the enhancements to the Node Manager we can do just this.


    Thanks to James Bayer, Will Howery and Teju for making me aware of this functionality on a recent ExaLogic training course.  Thanks to them my managed servers can now move multiple addresses without the constant forwarding of mail that I get from the Bristol office.  Now if they can just get the Bristol office to stop forwarding my mail…

    ExaLogic Documentation on Server Migration


    Musings on Fusion Middleware and SOA Picture of Antony Antony works with customers across the US and Canada in implementing SOA and other Fusion Middleware solutions. Antony is the co-author of the SOA Suite 11g Developers Cookbook, the SOA Suite 11g Developers Guide and the SOA Suite Developers Guide.


    « October 2016