Set up Node Manager for Agile Cluster Servers

This article describes how to create Node Manager for Agile Cluster and manage them remotely. Suppose we have three Linux machines, one for Admin Server, the other two are for Managed Servers. Below is Agile Cluster information.

# Machine slag9310w5c.mycompany.com
Weblogic Server Location: /opt/bea/wlserver_10.3
Admin Server Name: slag9310w5c-AgileServer
Listen Address: slag9310w5c.mycompany.com:7001
Agile Domain location: /opt/agile0/agileDomain

# Machine slag9310w5c-1.mycompany.com
Weblogic Server Location: /opt/bea/wlserver_10.3
Managed Server 1 Name: slag9310w5c-ManagedServer1
Listen Address: slag9310w5c-1.mycompany.com:7001
Agile Domain location: /opt/agile1/agileDomain

# Machine slag9310w5c-2.mycompany.com
Weblogic Server Location: /opt/bea/wlserver_10.3
Managed Server 2 Name: slag9310w5c-ManagedServer2
Listen Address: slag9310w5c-2.mycompany.com:7001
Agile Domain location: /opt/agile2/agileDomain


Add machines to each cluster node

# Node Manager for Admin Server
slag9310w5c --> slag9310w5c.mycompany.com:6666
# Node Manager for Managed Server 1
slag9310w5c-1 --> slag9310w5c-1.mycompany.com:6667
# Node Manager for Managed Server 2
slag9310w5c-2 --> slag9310w5c-2.mycompany.com:6668

This need to be done from Weblogic Admin Console. We create 3 Machines and add listen address/port for each. These are used for Node Managers.

And then assign each machine to corresponding server. For example we assign slag9310w5c machine to server slag9310w5c-AgileServer.

We can review them in config.xml to confirm we are doing right.


Configure Node managers

On each machine, we create its own Node Manager. We create these directories on each.

# Admin Server‘s Node Manager location: /opt/agile0/nodemanager/slag9310w5c
[oracle@slag9310w5c agile0]$ pwd
/opt/agile0
[oracle@slag9310w5c agile0]$ mkdir -p nodemanager/slag9310w5c

# Managed Server 1‘s Node Manager location: /opt/agile1/nodemanager/slag9310w5c-1
[oracle@slag9310w5c-1 agile1]$ pwd
/opt/agile1
[oracle@slag9310w5c-1 agile1]$ mkdir -p nodemanager/slag9310w5c-1

# Managed Server 2‘s Node Manager location: /opt/agile2/nodemanager/slag9310w5c-2
[oracle@slag9310w5c-2 agile2]$ pwd
/opt/agile2
[oracle@slag9310w5c-2 agile2]$ mkdir -p nodemanager/slag9310w5c-2

Copy startNodeManager.sh from Weblogic Server to each Node Manager‘s directory.

# On Admin Server‘s machine
[oracle@slag9310w5c slag9310w5c]$ pwd
/opt/agile0/nodemanager/slag9310w5c
[oracle@slag9310w5c slag9310w5c]$ cp /opt/bea/wlserver_10.3/server/bin/startNodeManager.sh startNodeManager.sh

And modify its NODEMGR_HOME to point to its real Node Manager directory, and JAVA_OPTIONS to includes weblogic.nodemanager.ServiceEnabled=true which is to fix a BUG of Weblogic that nmStart command never returns to console.

# On Admin Server‘s Node Manager dirctory, edit /opt/agile0/nodemanager/slag9310w5c/startNodeManager.sh
NODEMGR_HOME="/opt/agile0/nodemanager/slag9310w5c"
JAVA_OPTIONS="${JAVA_OPTIONS} -Dweblogic.nodemanager.ServiceEnabled=true"

Do same to 2 Managed Servers to create startNodeManager.sh and edit it. So we will have:

# On Admin Server, /opt/agile0/nodemanager/slag9310w5c/startNodeManager.sh
NODEMGR_HOME="/opt/agile0/nodemanager/slag9310w5c"
JAVA_OPTIONS="${JAVA_OPTIONS} -Dweblogic.nodemanager.ServiceEnabled=true"

# On Managed Server 1, /opt/agile1/nodemanager/slag9310w5c-1/startNodeManager.sh
NODEMGR_HOME="/opt/agile1/nodemanager/slag9310w5c-1"
JAVA_OPTIONS="${JAVA_OPTIONS} -Dweblogic.nodemanager.ServiceEnabled=true"

# On Managed Server 2, /opt/agile2/nodemanager/slag9310w5c-2/startNodeManager.sh
NODEMGR_HOME="/opt/agile2/nodemanager/slag9310w5c-2"
JAVA_OPTIONS="${JAVA_OPTIONS} -Dweblogic.nodemanager.ServiceEnabled=true"

Create nodemanager.properties file in each Node Manager‘s directory.

# On Admin Server, /opt/agile0/nodemanager/slag9310w5c/nodemanager.properties
ListenAddress=slag9310w5c.mycompany.com
ListenPort=6666
SecureListener=false
StartScriptEnabled=true
StartScriptName=startServerAdminFromNM.sh
NativeVersionEnabled=true
CrashRecoveryEnabled=true

# On Managed Server 1, /opt/agile1/nodemanager/slag9310w5c-1/nodemanager.properties
ListenAddress=slag9310w5c-1.mycompany.com
ListenPort=6667
SecureListener=false
StartScriptEnabled=true
StartScriptName=startServerManaged1FromNM.sh
NativeVersionEnabled=true
CrashRecoveryEnabled=true

# On Managed Server 2, /opt/agile2/nodemanager/slag9310w5c-2/nodemanager.properties
ListenAddress=slag9310w5c-2.mycompany.com
ListenPort=6668
SecureListener=false
StartScriptEnabled=true
StartScriptName=startServerManaged2FromNM.sh
NativeVersionEnabled=true
CrashRecoveryEnabled=true

By default, Node Manager will start startWeblogic.sh file. So we use StartScriptName to specify our own Agile Shell Script file. Make sure StartScriptEnabled is set to true. NativeVersionEnabled is used to enable emKill command to stop Agile Server remotely. CrashRecoveryEnabled enables Agile server to restart automatically if it crashes.

From above we see NodeManager will execute startServerAdminFromNM.sh for Admin Server, startServerManaged1FromNM.sh for Managed Server 1 and startServerManaged2FromNM.sh for Managed Server 2. These three files do not exist. So we need to create them. Just copy from the Shell Script in agileDomain/bin/ folder.

# Admin Server /opt/agile0/agileDomain/bin/ directory
[oracle@slag9310w5c bin]$ cp startServerAgileAdmin.sh startServerAdminFromNM.sh

# Managed Server 1 /opt/agile1/agileDomain/bin/ directory
[oracle@slag9310w5c-1 bin]$ cp startServerAgileManaged1.sh startServerManaged1FromNM.sh

# Managed Server 2 /opt/agile2/agileDomain/bin/ directory
[oracle@slag9310w5c-2 bin]$ cp startServerAgileManaged2.sh startServerManaged2FromNM.sh

Because Node Manager invokes these SH file from its own process, we need to edit these three files to use absolute Path for setEnv.sh execution.

# Admin Server /opt/agile0/agileDomain/bin/startServerAdminFromNM.sh
#. ./setEnv.sh   -- comment it
#cd ..           -- comment it
. /opt/agile0/agileDomain/bin/setEnv.sh

# Managed Server 1 /opt/agile1/agileDomain/bin/startServerManaged1FromNM.sh
#. ./setEnv.sh   -- comment it
#cd ..           -- comment it
. /opt/agile1/agileDomain/bin/setEnv.sh

# Managed Server 2 /opt/agile2/agileDomain/bin/startServerManaged2FromNM.sh
#. ./setEnv.sh   -- comment it
#cd ..           -- comment it
. /opt/agile2/agileDomain/bin/setEnv.sh

And be sure to add weblogic.nodemanager.ServiceEnabled=true before "weblogic.server". For example:

"$JAVA_HOME/bin/java"  ... ... -Dweblogic.nodemanager.ServiceEnabled=true weblogic.Server


Enroll each node manager to agileDomain

This needs to be done one Admin Server, 2 Managed Servers one by one. Before enroll, the Admin Server must be started manually.

Invoke Agile‘s sentEnv.sh to setup runtime environment

[oracle@slag9310w5c bin]$ pwd
/opt/agile0/agileDomain/bin

[oracle@slag9310w5c bin]$ source setEnv.sh
Your environment has been set.

Run weblogic.WLST program

[oracle@slag9310w5c bin]$ java weblogic.WLST

Initializing WebLogic Scripting Tool (WLST) ...
Jython scans all the jar files it can find at first startup. Depending on the system, this process may take a few minutes to complete, and WLST may not return a prompt right away.
Welcome to WebLogic Server Administration Scripting Shell
Type help() for help on available commands

Use connect command to connect to Admin Server

wls:/offline> connect(‘superadmin‘, ‘agile‘, ‘t3://slag9310w5c.mycompany.com:7001‘)
Connecting to t3://slag9310w5c.mycompany.com:7001 with userid superadmin ...
Successfully connected to Admin Server ‘slag9310w5c.mycompany.com-AgileServer‘ that belongs to domain ‘agileDomain‘.
Warning: An insecure protocol was used to connect to the server. To ensure on-the-wire security, the SSL port or Admin port should be used instead.

Enroll Admin Server‘s Node Manager to agileDomain

wls:/agileDomain/serverConfig> nmEnroll(‘/opt/agile0/agileDomain‘, ‘/opt/agile0/nodemanager/slag9310w5c‘)
Enrolling this machine with the domain directory at /opt/agile0/agileDomain ...
Successfully enrolled this machine with the domain directory at /opt/agile0/agileDomain.

wls:/agileDomain/serverConfig> exit()
Exiting WebLogic Scripting Tool.

Now we have Admin Server‘s Node Manager successfully enrolled to agileDomain. Next we need to do same on Managed Server‘s host with same steps.

source setEnv.sh
java weblogic.WLST
connect(‘superadmin‘, ‘agile‘, ‘t3://slag9310w5c.mycompany.com:7001‘)
nmEnroll(‘/opt/agile1/agileDomain‘, ‘/opt/agile1/nodemanager/slag9310w5c-1‘)  # on Managed Server 1 host
nmEnroll(‘/opt/agile2/agileDomain‘, ‘/opt/agile2/nodemanager/slag9310w5c-2‘)  # on Managed Server 2 host


Start each Node Manager

Logon Admin Server, start its Node Manager.

[oracle@slag9310w5c slag9310w5c]$ pwd
/opt/agile0/nodemanager/slag9310w5c
[oracle@slag9310w5c slag9310w5c]$ ./startNodeManager.sh
...
...
  
Jul 24, 2014 11:06:39 PM weblogic.nodemanager.server.Listener run
INFO: Plain socket listener started on port 6,666, host slag9310w5c.mycompany.com

Need to do same on each Managed Server and start their own Node Manager.


Start Agile Server from remote machine

After Node Manager of Admin Server is started successfully, we can start the Admin Server from other machines which has been installed Weblogic Server, using WLST script.

  • nmConnect: connect to remote Node Manager
  • nmStart: start remote Weblogic Server
  • nmServerStatus: check remote server status
  • nmKill: stop remote Weblogic Server

For example I am starting Agile Admin Server from my local Windows machine by connecting to Admin's Node Manasger.

Note: The password of Node Manager is different from Admin Console Logon password. Node Manager password is defined separately.

C:\Oracle\Middleware\wlserver_12.1\server\bin>setWLSEnv.cmd
Your environment has been set.

C:\Oracle\Middleware\wlserver_12.1\server\bin>java weblogic.WLST

Initializing WebLogic Scripting Tool (WLST) ...
Welcome to WebLogic Server Administration Scripting Shell
Type help() for help on available commands

wls:/offline> nmConnect(‘superadmin‘, ‘agile‘, ‘slag9310w5c.mycompany.com‘, ‘6666‘, ‘agileDomain‘,‘/opt/agile0/agileDomain‘,‘plain‘)
Connecting to Node Manager ...
Successfully Connected to Node Manager.
wls:/nm/agileDomain> nmStart(‘slag9310w5c.mycompany.com-AgileServer‘)
Starting server slag9310w5c.mycompany.com-AgileServer ...
Successfully started server slag9310w5c.mycompany.com-AgileServer ...


wls:/nm/agileDomain> nmServerStatus(‘slag9310w5c.mycompany.com-AgileServer‘)
RUNNING

wls:/nm/agileDomain> nmKill(‘slag9310w5c.mycompany.com-AgileServer‘)
Killing server slag9310w5c.mycompany.com-AgileServer ...
Successfully killed server slag9310w5c.mycompany.com-AgileServer ...

We can see the execution from Node Manager log

    
INFO: Starting WebLogic server with command line: /opt/agile0/agileDomain/bin/startServerAdminFromNM.sh
    
Jul 24, 2014 11:07:15 PM weblogic.nodemanager.server.ServerManager log
INFO: Server output log file is ‘/opt/agile0/agileDomain/servers/slag9310w5c.mycompany.com-AgileServer/logs/slag9310w5c.mycompany.com-AgileServer.out‘

startServerAdminFromNM.sh is called by Node Manager and server log is written to slag9310w5c.mycompany.com-AgileServer.out file.

To start Agile Managed Servers from remote machine, use the same command with corresponding parameters. For example starting Managed Server 1, we have commands:

setWLSEnv.cmd
java weblogic.WLST
nmConnect(‘superadmin‘, ‘agile‘, ‘slag9310w5c-1.mycompany.com‘,‘6667‘, ‘agileDomain‘,‘/opt/agile1/agileDomain‘,‘plain‘)
nmStart(‘slag9310w5c-1.mycompany.com‘)
nmServerStatus(‘slag9310w5c-1.mycompany.com‘)
nmKill(‘slag9310w5c-1.mycompany.com‘)

Another way to start Managed Servers, we can logon Weblogic Console web page to start them from Start button, showing in below screenshot.



Comments:

Hi,
we tried to setup nodemanager in our environment by following your post and succeeded as well. But I see strange problem in my case. When I try to start any of the managed servers through WLST command nmStart or through the weblogic.Admin utility the control never returns back though the server starts successfully. I mean everytime I had to do this , I need to issue CTRL+C to exit the command. How can I overcome this ?

Thanks
Vidhya Shankar

Posted by guest on December 17, 2014 at 02:34 PM CST #

Thank you Vidhya for your comment.
This is a known issue I ever met before. You can add below parameter to your shell script to start nodemanager.
-Dweblogic.nodemanager.ServiceEnabled=true

I also wrote this in this blog.

Posted by Jie Chen on December 18, 2014 at 02:35 PM CST #

Post a Comment:
  • HTML Syntax: NOT allowed
About

Search

Categories
Archives
« May 2015
SunMonTueWedThuFriSat
     
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
      
Today