X

Proactive insights, news and tips from Oracle WebLogic Server Support. Learn Oracle from Oracle.

  • February 1, 2018

T3 RMI Communication for WebLogic Server Running on Kubernetes

Overview

Oracle WebLogic Server supports Java EE and includes several vendor-specific enhancements. It has two RMI implementations and, beyond the standard Java EE-based IIOP RMI, WebLogic Server has a proprietary RMI protocol called T3. This blog describes the configuration aspects of generic RMI that also apply to T3, and also some T3-specific aspects for running WebLogic RMI on Kubernetes.

Background

T3 RMI is a proprietary WebLogic Server high performance RMI protocol and is a major communication component for WebLogic Server internally, and also externally for services like JMS, EJB, OAM, and many others. WebLogic Server T3 RMI configuration has evolved. It starts with a single multi-protocol listen port and listen address on WebLogic Server known as the default channel. We enhanced the default channel by adding a network access point layer, which allows users to configure multiple ports, as well as different protocols for each port, known as custom channels. When WebLogic Server is running on Kubernetes, the listen port number of WebLogic Server may or may not be the same as the Kubernetes exposed port number. For WebLogic Server running on Kubernetes, a custom channel allows us to map these two port numbers.

The following table lists key terms that are used in this blog and provides links to documentation that gives more details.

TERMINOLOGY  
Listen port The TCP/IP port that WebLogic Server physically binds to.
Public port Public port
The port number that the caller uses to define the T3 URL. Usually it is the same as the listen port, unless the connection goes through “port mapping
Port mapping An application of network address translation (NAT) that redirects a communication request from one address and port number combination to another. See port mapping.
Default channel Every WebLogic Server domain has a default channel that is generated automatically by WebLogic Server. See definition.
Custom channel Used for segregating different types of network traffic.
ServerTemplateMBean Also known as the default channel. Learn more.
NetworkAccessPointMBean Also known as a custom channel. Learn more.
WebLogic cluster communication WebLogic Server instances in a cluster communicate with one another using either of two basic network technologies: multicast and unicast. Learn more about multicast and unicast.
WebLogic transaction coordinator WebLogic Server transaction manager that serves as coordinator of the transaction
Kubernetes service See Kubernetes concepts https://kubernetes.io/docs/concepts/services-networking/service/, which defines Kubernetes back end and NodePort.
Kubernetes pod IP address Kubernetes assumes that pods can communicate with other pods, regardless of which host they land on. We give every pod its own cluster-private-IP address so you do not need to explicitly create links between pods or mapping container ports to host ports. Learn more.
ClusterMBean See ClusterBroadcastChannel.

WebLogic Server Listen Address

WebLogic Server supports two cluster messaging protocols: multicast and unicast. The WebLogic Server on Kubernetes certification was done using the Flannel network fabric. Currently, we only certify unicast communication.

By default WebLogic Server will use the default channel for unicast communication. Users can override it by setting a custom channel on the associated WebLogic Server ClusterMBean.

As part of the unicast configuration in a WebLogic cluster, a designated listen address and port is required for each WebLogic cluster member so that they can locate each other. By default, the default channel or custom channel has a null listen address and is assigned at run time as 0.0.0.0. In a multinode Kubernetes cluster environment, neither 0.0.0.0 nor localhost will allow other cluster members from different nodes to discover each other. Instead, users can use the Kubernetes pod IP address that the WebLogic Server instance is running on.

TCP Load Balancing

In general, WebLogic T3 is TCP/IP-based, so it can support TCP load balancing when services are homogeneous, such as in a Kubernetes service with multiple back ends. In WebLogic Server some subsystems are homogeneous, such as JMS and EJB. For example, a JMS front end subsystem can be configured in a WebLogic cluster in which remote JMS clients can connect to any cluster member. By contrast, a JTA subsystem cannot safely use TCP load balancing in transactions that span across multiple WebLogic domains that, in turn, extend beyond a single Kubernetes cluster. The JTA transaction coordinator must establish a direct RMI connection to the server instance that is chosen as the subcoordinator of the transaction when that transaction is either committed or rolled back.

The following figure shows a WebLogic transaction coordinator using the T3 protocol to connect to a subcoordinator. The WebLogic transaction coordinator cannot connect to the chosen subcoordinator due to the TCP load balancing.

Figure 1: Kubernetes Cluster with Load Balancing Service

To support cluster communication between the WebLogic transaction coordinator and the transaction subcoordinator across a Kubernetes environment, the recommended configuration is to have an individual NodePort service defined for each default channel and custom channel.

Figure 2: Kubernetes Cluster with One-on-One Service

Depending on the application requirements and the WebLogic subsystem used, TCP load balancing might or might not be suitable.

Port Mapping and Address Mapping

WebLogic Server supports two styles of T3 RMI configuration. One is defined by means of the default channel (see ServerTemplateMBean), and the other is defined by means of the custom channel (see NetworkAccessPointMBean).

When running WebLogic Server in Kubernetes, we need to give special attention to the port mapping.

When we use NodePort to expose the WebLogic T3 RMI service outside the Kubernetes cluster, we need to map the NodePort to the WebLogic Server listen port. If the NodePort is the same as the WebLogic Server listen port, then users can use the WebLogic Server default channel. Otherwise, users must configure a custom channel that defines a "public port" that matches the NodePort nodePort value, and a “listen port” that matches the NodePort port value.

The following graph shows a nonworking NodePort/default channel configuration and a working NodePort/custom channel configuration:

Figure 3: T3 External Clients in K8S

The following table describes the properties of the default channel versus the corresponding ones in the custom channel:

  Default Channel (ServerTemplateMBean) Custom Channel (NetworkAccessPointMBean)
Multiple protocol support (T3, HTTP, SNMP, LDAP, and more) Yes No
RMI over HTTP tunneling Yes (disable by default) Yes (disable by default)
Port mapping No Yes
Address Yes Yes

Examples of WebLogic T3 RMI configurations

WebLogic Server supports several ways to configure T3 RMI. The following examples show the common ones.

Using the WebLogic Server Administration Console

The following console page shows a WebLogic Server instance called AdminServer with a listen port of 9001 on a null listen address and with no SSL port. Because this server instance is configured with the default channel, port 9001 will support T3, http, iiop, snmp, and ldap.

Figure 4: T3 RMI via ServerTemplateMBean on WebLogic console

The following console page shows a custom channel with a listen port value of 7010, a null listen address, and a mapping to public port 30010. By default, the custom channel supports T3 protocol.

Figure 5: T3 RMI via NetworkAccessPointMBean on WebLogic Console

Using WebLogic RESTful management services

The following shell script will create a custom channel with listen port ${CHANNEL_PORT} and a paired public port ${CHANNEL_PUBLIC_PORT}.

#!/bin/sh

HOST=$1
PORT=$2
USER=$3
PASSWORD=$4
CHANNEL=$5
CHANNEL_PORT=$6
CHANNEL_PUBLIC_PORT=$7

echo "Rest EndPoint URL http://${HOST}:${PORT}/management/weblogic/latest/edit"
if [ $# -eq 0 ]; then
echo "Please specify HOST, PORT, USER, PASSWORD CHANNEL CHANNEL_PORT CHANNEL_PUBLIC_PORT"
exit 1
fi

# Start edit
curl -j --noproxy '*' --silent \
--user $USER:$PASSWORD \
-H X-Requested-By:MyClient \
-H Accept:application/json \
-H Content-Type:application/json \
-d "{}" \
-X POST http://$HOST:$PORT/management/weblogic/latest/edit/changeManager/startEdit

# Create
curl -j --noproxy '*' --silent \
--user $USER:$PASSWORD \
-H X-Requested-By:MyClient \
-H Accept:application/json \
-H Content-Type:application/json \
-d "{
name: '${CHANNEL}'
}" \
-X POST http://$HOST:$PORT/management/weblogic/latest/edit/Servers/myServer/networkAccessPoints

curl -j --noproxy '*' --silent \
--user $USER:$PASSWORD \
-H X-Requested-By:MyClient \
-H Accept:application/json \
-H Content-Type:application/json \
-d "{
listenPort: ${CHANNEL_PORT},
publicPort: ${CHANNEL_PUBLIC_PORT}
}" \
-X POST http://$HOST:$PORT/management/weblogic/latest/edit/Servers/myServer/networkAccessPoints/${CHANNEL}

curl -j --noproxy '*' --silent \
--user $USER:$PASSWORD \
-H X-Requested-By:MyClient \
-H Accept:application/json \
-H Content-Type:application/json \
-d "{}" \
-X POST http://$HOST:$PORT/management/weblogic/latest/edit/changeManager/activate

Using a WLST script

The following WLST script creates a custom T3 channel named t3Channel that has a listen port listen_port and a paired public port public_port.

host = sys.argv[1]
port = sys.argv[2]
user_name = sys.argv[3]
password = sys.argv[4]
listen_port = sys.argv[5]
public_port = sys.argv[6]

print('custom host : [%s]' % host);
print('custom port : [%s]' % port);
print('custom user_name : [%s]' % user_name);
print('custom password : ********');
print('public address : [%s]' % public_address);
print('channel listen port : [%s]' % listen_port);
print('channel public listen port : [%s]' % public_port);

connect(user_name, password, 't3://' + host + ':' + port)

edit()
startEdit()
ls()
cd('/')
cd('Servers')
cd('myServer')
create('t3Channel','NetworkAccessPoint')
cd('NetworkAccessPoints/t3Channel')
set('Protocol','t3')

set('ListenPort',int(listen_port))
set('PublicPort',int(public_port))

print('Channel t3Channel added ...')
activate()
disconnect()

Summary

WebLogic Server uses RMI communication using the T3 protocol to communicate between WebLogic Servers instances and with other Java programs and clients. When WebLogic Server runs in a Kubernetes cluster, there are special considerations and configuration requirements that need to be taken into account to make the RMI communication work. This blog describes how to configure WebLogic Server and Kubernetes so that RMI communication from outside the Kubernetes cluster can successfully reach the WebLogic Server instances running inside the Kubernetes cluster.

For many WebLogic Server features using T3 RMI, such as EJBs, JMS, JTA, and WLST, we support clients inside and outside the Kubernetes cluster. In addition, we support both a single WebLogic domain in a multinode Kubernetes cluster, and multiple WebLogic domains in a multinode Kubernetes cluster as well.

Be the first to comment

Comments ( 0 )
Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.Captcha
Oracle

Integrated Cloud Applications & Platform Services