Load balancing for Glassfish V2 deployments using BIG-IP System

Load balancing Glassfish V2 instances with BIG-IP System There was some interest from people visiting glassfish booth at this years Javaone, on how to use F5 Networks BIG-IP system with Glassfish. This blog explains on how to do this.

Contents:

  1. Introduction
  2. Prerequisites and General Configuration Information
  3. Configuring Glassfish V2
  4. Configuring BIG-IP System for Glassfish
  5. Connecting to the BIG-IP Device
  6. Creating the Pool
  7. Creating the HTTP Virtual Server
  8. Configuring an HTTP Health Monitor
  9. Configuring the BIG-IP System for Glassfish Deployments With SSL Traffic
  10. Creating the Loopback Virtual Server for the SSL Proxy
  11. Creating the SSL Proxy
  12. Testing

1. Introduction

Glassfish V2 is an open source, production ready Application server with great new features like clustering, light-weight high availability with in-memory replication, seamless load balancing administration etc., (The features in Glassfish V2 is explained very well in this year's Javaone session by Dhiru Pandey and Larry White, a summary can be found here).

This blog shows how the Glassfish V2 and F5 Networks Big-IP system can be integrated for an effective way to direct traffic for Glassfish Application Server deployments using the BIG-IP application traffic management device. When deployed with the Glassfish V2, the BIG-IP product can offer fast delivery, always-on access, peak security, and easy expansion for applications running on the Application Server.

F5 Networks' award-winning application traffic management products are designed to give enterprises increased security and higher uptime. With Glassfish V2, these products can provide better performance from applications that are based on the Application Server, as well as an improved return on investment for e-business infrastructures.

For more information on the BIG-IP system, see the F5 Networks web site.

2. Prerequisites and General Configuration Information

The following are prerequisites for using this guide:

  • An installation of Glassfish V2, see the excellent screencast by Alexis on '7 minutes to get started with GlassFish'.
  • The BIG-IP system running version 4.5.x or 4.6.x.
  • A brief review of the basic configuration tasks and the few pieces of information, such as IP addresses, that you should gather in preparation for completing the BIG-IP system configuration.

Note: All of the configuration procedures in this document are performed on the BIG-IP system. For specific information on how to configure GlassfishV2, consult the documentation provided at Project Glassfish.

This article is written with the assumption that you are familiar with both the BIG-IP system and Glassfish V2. For more information on configuring these products, consult the appropriate documentation.

Configuration Example

The BIG-IP system provides intelligent traffic management and high availability for Glassfish Application Server deployments. Through advanced health-checking capabilities, the BIG-IP product recognizes when resources are unavailable or under-performing and directs traffic to another resource. The BIG-IP device tracks Glassfish Application Server end-user sessions, which helps the client maintain session state with the servers. The following diagram shows an example deployment with Glassfish Application Server and the BIG-IP system. To deploy this sample application, a horizontally scalable, tiered network architecture will be used. The goal is to load-balance requests across a group of application servers.

Figure 1: Deployment and Configuration of Sample Glassfish Cluster
(Click to Enlarge)

3. Configuring Glassfish V2

Download the Glassfish V2 promoted build (Build 47, as of today 05/19/07). Follow the instructions to unbundle and configure GlassFish. Repeat this for other machines, if you want to setup a cluster with instances on different machine. In our example we are creating the following configuration
  1. Create cluster 'cluster1' - 'asadmin create-cluster cluster1'
  2. Node Agent 'node-agent-1' on Machine 1 - 'asadmin create-node-agent node-agent-1', 'asadmin start-node-agent node-agent-1'
  3. Node Agent 'node-agent-2' on Machine 2 - 'asadmin create-node-agent --host machine2 node-agent-2', 'asadmin start-node-agent node-agent-2'
  4. Instance 'i1' on 'node-agent-1' in cluster 'cluster1' - 'asadmin create-instance --nodeagent node-agent-1 --cluster cluster1 i1'
  5. Instance 'i2' on 'node-agent-2' in cluster 'cluster1' - 'asadmin create-instance --nodeagent node-agent-2 --cluster cluster1 i2'
  6. Start cluster 'cluster1' - 'asadmin start-cluster cluster1'
  7. Deploy the sample 'clusterjsp' - cd ${glassfish.home}/samples/quickstart/clusterjsp;${glassfish.home}/bin/asant deploy
'clusterjsp' is a sample which we will use to demonstrate the session failover capability as its configured to use in-memory replication feature of Glassfish V2.
As you see there are so few steps to configure a cluster and an app to make it highly available, I think its amazing what Glassfish is giving us in terms of all the features and its FREE.

Sample Application

The sample application clusterJSP is used to demonstrate load balancing and failover functionality. The response on the sample application's JavaServer Pages (JSP) page displays many useful session and request properties. You can identify the application server instance on which the request was processed. The name of the machine is printed in the response that begins Executed From Server:, for example:

Executed From Server: server-10.foo.com

Hardware Setup

Do the following to configure the hardware:

  1. Follow the instructions to set up BIG-IP in the Reference Guide. (Note: For access to technical documentation, including the reference guide, register for a free account on the Ask F5 web site). Ensure that the Configuration utility web interface is available and ready for configuration.
  2. Make sure that the machines that host each instance of Glassfish are available and ready to connect to the BIG-IP system.
  3. Configure the interfaces and VLANs.

4. Configuring BIG-IP System for Deployment With Glassfish

To configure the BIG-IP product to load-balance instances of Glassfish Application Server, complete the procedures covered in the following sections:

Important: If your Glassfish Application Server deployment uses SSL, follow the procedures in the section below called Configuring the BIG-IP System for Glassfish Application Server Deployments With SSL Traffic.

The BIG-IP system offers both web-based and command-line configuration tools so that users can work in the environment with which they are most comfortable. This guide contains procedures to configure the BIG-IP system using the BIG-IP web-based Configuration utility. If you are familiar with using the bigpipe command-line interface, you can use the command line to configure the BIG-IP device. However, it is recommended that you use the Configuration utility.

5. Connecting to the BIG-IP Device

The first step in this configuration is to connect to the BIG-IP system. The following procedures show how to access the BIG-IP web-based Configuration utility using a web browser.

Connecting to the BIG-IP System Using the Configuration Utility

1. In a browser, type the following URL: https:// < Administrative IP of the BIG-IP device >. When a Security Alert dialog box appears, click Yes. The authorization dialog box appears.

2. Type your user name and password, and click OK. The Configuration Status screen opens. Once you are logged onto the BIG-IP system, the initial screen, called the Configuration Status page, displays. From the Configuration Status page, you can access the Configuration utility, documentation such as manuals and release notes, and software downloads.

3. From the Configuration Status screen, click Configure your BIG-IP using the Configuration utility. The Configuration utility opens to the Network Map screen.

6. Creating the Pool

The first procedure in this configuration is to configure a pool for the instances of Glassfish Application Server. A BIG-IP pool is a set of devices grouped together to receive traffic according to a load-balancing method. In this example, you configure one pool for your Application Server instances. For this pool you use cookie persistence Insert mode, the recommended persistence method for Glassfish Application Server.

Creating the Pool From the Configuration Utility

1. In the navigation pane, click Pools. The Pools screen opens.

2. Click the Add button. The Add Pool screen opens.

3. In the Pool Name box, enter a name for your pool. In this example glassfishV2_Pool is used.

4. In the Load Balancing Method box, enter your preferred load-balancing method (different load-balancing methods may yield optimal results for a particular network). In this example, Round Robin (member) is selected, where connections are distributed evenly across all members in the pool.

5. In the Resources section, add the web servers to the pool.

  1. In the Member Address box, type the IP address of the Application Server. In this example, 122.10.10.1 is typed in the box.
  2. In the Service box, type the service number you wish to use for this server, or specify a service by choosing a service name from the list (for example, 38080).
  3. In this example, 38080 is typed, the default Application Server port.
  4. The Member Ratio and Member Priority boxes are optional.
  5. Click the Add button (>>) to add the member to the Current Members list. Repeat Steps a through d for each server you want to add to the pool. In this example, these steps are repeated for the other server (122.10.10.2). See Figure 2.

6. The other fields in the Add Pool screen are optional. Configure these fields as applicable for your network. (For additional information about configuring a pool, click the Help button.)

7. Click the Done button.

Figure 2: Creating the Pool
(Click to Enlarge)

8. In the Pool screen, from the Pool Name list, click the name of the pool you just created. In this example glassfishV2_Pool is selected.

9. Click the Persistence tab. The Persistence screen for the pool opens.

10. In the Persistence Type section, click the option button for Active HTTP Cookie.

11. From the Method list, select Insert.

12. In the Expiration box, type an expiration for the cookie. In this example, 30 is typed in the Minutes box.

Important: The cookie expiration should be at least equal to the application session timeout for the instances of Glassfish. The default application session timeout is 30 minutes. You could also leave the Expiration blank, and the cookie will expire when the browser is closed.

13. Click the Apply button.

Figure 3: Configuring the Cookie Persistence
(Click to Enlarge)

Command-Line Configuration

If you are using the command line, type the following:

app-bigip:~# b pool glassfishV2_Pool { member 122.10.10.1:38080 member
122.10.10.2:38080 lb_method rr persist cookie cookie_mode insert
cookie_expiration 0d 00:30:00}

The command-line alternative substitutes for steps 1 through 13 above. The command should be entered on one line.

7. Creating the Virtual Server

The next step in this configuration is to define a virtual server that references the pool you just created.

Creating the HTTP Virtual Server Using the Configuration Utility

1. In the navigation pane, click Virtual Servers. The Virtual Servers screen opens.

2. Click the Add button. The Add Virtual Server screen opens.

3. Enter the IP address and service for the virtual server, then click the Next button. In this example 192.10.10.1 is used with a service name of 80. The Configure Basic Properties screen displays. Click the Next button again.

4. Click the Pool option button, and from the list select the pool you created in the Creating the Pool section. In this example glassfishV2_Pool is selected (see Figure 4).

Figure 4: Creating the Virtual Server
(Click to Enlarge)

5. Click the Done button. For additional information about configuring a virtual server, click the Help button. To view the virtual server, click the virtual server in the list. In this example the virtual server properties are shown in Figure 5.

Figure 5: Configuring the Virtual Server
(Click to Enlarge)

Command-Line Configuration

If you are using the command line, type the following:

app-bigip:~# b virtual 192.10.10.1:80 use pool glassfishV2_Pool

The command-line alternative substitutes for steps 1 through 5 above.

8. Configuring an HTTP Health Monitor

Now configure the optional HTTP Extended Content Verification (ECV) monitor. In this example an HTTP ECV monitor is configured for the instances of Glassfish Application Server.

1. In the navigation pane, click Monitors. The Network Monitors screen opens.

2. Click the Add button. The Add Monitor dialog box opens.

3. In the Add Monitor screen, type the name of your monitor (it must be different from the monitor template name). In this example, glassfishV2_http_monitor is typed. In the Inherits From box, select the HTTP monitor template from the list. Click the Next button.

Figure 6: Creating the HTTP Monitor
(Click to Enlarge)

4. In the Configure Basic Properties section, type an Interval and Timeout value. We recommend at least a 1:3 +1 ratio between the interval and the timeout (for example, the default setting has an interval of 5 and a timeout of 16). A slightly higher ratio is recommended. In this example, 30 is entered for the Interval and 91 for the Timeout. Click the Next button. The Configure ECV HTTP Monitor screen opens.

5. In the Configure ECV HTTP Monitor screen, you can add a Send String and Receive Rule specific to that application. Complete the relevant information, and click the Done button. The Add Monitor dialog box closes, and you return to the Network Monitors screen.

Figure 7: Configuring the HTTP Monitor
(Click to Enlarge)

Command-Line Configuration

If you are using the command line, type the following:

app-bigip:~# b monitor glassfishV2_http_monitor '{use http 
interval 30 timeout 91 send
"GET /index.html HTTP/1.0" recv "" }'

The command-line alternative substitutes for steps 1 through 5 above.

6. From the Network Monitors screen, click the Basic Associations tab. The Basic Association screen opens.

7. In the Node section, select from the list the name of the monitor you created in Step 3. In this example glassfishV2_http_monitor is selected.

8. In the Node column, locate the Glassfish Application Server nodes relevant to this monitor, and make a check mark in the Add box for each node. In this example, we checked the Add box for 122.10.10.1:38080 and 122.10.10.2:38080.

Figure 8: Associating the Monitor to the Nodes
(Click to Enlarge)

9. Click Apply. You now see the glassfishV2_http_monitor in the Existing Associations column of the Node section for each instance of Glassfish Application Server. For additional information about associating a monitor, click the Help button.

Command-Line Configuration

If you are using the command line, type the following:

app-bigip:~# b node 122.10.10.1:38080 122.10.10.2:38080 monitor use
glassfishV2_http_monitor

The command-line alternative substitutes for steps 6 through 9 above.

9. Configuring the BIG-IP System for Glassfish Deployments With SSL Traffic

If your Glassfish deployment requires SSL, the configuration on the BIG-IP system is slightly different. For Application Server deployments using SSL, you need to configure an SSL proxy and a loopback virtual server on the BIG-IP system, in addition to creating the pool and health monitor.

Note: If you are not using SSL in your deployment, you do not need to perform these steps.

To configure the BIG-IP for directing SSL traffic to the instances of Glassfish Application Server, you need to complete the following procedures from the earlier sections of this document:

Then follow these additional procedures:

10. Creating the Loopback Virtual Server for the SSL Proxy

The SSL proxy uses a loopback virtual server for the SSL proxy. To create this loopback virtual server, use the following steps.

Note: Before you configure the virtual server, you must have already configured the pool (see Creating the Pool).

To create the loopback virtual server, do the following:

1. In the navigation pane, click Virtual Servers. The Virtual Servers screen opens.

2. Click the Add button. The Add Virtual Server screen opens.

3. Enter the IP address and service for the loopback virtual server, then click the Next button. In this example 150.10.10.1 is used with service of 80. The Configure Basic Properties screen displays. Click the Next button again.

4. Click the Pool option button, and from the list, select the pool you created in the Creating the Pool section. In this example glassfishV2_http is selected.

5. Click the Done button. For additional information about configuring a virtual server, click the Help button.

6. To view the virtual server, click the virtual server in the list. For more information on configuring the proxy addresses, refer to the BIG-IP Reference Guide.

Command-Line Configuration

If you are using the command line, type the following:

app-bigip:~# b virtual 150.10.10.1:80 use pool glassfishV2_http

The command-line alternative substitutes for steps 1 through 6 above.

11. Creating the SSL Proxy

The next step is to create an SSL proxy. An SSL proxy is a gateway for decrypting HTTP requests to an HTTP server and encrypting the reply. The SSL proxy on the BIG-IP system offloads the task of SSL encryption/decryption from the server, which frees processing cycles for those servers, and provides a central location for certificate management.

Important: Before creating the SSL proxy on the BIG-IP system, you should have a certificate issued by a recognized certificate authority.

To create an SSL proxy from the Configuration utility, do the following:

1. From the navigation pane, click Proxies. The Proxies screen opens.

2. Click the Add button. The Add Proxy screen appears.

3. In the Proxy Type section, make a check mark in the SSL box.

4. In the Proxy Address box, type the originating (source) IP address. This must be a valid IP address or host name. For a web site, use the registered address to which your clients connect. In this example 192.10.10.2 is used.

5. In the Proxy Service box, type https, or choose https from the list.

6. In the Destination Address box, type the address of the loopback virtual server you created in the Creating the Loopback Virtual Server for the SSL Proxy section.

7. In this example 150.10.10.1 is typed.

8. In the Destination Service box, type the same port you used for the pool in the Creating the Pool section. In this example 80 is typed.

9. In the SSL Certificate box, type the name of the SSL certificate for the server, or select it from the list.

10. In the SSL Key box, type the SSL key for the server, or select it from the list of installed keys. Be sure to choose the key that you used to create the certificate you selected in the SSL Certificate box.

Figure 9: Adding the SSL Proxy
(Click to Enlarge)

11. Click the Next button.

12. From the Rewrite Redirects list, select All. When you select All, the proxy always rewrites URIs as if they matched the originally requested URIs.

13. The other fields in the Add Proxy window are optional. Configure these fields as applicable for your network. (For additional information about configuring a Proxy, click the Help button.)

14. Click the Done button to add the Proxy.

Command-Line Configuration

If you are using the command line, type the following:

app-bigip:~# b proxy 192.10.10.2:https target virtual 150.10.10.1:http
clientssl
enable clientssl key glassfishV2_key clientssl cert glassfishV2_certificate

The command-line alternative substitutes for the steps above.

Important: Be sure to perform the procedures in the section Configuring an HTTP Health Monitor before finishing the configuration.

12. Testing

Testing the Setup

1. In the browser (for example, Mozilla), go to http://192.10.10.1/clusterjsp/. The request lands in BIG-IP, which then routes it to one of the Application Server instances. The response on the JSP page displays many useful session and request properties. You can identify the Application Server instance on which the request was processed. Look for the name of the machine in the response, Executed From Server:...

Here is an example:

Executed From Server: server-10.foo.com

2. Establish a new session. On a different machine or in a browser (for example, Firefox), go to http://192.10.10.1/clusterjsp/.

If the response cites another Application Server instance (in the example in Step 1, the other instance might be server-11.foo.com), you are assured that the BIG-IP system is distributing the requests within the available Application Server instances.

Testing the Load Balancer Algorithm

The plan is to use the sticky-round-robin algorithm. To verify that this algorithm is functioning properly on both browser windows, enter values in the two text fields for Name and Value of Attributes and click the Add Session Data button.

The second request in each session must land in the same Application Server instance that processed the first request. If the Application Server instance name in the response for the second request is the same as that for the first response, the sticky algorithm is in working order.

Verification of Failover

To verify that the failover capability is functioning properly, do the following:

1. In a new browser window, go to http://192.10.10.1/clusterjsp/.

2. Enter values in the two text fields for Name and Value of Attributes and click the Add Session Data button. The browser prints the session data in a response. Note the Application Server instance name cited in the response beginning Executed From Server.

3. Stop that Application Server instance. Type the following:


Application_Server_install_dir
/bin/asadmin stop-instance --host
DAS_hostname--port DAS_port_number instance_name

In the preceding, DAS_hostname and DAS_port_number are the host name and port number, respectively, for the Application Server's Administration Server. instance_name is the name of the instance in the Application Server cluster to be stopped.

Repeat Step 2.

This scenario would indicate a smooth go. The BIG-IP system assigns this request to an instance that is running, which then checks if the ID of the requested session is valid. If so, the instance acquires the ID using the in-memory replication feature. Finally, the response posts the data for both the current and the pre-failover sessions.

Fish for a Flat Screen TV

Blog about GlassFish for a chance to win a 52" LCD HD TV. ยป More
Comments:

Nice blog!

Posted by Nazrul on May 19, 2007 at 08:11 AM PDT #

Prashanth- This is some really cool stuff - exactly the type of advanced design and configuration guidance our users and community are interesting in learning more about. Email me as I would like to discuss how we might be able to republish this tip and link to your blog or other appropriate pages.

Best regards,
- Jeff

Posted by Jeff Browning on September 14, 2007 at 02:47 AM PDT #

Thanks Prashanth;
I'm doing something almost identical with a NetScaler,
it was good to have a detailed step by step guide on another load balancer to check my work against.

Posted by Dick Davies on November 24, 2008 at 05:59 PM PST #

Great post, just missing the one piece that I was most interested in. When terminating SSL at the load balancer, how does one make Glassfish know that the original request was SSL? We have app behavior that depends on that knowledge. We're moving from Websphere, and its load-balancing plugins injected special headers indicating the original protocol and negotiated SSL context. So all the servlet request methods returned info related to the original request rather than the proxied internal request. Does Glassfish have any out-of-the box mechanism for that?

Posted by Glen Taylor on March 24, 2009 at 06:39 AM PDT #

Nice writeup!

@ Glen - you can use an iRule to insert those headers back into the request before it's passed on to an application server instance, or you can just re-write all responses coming out of your application servers to use SSL instead. It depends on the scenario.

If your doing the iRule thing, you'd create a specific irule for the SSL based requests by assigning it to the virtual server with the SSL profile. The rule would look something like:

when HTTP_REQUEST {
HTTP::header insert "some_header" "some value"
}

This is probably easiest if your servers are already expecting something to determine if it's SSL.

If you just need to rewrite all responses coming from the application server (e.g. replacing <img src="http://"> with <img src="https://">, you would attach an iRule just like you did previously, but the rule would be something like;

when HTTP_RESPONSE {
# Snarf the content-length header. If one doesn't exist, set it to
# about 1m.
set content_length = 0
if { [HTTP::header exists "Content-Length"] } {
set content_length = [HTTP::header "Content-Length"]
} else {
set content_length "1000000"
}

# If there's data in the response, collect it
if { $content_length > 0 } {
HTTP::collect $content_length
}
}

# this event is fired after http::collect
when HTTP_RESPONSE_DATA {
# simple search and replace. This will replace ALL the http:// with https://
# though, even in the actual <body> tags in the HTML.
regsub -all "http:\\/\\/" [HTTP::payload] "https:\\/\\/" $newpayload
HTTP::payload replace 0 $clen $newpayload

# release the payload back to the browser.
HTTP::release
}

Note that the two iRules I pasted are 9.x specific, so they probably won't work on the 4.x devices. I also didn't test 'em, so YMMV if you just cut and paste! There should be an equiv set of commands for 4.x.. at least for the 1st rule, not sure about the 2nd.

The F5 "devcentral" site has a lot of good info on how you also might accomplish this.
HTH.

Posted by Jeff McCombs on April 07, 2009 at 04:53 AM PDT #

I got so wrapped up digging into Glen's question, I forgot to ask my own!

Why do you suggest using sticky sessions? That "ties" requests to a single server (at least until the cookie TTL is reached). But if you are shooting for a true load balanced configuration, wouldn't you leave the server you end up on in the hands of the F5?

Only reason I'm asking, is because I'm having trouble with an application in a clustered configuration UNLESS I have the persistence method enabled.. and it seems like that shouldn't be the case.

Posted by Jeff McCombs on April 07, 2009 at 05:02 AM PDT #

Also see:
http://blogs.sun.com/bounds/entry/load_balancing_for_glassfish_v2

Posted by Eduardo Pelegri-Llopart on December 07, 2009 at 11:12 PM PST #

Post a Comment:
  • HTML Syntax: NOT allowed
About

bloggerprashanth

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today