Load-balancer plugin in GlassFish now supports Apache 2.2

Support for Apache 2.2 will be coming in GlassFish Server v2.1. Users can download this version of GlassFish server from GlassFish download page. The enterprise edition will even contain installer support for installing load-balancer plugin on Apache 2.2.

For the benefit  of GlassFish user, steps to manually configure load-balancer plugin with Apache2.2 is provided in this blog.


 Platform supported

  • Solaris SPARC
  • Solaris x86
  • Linux

For exact version of the platform supported, please refer to release notes of GlassFish Server 2.1

NOTE : Apache 2.2 in not supported on windows right now. However Apache 2.0.x is supported on windows.  


Download Location

The load-balancer plugin library can be downloaded from http://download.java.net/javaee5/external/<OS>/aslb/jars where OS can be SunOS, SunOS_X86 and Linux.

The latest version of load-balancer plugin available is aslb-9.1.1-b5.jar. User can download any other latest version compared to above version, if available.


Steps to configure load-balancer plugin on Apache2.2

  1. Build Apache 2.2 and install it. If user wants to use auto-apply feature, then it should be build with openssl and server certificate must be installed. You can refer to Apache2.0 documentation for building Apache2.2 with ssl and installing certificate. Use --with-included-apr option when building Apache 2.2 to build bundled apr. Apache 2.2 install directory will be referred to as <apache2.2-install-dir>.

  2. Download load-balancer plugin and unjar it. Then unzip SUNWaspx.zip and SUNWaslb.zip. This directory will be referred to as <lbplugin-unzip-dir>.

  3. Create directory

    1. <apache2.2-install-dir>/modules/errorpages
    2. <apache2.2-install-dir>/modules/resource
    3. <apache2.2-install-dir>/sec_db_files
  4. Copy file - <lbplugin-unzip-dir>/lib/webserver-plugin/<OS>/apache2.2/mod_loadbalancer.so to <apache2.2-install-dir>/modules. Also change permission of the file to executable.

  5. Copy files - <lbplugin-unzip-dir>/lib/webserver-plugin/<OS>/apache2.2/errorpages/default-error.html and <lbplugin-unzip-dir>/lib/webserver-plugin/<OS>/apache2.2/errorpages/sun-http-lberror.html to <apache2.2-install-dir>/modules/errorpages directory.

  6. Copy files - <lbplugin-unzip-dir>/lib/webserver-plugin/<OS>/apache2.2/LBPluginDefault_root.res and <lbplugin-unzip-dir>/lib/webserver-plugin/<OS>/apache2.2/LBPlugin_root.res to <apache2.2-install-dir>/modules/resource directory.

  7. Copy files - <lbplugin-unzip-dir>/lib/webserver-plugin/<OS>/apache2.2/secmod.db, <lbplugin-unzip-dir>/lib/webserver-plugin/<OS>/apache2.2/cert8.db and <lbplugin-unzip-dir>/lib/webserver-plugin/<OS>/apache2.2/key3.db to <apache2.2-install-dir>/sec_db_files directory.

  8. Copy file <lbplugin-unzip-dir>/lib/install/templates/loadbalancer.xml.example to <apache2.2-install-dir>/conf directory.

  9. Copy files - <lbplugin-unzip-dir>/lib/dtds/sun-loadbalancer_1_1.dtd and <lbplugin-unzip-dir>/lib/dtds/sun-loadbalancer_1_2.dtd to <apache2.2-install-dir>/conf directory.

  10. Changes in file <apache2.2-install-dir>/conf/httpd.conf

    1. Change
      FROM:
      ServerName www.example.com:80
      TO:
      ServerName <host-name>:80

    2. Change
      FROM:
      #Include conf/extra/httpd-mpm.conf
      TO:
      Include conf/extra/httpd-mpm.conf

    3. Change
      FROM:
      #Include conf/extra/httpd-vhosts.conf
      TO:
      Include conf/extra/httpd-vhosts.conf

    4. Change
      FROM:
      #Include conf/extra/httpd-ssl.conf
      TO:
      Include conf/extra/httpd-ssl.conf

    5. Append following entry
      ##BEGIN EE LB Plugin Parameters
      LoadFile /usr/lib/libCstd.so.1 (THIS LINE IS ONLY NEEDED ON SOLARIS SPARC)
      LoadModule apachelbplugin_module modules/mod_loadbalancer.so
      #AddModule apachelbplugin_module
      <IfModule apachelbplugin_module>
      config-file <apache2.2-install-dir>/conf/loadbalancer.xml
      locale en
      </IfModule>
      ##END EE LB Plugin Parameters

  11. Changes in file <apache2.2-install-dir>/conf/extra/httpd-mpm.conf

    1. Change prefork MPM settings
      Set StartServers to 1
      Set MaxClients to 1

  12. Changes in file <apache2.2-install-dir>/conf/extra/httpd-vhosts.conf

    1. Append
      ##BEGIN EE LB Plugin Parameters
      <VirtualHost <machine-ip-address>>
          ServerName <host-name>
          DocumentRoot "<apache2.2-install-dir>/htdocs"
      </VirtualHost>
      ##END EE LB Plugin Parameters

    2. User can remove other existing VirtualHost entries from this file

  13. Changes in file <apache2.2-install-dir>/conf/extra/httpd-ssl.conf

    1. Change
      FROM:
      <VirtualHost _default_:443>
      TO:
      <VirtualHost <machine-ip-address>:443>
      b)Change
      FROM:
      ServerName www.example.com:443
      TO:
      ServerName <host-name>:443

  14. If user want to use feature auto-apply feature

    1. Export DAS certificate using command : certutil -L -d <application-server-DAS-dir>/config -n s1as -a -o sjsas.crt

    2. Copy certificate file(sjsas.crt) to <apache2.2-install-dir>/conf/ssl.crt directory. If directory does not exist, create one.

    3. Append in file <apache2.2-install-dir>/conf/extra/httpd-ssl.conf
      ##BEGIN EE LB Plugin Parameters
      <Location /lbconfigupdate >
      SSLVerifyClient require
      SSLVerifyDepth 1
      SSLRequireSSL
      SSLCACertificateFile <apache2.2-install-dir>/conf/ssl.crt/sjsas.crt
      SSLRequire ( %{SSL_CIPHER} !~ m/\^(EXP|NULL)-/ \\
      and %{SSL_CLIENT_S_DN_O} eq "Sun Microsystems" \\
      and %{SSL_CLIENT_S_DN_OU} eq "Sun Java System Application Server" \\
      and %{SSL_CLIENT_M_SERIAL} eq "<Cert-Serial-Number>" )
      </Location>
      <Location /getmonitordata >
      SSLVerifyClient require
      SSLVerifyDepth 1
      SSLRequireSSL
      SSLCACertificateFile  <apache2.2-install-dir>/conf/ssl.crt/sjsas.crt
      SSLRequire ( %{SSL_CIPHER} !~ m/\^(EXP|NULL)-/ \\
      and %{SSL_CLIENT_S_DN_O} eq "Sun Microsystems" \\
      and %{SSL_CLIENT_S_DN_OU} eq "Sun Java System Application Server" \\
      and %{SSL_CLIENT_M_SERIAL} eq "<Cert-Serial-Number>" )
      </Location>
      ##END EE LB Plugin Parameters

      NOTE:

      1. Cert-Serial-Number need to be generated from provided DAS certificate file(sjsas.crt).  Command for that is : keytool -printcert -file sjsas.crt. Note Serial number from output of above command. Take its value and change all lower case characters to upper case.
      2. Application Server name is assumed to be Sun Java System Application Server. If it is different, then it needs to be reflected here as well. Above command will print that as well.
  15. Changes in file <apache2.2-install-dir>/bin/envvars

    1. Suffix <lbplugin-unzip-dir>/lib and <apache2.2-install-dir>/modules directory to LD_LIBRARY_PATH on unix platform

  16. For auto-apply feature to work, the run-as user for apache must have writable permission for <apache2.2-install-dir>/conf directory. If security is not an issue, user can even set <apache2.2-install-dir>/conf directory permission to 777.


Comments:

Thanks for the post. Things were going well but when I try to start apache, the mod_loadbalancer.so tries to load libssl3.so which doesn't appear to exist anywhere on my system.

Its an ubuntu 8.04 installation with apache2.2 installed as an ubuntu package including ssl and using aslb-9.1.1-b10.jar.

I've tried googling but I can't find anything that tells me how to get this .so. Any suggestions?

Posted by Julian Paas on June 12, 2009 at 12:44 AM PDT #

You can run command "locate libssl3.so" to verify if file exists on your system. If it exists then add path to LD_LIBRARY_PATH.
If it does not exist then you can download Sun GlassFish Enterprise Server and install it. It has those libraries bundled.

However load-balancer plugin is not tested on ubuntu. It may have issues running on that platform.

Thanks,
Kshitiz

Posted by Kshitiz on June 12, 2009 at 01:35 AM PDT #

background:
app-server-home = /opt/glassfish
and
i assume DAS home is /opt/glassfish/domains/domain1

first issue step 14: The command ###certutil -L -d /opt/glassfish/domains/domain1/config/ -a -o sjas.crt generates an error message
certutil: function failed: security library: bad database.
with an empty sjas.crt file
Am i exec in the wrong path or is it a configuration issue

second issue having read through the apache log file a noticed a couple of errors
### ERROR:NSS could not be initialized; The issue may be missing security DB files under /opt/apache2/sec_db_files; ...Aborting Plugin initialization ... -- i verified the files all 3 of them are there but the cert file is cert8.db(aslb-9.1.1-b10.jar) instead of cert7.db, renaming didn't help
and lastly
###[error] server reached MaxClients setting, consider raising the MaxClients setting....

is this to be expected having reduced the maxclients to 1

regards,

Posted by tshepiso on June 16, 2009 at 07:26 PM PDT #

Thanks Kshitiz. I was able to get all the .so files I needed. But in the end it fails with an error inside glibc. I had a hard time resolving the dependency on libstdc++-libc6.2-2.so.3. The only way I was able to do so was to download libstdc++2.10-glibc2.2_2.95.4-27_i386.deb and install it manually. I suspect that in the end it failed because this library was i386 architecture and my server is x64.

Part of the reason I used ubuntu is because it is declared as officially supported by glassfish. http://docs.sun.com/app/docs/doc/820-4329/abpak?a=view

But I guess since it is only supported as a developer platform that implies no load balancing.

Posted by Julian Paas on June 16, 2009 at 10:31 PM PDT #

Please post your query to either forum - http://forums.java.net/jive/forum.jspa?forumID=56 or send query to alias dev@glassfish.dev.java.net

Thanks,
Kshitiz

Posted by KSHITIZ SAXENA on June 17, 2009 at 01:10 AM PDT #

I've managed to get the entire stack working with Solaris x86 and apache 2.2.11, there are a few tricks and annoyances:

1) Setting Maxclients to 1 will result in the web server only being able to handle.... yup, 1 client at a time. Given that most web browsers pipeline 4+ concurrent requests to a server (esp if static content is getting served up via the same apache instance), this is more or less worthless. We had to set the values up to something a bit more sane, like 50. By setting the StartServers, MinSpareServers and MaxSpareServers to the same value, you avoid having new instances constantly come up and default to sending requests to the first server listed. The downside is that all of these workers will ping the server if health checking is enabled. (one could go with the worker mpm, but things like php throw warnings about how apache and threads are not well tested and that you will need to proceed at your own risk.

2) for glassfish, the default setting for the max number of threads is 5... again, great for a developer, but two orders of magnitude less that what is required for anything else.

3) the aslb plugin logs debug information at the 'alert' and 'crit' level, resulting in the error log getting loaded up with 7 lines of stuff for every request that is not going to the LB plugin. If you set the LogLevel high enough to actually avoid these, you then miss out on actual errors from the aslb which are logged at the 'warn' level

4) the aslb is linked against dozens of unused libraries (it probably needs one small function from libns-httpd40.so but that lib requires, i kid you not, 22 other shared libs that are outside of the usual system library path). If you also want to use php with ldap or ldap auth for other apps outside of the aslb, load the aslb module _last_ or you will end up in shared library hell.

--john

Posted by John on June 19, 2009 at 08:29 AM PDT #

Hello everybody,

Just want to share with you.
I got the lb-plugin working on Ubuntu 8.04 (2.6.24-24-generic) by using
apache-httpd-2.2.11
openssl-0.9.7g
glassfish2.1

In addition to the procedure explained by Kshitiz, I carried out the following steps.

1 Export the loadbalancer.xml from GF Admin Console: "HTTP-Load Balancer -> lb1 -> Export" to $httpd_home/conf

2. Open $httpd_home/bin/envvars and append

$glassfish_home/lib/webserver-plugin/linux/apache2.2 // contains all important files of load balancer plugin
$sjsws_home/lib // contains libsasl.so

$httpd_home/modules // contains mod_jk.so and mod_loadbalancer.so
into LD_LIBRARY_PATH, as shown below.

LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$httpd_home/lib:/opt/httpd2.2.11/modules
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$glassfish_home/lib:$glassfish_home/lib/webserver-plugin/linux/apache2.2:$sjsws_home/lib
export LD_LIBRARY_PATH

SJSWS = Sun Java System Web Server 7

3 Create a security certificate for apache. See Sun GlassFish Enterprise Server 2.1 High Availability
Administration Guide (820-4341)

My special thanks to Kshitiz for your installation and configuration procedure.

Posted by rpimn on June 25, 2009 at 08:15 AM PDT #

On both Solaris 10 x86 and Fedora Core 10 x86 I have been unable to get the LB plugin to function.

Both platforms crash with a signal 11 with every request.

Has anyone seen this, or know how to fix it?

I'm using aslb-9.1.1-b11.jar on both platforms.

Posted by Sean McG on July 15, 2009 at 02:42 PM PDT #

I will request all of you to please post your query to either forum - http://forums.java.net/jive/forum.jspa?forumID=56 or send query to alias dev@glassfish.dev.java.net. That is correct place to issues resolved.

I hope you are not using 64 bit version of apache. It has been well tested on Solaris 10 x86, and should work fine.

Thanks,
Kshitiz

Posted by Kshitiz on July 15, 2009 at 04:03 PM PDT #

Hi kshitiz,
I have the problem

[alert] ERROR:NSS could not be initialized; The issue may be missing security DB files under /opt/apache/sec_db_files; Please ensure that secmod.db, key3.db and cert7.db files are present under /opt/apache/sec_db_files; Refer documentation for more details; Aborting Plugin initialization ...

I am with conditions;
1- Sun DAS is running on different machiine than apache
2- Ubuntu
3- Apache2.0.x

followed these instructions;
http://blogs.sun.com/sprabhu7/entry/install_configure_lb_on_apache#1

these way arounds arent resolving the problem;
http://www.nabble.com/Apache-2.2-%2B-mod_loadbalancer.so-%3D-solved-td22538976.html

Please help. [or private email]
r a o h a m m a d @ gmail. co m

Posted by Hammad on August 08, 2009 at 06:40 PM PDT #

Please post your query to either forum - http://forums.java.net/jive/forum.jspa?forumID=56 or send query to alias dev@glassfish.dev.java.net. That is correct place to issues resolved.

Please check your nss version. Use command
1. ident /usr/lib/libnss3.so
2. nss-config --version

Load-balancer plugin does not work with version 3.12.x. However it works well with 3.11.x.

Thanks,
Kshitiz

Posted by Kshitiz on August 09, 2009 at 08:08 PM PDT #

I'm wondering if you have an ETA on Apache 2.2 support under Windows? This is extremely important to me as I have a module I need to use that only works under 2.2. It seems 2.2 for Windows has been out for a while now so is there some tricky technical issue holding up the plugin from supporting 2.2? Any info would be extremely helpful and I can go down other, less desirable paths if I need to so please provide an honest assessment of if/when this support will arrive. Thank you.

Posted by darkwolf on September 03, 2009 at 05:09 AM PDT #

Truly speaking I do not have any data on when this support will come. This will even depend on Oracle acquisition of Sun. I will update the thread when I have any information on this.

Meanwhile please try using other platform, i.e., Solaris/Linux.

Thanks,
Kshitiz

Posted by Kshitiz on September 05, 2009 at 10:27 PM PDT #

While of course not what I would have liked to hear I am still happy that you provided me a fast and honest assessment so that I can determine how best to proceed. Thank you .

Posted by darkwolf on September 06, 2009 at 02:09 AM PDT #

Hi All,
The loadbalancing is working well for Apache 2.2 but blocks all the Comet HTTP Streams coming from Glassfish 2.1.1 back to the clients, how can I enable that feature ?

Thanks.

Posted by Mehdi OSMAN on December 28, 2009 at 07:17 PM PST #

I'm sorry, but I need to ping again for an update on Apache 2.2 plugin for Windows. It would be a major issue for us to move over to another platform at this point and it seems as if the Oracle/Sun thing is a done deal so was hoping you have better clarity. Thanks and sorry for harping on you, but a 2.2 plugin for Windows could really help my client and I out.

Posted by darkwolf on January 19, 2010 at 02:00 AM PST #

It is still early days for Sun and Oracle to work together. Deal is still pending approval. We will need some more time to decide the road map.

Thanks,
Kshitiz

Posted by kshitiz on January 20, 2010 at 12:05 PM PST #

I was just informed that I will be allowed to remove an Apache 2.2 level module from our application, thus I may be able to downgrade to Apache 2.0 on Windows. However, it would of course be preferable to leave things at 2.2. I won't be tackling this for about 2 weeks so before I go and downgrade to Apache 2.0 I thought I'd check one more time to see if a Windows Apache 2.2 plugin is in the works. Thanks.

Posted by darkwolf on March 15, 2010 at 05:49 AM PDT #

Downgrading to Apache 2.0 has yielded no success at all. It has been impossible to get the plug-in to work and I can't spend much more time on it. Apache won't even start and I've tried it on both a local XP and a 2003 server instance running up in Amazon and used completely new and raw installs of Apache in both default directory and putting it in a simple directory off C: (removing the spacing in the pathing issue). I found a long thread in which you supported someone that apparently got it to work, but then had additional issues. The tips you provided them like setting the PATH and all don't work for me. The thread just kind of died off, but left me with a feeling that this plug-in is not ready for prime time on Windows even after downgrading. This is all very frustrating since I spent significant time downgrading our Apache config and had issues with one of the modules in 2.0 that I had to work through, etc. It looks like the last time this was dealt with was back in early 2009. Is support for Windows Apache just plain dead or just a play project? This is for a production environment that cannot have a flaky plug-in so by some miracle is there actually some Apache 2.2 or very stable and easy to install 2.0 plug-in that works tucked away somewhere (btw, I was using the automated installer after first doing things manually with the latest plug-in download and that didn't help). The potential power of this plug-in is exactly what my client is looking for, but I don't know what to do as I'm stuck on Windows.

Posted by darkwolf on April 29, 2010 at 09:52 AM PDT #

Post a Comment:
  • HTML Syntax: NOT allowed
About

kshitiz

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today