A Journey to Cache Land

A while back I wrote an entry about scaling J2EE applications and extolled the benefits of using caching to achieve non-linear scaling.  I was at a customer this week and they wanted to improve the resiliency of their environment.  They were having problems with their application locking up and so wanted to set up Web Cache to act as a more resilient solution.  Lets look at how we did this.

Web Cache as a Load Balancer

In this case the customer effectively wanted to use Web Cache as a load balancer.  This is shown in the diagram below.

First we defined the two application servers A and B in the Web Cache.  This basicaly meant registering the host name and port number of the servers with the web cache.  We also allocated capacity to the servers, which is a relative measure of the power of the servers.

Next we defined a site in the web cache.  This is the address exposed to the outside user population.  If the web cache is not already listening on this address port combination then we need to add a new listening port with the appropriate characteristics.

Finally we can map the servers to the site so that web cache knows how to route requests.  This will be done on a round robin basis.  If the applications being accessed are stateful then we need to set up session affinity, usually cookie based.

The web cache is now set up and ready to go.  It may be necessary to add new virtual host settings on the application servers so that they have virtual hosts matching the web server listening hostname and port.

Normally the web  cache will now distribute requests across the two application servers.

In Event of Failure

So what happens if one of the applications servers starts giving errors?

Well once the error threshold for the web cache is exceeded (default setting is 3) the web cache marks the application server as down and starts pinging it.  The ping url and interval are configurable.  When the application server responds to a ping then it is marked as up and requests will be routed to it again.

By default the ping url is "/".  It is good practice to have the ping url point to a page that exercises the whole stack, for example touching the web server, J2EE and going on to query the database.  This ensures that if, for example, application server B is unable to see the database then it will not be brough back online by the web cache just because the web server is up and serving the root (/) page.

Monitoring

Web Cache is administered with a web based tool that is very intuitive (IMHO) to use and in addition to administration also provides some useful information on current status of back end servers as well as throughput statistics and cache hit rates.

The latest releases allow you to query the most requested pages.  If you ask for the most requested uncached pages you can then decide if they should be cached or not.

Summary

Web Cache is good for more than just caching pages.  However if the machine running the web cache fails then the back end application servers will be inaccessible.  If you want a true HA environment you need to have a fail over web cache ready to take over the IP address and hostname.

In none of this discussion have we mentioned what back end application servers can be used.  Well the reason for that is that web cache doesn't care.  It will sit in front of any HTTP server, Oracle, IBM, BEA, Microsoft, Sun etc.  A useful part of the oracle toolkit and more than just a cache.