CAPS 6 & Java MQ: Part 3 - HA Clusters with MySQL Cluster

Why the delay?

I've had many requests recently to publish this final part of my blog series on using CAPS 6 with Java MQ for HA using MySQL Cluster. I didn't post anything for a while since the GA release of CAPS 6 did not support MQ 4.2 which is required for MySQL Cluster support (using NDB storage).

HA Clusters

HA clusters in Java MQ provide both service and data high availability, and requires the use of a HA relational database. The HA database (e.g. MySQL Cluster) is responsible for providing clustering, failover and recovery of the database servers / nodes and the JavaMQ brokers treat the database as a shared store.

You can read more about HA clusters here.

Key considerations when setting up HA clusters

  • Do I need a master broker?
  • No, there is no need for a master broker in a HA cluster. This also means that there is no single point of failure in a HA cluster configuration.

  • How many brokers and database servers should I run per cluster?
  • The number of brokers in a broker cluster and the number of database servers/nodes in your RDBMS cluster can be scaled independently of each other. There is, typically, a performance overhead associated with scaling the number of brokers and database nodes due to the additional data replication that is required between these nodes. An architect must be able to understand the impact on application performance and reliability of their system if brokers or database nodes fail and the necessary recoery time when they are restored.

  • What is 'split-brain' and how do we solve it?
  • A split-brain condition occurs if brokers/nodes within the same cluster cannot communicate with each other but are still able to process requests from their JMS clients. This is a problem in JMS systems since it is normally important that queue semantics are upheld - i.e. the 'split' cluster must still ensure that there is once only delivery of a queued message to a single client consumer. For Java MQ, this can be achieved in the message store (i.e. HA database). Specifically, if MySQL Cluster is used, a management server (a form of 'quorum') can be used that each database server in the cluster connects to and is deemed to be active only if it can see the management server.

Configuring Java MQ HA clusters for CAPS 6

The Java CAPS configuration for using HA clusters is the same as it is for conventional clusters since there are no additional client-side parameters to configure. It is necessary that the jms-service type is changed to REMOTE mode when running clustered brokers.

  1. Install and configure MySQL Cluster 6.x
  2. MySQL Cluster 6.x is currently supported. The MySQL server needs to be configured to use the NDB storage mechanism for clustered support on at least two hosts with a separate host for the management server 'quorum'. It is possible to install the management server on one of the MySQL server nodes but a separate server is recommended for true redundancy:
    • Host 1: MQ 4.2 or greater, MySQL Server/NDB Storage
    • Host 2: MQ 4.2 or greater, MySQL Server/NDB Storage
    • Host 3: Management Server (can run on host 1 or 2)
    Click here for a good step-by-step guide on how to set up MySQL Cluster on two servers.

  3. Configure MQ HA clustered brokers
  4. If you are already running brokers configured in a conventional cluster, you will firstly need to reset the stores of those brokers by running each broker with the -reset store paramater: imqbrokerd -tty -reset store -name broker1 -port 7677 imqbrokerd -tty -reset store -name broker2 -port 7678 Next stop the brokers and add the following properties to the file of each broker (e.g. in $IMQ_VARHOME/instances/broker1/props/:
    NB: The imq.cluster.clusterid property is mandatory and used in the database table name instead of the brokerid.

  5. Add JDBC drivers to each Java MQ installation classpath
  6. Copy MySQL Connector/J JDBC driver to .../imq/lib/ext of your CAPS 6 appserver's installation from where the remote broker will be started.

  7. Create message store tables
  8. Ensure that the MySQL Cluster and NDB nodes are running and then start the brokers and create the required tables from any broker instance with: imqdbmgr create tbl

  9. Deploy your CAPS 6 applications
  10. Finally deploy and start your CAPS 6 applications as detailed in the previous parts. You are now ready to begin testing failover and recovery of your CAPS 6 applications.

Performance Tuning

Be aware that due to the number of server processes that are running (app server, broker, MySQL server, NDB node, management server), there are potential overheads of this approach and, coupled with that, many opportunities for tuning the performance. There are some pointers on the CAPS Grok about changing the JDBC connection pools with JavaMQ and other tuning advice here.

So long and thanks for all the fish!

Posted by sys on August 26, 2009 at 04:52 AM BST #

I wish some high-priority bugs were listed on the documentation in big letters... instead of wasting customer time.

Posted by sys on December 21, 2009 at 06:02 AM GMT #

Post a Comment:
  • HTML Syntax: NOT allowed

Louis Polycarpou


« April 2014