Avoiding mysql_connect errors by using SMF

As mentioned earlier I spent some time with a customer last week who is deploying an apache/mysql/php app on Solaris 10. One of the big reasons that they are interested in using Solaris 10 is due to the Service Management Framework - smf(5). Availability is critically important for their system, so the restart cababilities of smf(5) were a major attraction. For a more in depth look into SMF checkout the SMF Community over on OpenSolaris.

Why use SMF in this scenario

The major issue these folks wanted to avoid is the dreaded mysql connection errors that plague apache/mysql/php applications if your database goes down. I guess that pretty much anyone who has ever browsed a php based site (which is just about everyone who uses the web) has encountered it at least once or twice. So in order to make use of smf(5) we created two service manifests, one for the downloaded mysql I mentioned previously and one for the bundled apache daemon with a dependency to ensure that libphp5 exists, and that the webserver will be shutdown and restarted if mysql needs to be restarted for any reason. Lets refer to this development platform as SAMP.

The Manifests

The two manifests are mysql-server.xml and apache-mp.xml respectively (a tarball is provided here). The respective uri's are svc:/application/samp/mysql and svc:/application/samp/apache-mp, with the apache-mp service having a dependency on the mysql service. The mysql service is dependant on hitting the multi-user milestone (svc:/milestone/multi-user), this is more by personal choice than anything else. Both of the manifests also contain dependencies on files in the installation, just as a sanity check, with apache-mp expecting to find /usr/apache/libexec/libphp5.so and mysql expecting to find /usr/local/mysql/support-files/mysql.server.

Background install and cleanup

Firstly ensure that you have mysql installed, and php compiled up as mentioned in my previous post. Please note that for this example we are using a different version of mysql than the one that comes bundled with Solaris, and as such we have installed it in /usr/local.

Next up we want to remove the legacy run scripts for Apache that are installed by default, now a caveat here, there isn't any need to remove the K16apache script from /etc/rc2.d, as SMF will execute these with an argument of 'stop', so we just need to remove /etc/rc3.d/S50apache (its just a symlink to /etc/init.d/apache if you want to undo this later).

Install Our Manifests

To install our manifests we make use of the svccfg command, first off we validate our manifests
# svccfg validate ./mysql-server.xml
# svccfg validate ./apache-mp.xml
And next up we import them
# svccfg import ./mysql-server.xml
# svccfg import ./apache-mp.xml
The manifests that we have specifed here have an entry of
<create_default_instance enabled='true' />
So our instances will be created immediately, so lets check out whats running in our samp namespace with svcs.
# svcs -l \*samp\*
STATE          STIME    FMRI
online         19:20:17 svc:/application/samp/mysql:default
online         19:20:18 svc:/application/samp/apache-mp:default
And we can take a look at the dependicies that your apache-mp service has with
# svcs -d svc:/application/samp/apache-mp
STATE          STIME    FMRI
online         11:04:55 svc:/application/samp/mysql:default

A little bit of testing....

Now if you haven't used smf you are probably wondering why go to all of this trouble, so lets give a quick example. Lets do a kill -9 on our mysqld and see what happens. Before we do that, lets take a look at the process ids and contract id's for the relevant services.
# svcs -p -l \*samp\*
fmri         svc:/application/samp/mysql:default
name         MySQL 4.1.12 (Experimental, Downloaded from mysql.com)
enabled      true
state        online
next_state   none
state_time   Tue Jul 05 11:04:55 2005
logfile      /var/svc/log/application-samp-mysql:default.log
restarter    svc:/system/svc/restarter:default
contract_id  69
dependency   require_all/none file://localhost/usr/local/mysql/support-files/mysql.server (online)
dependency   require_all/error svc:/milestone/multi-user:default (online)
process      427 /bin/sh ./bin/mysqld_safe --datadir=/usr/local/mysql/data --pid-file=/usr/local
process      450 /usr/local/mysql-standard-4.1.12-pc-solaris2.10-i386/bin/mysqld --defaults-extr

fmri         svc:/application/samp/apache-mp:default
name         Bundled Apache with PHP 5 and MySQL
enabled      true
state        online
next_state   none
state_time   Tue Jul 05 11:04:56 2005
logfile      /var/svc/log/application-samp-apache-mp:default.log
restarter    svc:/system/svc/restarter:default
contract_id  81
dependency   require_all/none file://localhost/usr/apache/libexec/libphp5.so (online)
dependency   require_all/error svc:/application/samp/mysql:default (online)
process      598 /usr/apache/bin/httpd
process      599 /usr/apache/bin/httpd
process      600 /usr/apache/bin/httpd
process      601 /usr/apache/bin/httpd
process      602 /usr/apache/bin/httpd
process      603 /usr/apache/bin/httpd
and in for the kill ........
# pkill mysql
Now lets take a look at the service states again...
# svcs \*samp\*
STATE          STIME    FMRI
online         11:45:24 svc:/application/samp/apache-mp:default
online         11:45:45 svc:/application/samp/mysql:default
And its all restarted, quite cute, and handy. Just for confirmation lets look at the pids and contract id's again.
# svcs -vp \*samp\*
STATE          NSTATE        STIME    CTID   FMRI
online         -             11:45:24     86 svc:/application/samp/apache-mp:default
               11:45:24      704 httpd
               11:45:25      705 httpd
               11:45:25      706 httpd
               11:45:25      707 httpd
               11:45:25      708 httpd
               11:45:25      709 httpd
online         -             11:45:45     88 svc:/application/samp/mysql:default
               11:45:44      731 mysqld_safe
               11:45:44      748 mysqld

Developing your own manifests

A little side note here, if your developing manifests yourself and you get an error message like the following
# svccfg validate ./foo.xml
svccfg: couldn't parse document
take a look at xmllint(1), which in the case above where I nuked the closing tag for create_default_instance, gives
# xmllint ./foo.xml
./foo.xml:91: parser error : Opening and ending tag mismatch: create_default_instance line 34 and service
./foo.xml:92: parser error : expected '>'
./foo.xml:93: parser error : Premature end of data in tag service_bundle line 28
which is somewhat more usefull than the rather cryptic "couldn't parse document", but I digress.
Technorati Tag(s) :

Cool post! Is there a way to use arbitrary scripts/utilities to test if a service is up? PIDs are cool, but executing an application level test would prove if the service is properly serving clients. - Ryan

Posted by Matty on July 08, 2005 at 06:20 PM IST #

It should be possible, I'll need to do a bit of research on it though.

Posted by fintanr on July 11, 2005 at 04:03 AM IST #

It isn't explicitly possible yet to tell smf(5) to run arbitrary scripts to monitor a service. But, we recognize the need for this functionality, and are actively working on it.

Posted by Liane Praza on August 23, 2005 at 02:11 PM IST #

Thanks for clarifying this Liane.

Posted by fintanr on August 26, 2005 at 03:52 AM IST #

Post a Comment:
  • HTML Syntax: NOT allowed



« July 2016