Friday Jun 01, 2007

Debugging AMP

In a previous post, I mentioned the availability of the dtrace extension for Cool Stack's PHP.  Using this extension and the Cool Stack MySQL, it is possible to analyze the performance of your application running on this stack. At JavaOne, we demoed this using the open source MediaWiki and SugarCRM applications.  dtrace is especially useful in analyzing complex multi-tier applications like AMP. Thanks to Angelo Rajadurai for the creation of the scripts that I describe below.

Analyzing PHP calls

So, let's look at a simple dtrace script that counts how many times a particular PHP function is called :

#!/usr/sbin/dtrace -Zqs

php\*:::function-entry
/arg0/
{
        @[copyinstr(arg0)]=count();
}

You can copy the above to a file named php.d and start it just before the operation you want to analyze. For example, if you want to see which php functions are called when you click on Edit in your mediawiki site, start the script, click the link, then terminate the script via \^C.

Analyzing MySQL calls

The following script will print the SQL command issued by the mysql process. Invoke the script with the pid of the mysqld process as an argument, just before you execute the operation you want to debug.

#!/usr/sbin/dtrace -qs

/\* This program prints the SQL commands executed by MySQL
 \* process. It takes the pid of the mysqld process as its
 \* only argument.
 \*
 \* This script observes the occurance of dispatch_command()
 \* function and prints the SQL statement (arg2). Before you
 \* use this D-script replace the mangled name below with the
 \* output run from the mysql bin directory :
 \*/usr/ccs/bin/nm mysqld | awk -F '{ print $NF; }' | grep dispatch_command
 \*
 \*/

pid$1::_Z16dispatch_command19enum_server_commandP3THDPcj:entry
{
        printf("%d::%s\\n",tid,copyinstr(arg2));
}

 

Tracing thru PHP & MySQL 

Finally, here's a script to trace through both the PHP and MySQL calls. The output is color-coded : blue for PHP functions and green for MySQL functions (admittedly there are many different ways to format the output and this one's a bit of a hack - but it works). This script should be passed the pid of the mysqld process as an argument. Note that this script assumes that both PHP and MySQL are running on the same system. If they are running on different systems, then you will have to run the individual scripts shown above on the different systems.

#!/usr/sbin/dtrace -ZqFs

/\* We use terminal escape sequence to change color
 \* PHP - Blue
 \* MYSQL - Green
 \*/
 
php\*:::function-entry
/arg0/
{
        printf("\\033[01;34m%s\\033[0m\\n",copyinstr(arg0));
}

php\*:::function-return
/arg0/
{
        printf("\\033[01;34m%s\\033[0m\\n",copyinstr(arg0));
}

pid$1::_Z16dispatch_command19enum_server_commandP3THDPcj:\*
{
        self->sql = copyinstr(arg2);
        printf("\\033[01;32m%s\\033[0m\\n",self->sql);
}

 

 

Monday May 14, 2007

OpenSolaris WebStack launched

New OpenSolaris Project: Web Stack 

We just launched a new OpenSolaris project called Web Stack.  With the great response we have seen for Cool Stack, we thought this would be a good time to get the community to participate. Although the starting point for Web Stack is Cool Stack 1.1, we hope it will grow much beyond that. Please vist the project site to get more details on why we are doing this. We'd love to get your feedback on whether you think this is a good idea and what open source web technologies/apps you would like to see added.

And of course, we hope that you will participate in the project. You can help by building portions of the stack, add new libraries and applications, test binaries etc. So please do let us know whether you can help.

 

Shanti

 


Wednesday May 09, 2007

Building mod_jk

Here are the instructions for building mod_jk.

1. Download the source and untar it.

2. Ensure you have Sun Studio compiler in your PATH followed by /usr/ccs/bin. Save the following in 'make_solaris.sh' in the 'native' directory :

#!/bin/sh
INSTALLDIR=/opt/coolstack
APACHEDIR=$INSTALLDIR/apache2
make distclean
PATH=$INSTALLDIR/bin:$PATH
export PATH
./configure --with-apxs=/opt/coolstack/apache2/bin/apxs --with-java-home=/usr/java --with-prefix=$INSTALLDIR
make
make install

3. Go to the 'native' directory and run ./make_solaris.sh

4. Edit your httpd.conf file to include :

  LoadModule jk_module        modules/mod_jk.so

5. Restart the apache httpd server.

Let me know if this doesn't work for you.

Shanti

 

Tuesday May 08, 2007

Cool Stack at JavaOne

If you're at JavaOne this week, please do visit the "Solaris + AMP" pod (#976). We are demoing the use of SMF and dtrace on Cool Stack. You can see how dtrace can be used to debug and trace the code path through your entire application, starting from the Javascript in the browser, through PHP and finally to MySQL at the back-end. We are also distributing Cool Stack 1.1 on a DVD at the pod.

I will be there on Thursday between 11:00 - 3:00 PM, so please do stop by and say hi. 

Shanti 

Tuesday May 01, 2007

Dtrace support for PHP

Dtrace is one of the coolest things in Solaris 10, adding great observability to applications. A Dtrace provider for PHP that adds probes to function entry and exit points has long been available. I finally got around to integrating this with the php in Cool Stack. 

Bryan Cantrill has some excellent examples of how to use dtrace to trace through your php code, down to the system libraries and kernel (if you want to go that far !) .

Instructions for installing the php dtrace extension

1. Download the shared library for the extension :

    dtrace.so.x86.bz2

    dtrace.so.sparc.bz2

2. Install the extension :

    # bunzip2 dtrace.so.<arch>.bz2
  # cp dtrace.so.<arch> /opt/coolstack/php5/lib/php/extensions/no-debug\*/dtrace.so

3. Enable the dtrace extension :

  Edit your php.ini (default is in /opt/coolstack/php5/lib/php.ini) and add the line :

  extension="dtrace.so"

4. Check if the extension has been correctly loaded by executing a sample php script that takes a long time to run (you can use Bryan's blastoff.php to do this). 

#/opt/coolstack# dtrace -l |grep php
39131   php16088         dtrace.so                php_dtrace_execute function-entry
39132   php16088         dtrace.so       php_dtrace_execute_internal function-entry
39133   php16088         dtrace.so                php_dtrace_execute function-return
39134   php16088         dtrace.so       php_dtrace_execute_internal function-return
#

5. You should now be able to use the dtrace provider whenever you want using dtrace scripts. The dtrace provider will work with mod_php if you're running Apache or if you're running php stand-alone. Here is a sample command that will show which functions are called :

# dtrace -n function-entry'{printf("called %s() in %s at line %d\\n", \\
copyinstr(arg0), copyinstr(arg1), arg2)}' -q


Please feel free to contribute additional dtrace scripts.

 

Shanti



Cool Stack support is now available !

We now have Developer support for Cool Stack (or Solaris + AMP).  This dedicated online support service for developers provides technical
assistance for code support, diagnostic advice, and programming
questions that may include:

  • Sanity Checks
  • Code level support
  • Best practice guidance
  • Workarounds when available
  • And other forms of technical assistance

Sun Developer Expert Assistance Service is available to all developers,
with a cost of $49 (USD) per request or unlimited requests for an
annual subscription cost of $249 (USD).

Check out the details at http://developers.sun.com/services/expertassistance/

 

Shanti

 

Wednesday Apr 18, 2007

Web2.0 Expo

I attended the Web 2.0 Expo held in San Francisco between April 15-18. Here are some thoughts on some of the things that caught my attention.

 Rich Internet Applications

There was a lot of talk about RIAs, many products ranging from full-blown development environments to languages, language environments etc. Some examples include Apollo (new announcement from Adobe), Django, curl (no - this is not the OSS libcurl), Silverlight (Microsoft's response to Adobe !)
So it seems that the applications are once again moving onto the desktop - after moving from fat client to the web, we have now realized that the web environment really is not rich enough for many applications.

I can't help wondering why no one ever mentions Java applets in this context.  The technology was probably 10 years too early. Perhaps, if Sun had announced Java applets today at the Web 2.0 Expo, it would have passed muster as a RIA development technology. Think about it - Apollo has no threading support and is re-inventing the wheel of the "sandbox" that applets defined. Of course, we would have to tweak applets for today's multi-media apps, but I'll take programming in Java over Javascript any day. But alas, the day of the applet has passed. So let's move on.

RDF vs Microformats

There were several sessions talking about Microformats, the simple data formats that are more human readable. (It's funny how XML was touted as being both human readable and machine readable - today, it seems that microformats have that distinction).  This does seem like cool stuff - Mozilla is working on microformats support in Firefox, which means for example, that you can drag and drop contacts, appointments etc. to your desktop address box and calendar.

There wasn't a whisper about RDF . From the description of microformats, it appears that it is solving the same kinds of problems we expected RDF to solve. Microformats have a leg up on RDF because they work on any browser where as older browsers will barf at RDF. If developers start using microformats (some already have), I suspect RDF will be dead as far as its use in typical web pages (foaf:person vs hCard) and it will remain exclusively in the domain of the semantic web - which may be fine I guess. 

Browsers

Who can forget the browser wars ? If any ex-Netscape engineer out there reads this, please rejoice.  Everywhere I looked, every demo I saw (including screen-shots in presentations) used Firefox, not IE. With the exploding number of extensions and plug-ins, I really think this time around Firefox will win. It's just a matter of time !


Monday Apr 16, 2007

SMF support for MySQL in Cool Stack



Here are instructions on how to use SMF to manage MySQL. These instructions assume no knowledge of SMF. All commands should be executed as root.

Create the manifest

 The manifest defines the service and controls the privileges with which the service will execute. We use the service name csk-mysql to distinguish this as a Cool Stack service.

Create a file named /var/svc/manifest/network/cskmysql.xml with the following contents :


<?xml version='1.0'?>
 <!DOCTYPE service_bundle SYSTEM '/usr/share/lib/xml/dtd/service_bundle.dtd.1'>
<!--
    Copyright 2006,2007 Sun Microsystems, Inc.  All rights reserved.
    Manifest for CSKmysql - should reside in /var/svc/manifest/network
-->

<service_bundle type='manifest' name='CSKmysql:mysql'>

<service
        name='network/csk-mysql'
        type='service'
        version='1'>
        <create_default_instance enabled='false' />
        <single_instance />

       <!--
         Wait for network interfaces to be initialized.
        -->
       <dependency name='network'
           grouping='require_all'
           restart_on='error'
           type='service'>
           <service_fmri 

           value='svc:/milestone/network:default'/>
        </dependency>

        <!--
          Wait for all local filesystems to be mounted.
        -->
        <dependency name='filesystem-local'
            grouping='require_all'
            restart_on='none'
            type='service'>
            <service_fmri
               value='svc:/system/filesystem/local:default'/>
         </dependency>

         <exec_method
             type='method'
             name='start'
             exec='/opt/coolstack/lib/svc/method/svc-cskmysql start'
             timeout_seconds='60'>
             <method_context
working_directory='/opt/coolstack'>
                 <method_credential
                   user='mysql' group='mysql'
                   privileges='basic,!proc_session,!proc_info,!file_link_any,net_privaddr' />
             </method_context>
         </exec_method>

         <exec_method
             type='method'
             name='stop'
             exec='/opt/coolstack/lib/svc/method/svc-cskmysql stop'
             timeout_seconds='60'>
             <method_context />
         </exec_method>

         <exec_method
             type='method'
             name='refresh'
             exec='/opt/coolstack/lib/svc/method/svc-cskmysql restart'
             timeout_seconds='60'>
             <method_context working_directory='/opt/coolstack'>
                 <method_credential
                   user='mysql' group='mysql'
                   privileges='basic,!proc_session,!proc_info,!file_link_any,net_privaddr' />
             </method_context>
         </exec_method>

 </service>
 </service_bundle>

Create the method

Create the file /opt/coolstack/lib/svc/method/svc-cskmysql referenced in the manifest with the following contents and make it executable. You may have to create the directories below /opt/coolstack/lib first. This file needs to be edited to set DB_DIR to the path of your data directory (where data files reside), and MYSQL_DIR if you are using the 64-bit MySQL version.

#!/usr/bin/sh
#
#   Copyright 2007 Sun Microsystems, Inc.  All rights reserved.
#        Method file for CSKMySQL
#
# This uses the MySQL packages from CoolStack 1.1 (CSKmysql)
# If you're using the 32bit mysql from CSKamp, change MYSQL_DIR below to mysql_32bit.
# This file should reside in /opt/coolstack/lib/svc/method
#
# NOTE: Make sure DB_DIR is owned BY the mysql user and group and chmod 700
#

. /lib/svc/share/smf_include.sh

DB_DIR=/data
PIDFILE=${DB_DIR}/`/usr/bin/uname -n`.pid
MYSQL_DIR=/opt/coolstack/mysql_32bit

mysql_stop () {
        if [ -f ${PIDFILE} ]; then
            /usr/bin/pkill mysqld_safe >/dev/null 2>&1
            /usr/bin/kill `cat ${PIDFILE}` > /dev/null 2>&1 && echo -n ' mysqld'
        fi
}

mysql_start () {
        $MYSQL_DIR/bin/mysqld_safe --user=mysql --datadir=${DB_DIR} --pid-file=${PIDFILE} > /dev/null &
}

##
# Start of script
#
case "$1" in
    start)
        mysql_start
        ;;
    stop)
        mysql_stop
        ;;
    restart)
        mysql_stop
        while pgrep mysqld > /dev/null
        do
            sleep 1
        done
        mysql_start
        ;;
    \*)
        echo ""
        echo "Usage: `basename $0` { start | stop | restart }"
        echo ""
        exit 64
        ;;
esac


Change file ownership

Ensure that the mysql user and group exist and this user owns $DB_DIR. It's also a good idea to chmod 0700 all files in $DB_DIR.

Start the csk-mysql service

Import the new mysql config :

# svccfg -v import /var/svc/manifest/network/cskmysql.xml

We are now ready to start our service. Start it as follows :

# svcadm -v enable csk-mysql

If the service starts successfully, you should see mysqld running. A log of the
service startup will be in /var/svc/log/network-csk-mysql:CSKmysql.log file. You can also get more detailed information for troubleshooting startup failures from the command svcs -x. More information on SMF can be found at  http://www.sun.com/bigadmin/content/selfheal/smf-quickstart.html

 


Thursday Apr 12, 2007

SMF support for Cool Stack Apache

Here are instructions on how to use SMF to manage Apache. These instructions assume no knowledge of SMF. All commands should be executed as root.

Create the manifest

 The manifest defines the service and controls the privileges with which the service will execute. We use the service name csk-httpd to distinguish this as a Cool Stack service (and not to cause any confusion with the httpd service that already exists).

Create a file named /var/svc/manifest/network/cskapache2.xml with the following contents :

<?xml version="1.0"?>
<!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1">
<!--
    Copyright 2006-2007 Sun Microsystems, Inc.  All rights reserved.
    CSKapache2 manifest - should reside in /var/svc/manifest/network.
-->

<service_bundle type='manifest' name='CSKamp:apache'>

<service
    name='network/csk-http'
    type='service'
    version='1'>

    <!--
       Because we may have multiple instances of network/http
       provided by different implementations, we keep dependencies
       and methods within the instance.
    -->

    <instance name='CSKapache2' enabled='false'>
    <!--
       Wait for network interfaces to be initialized.
    -->
       <dependency name='network'
           grouping='require_all'
           restart_on='error'
           type='service'>
           <service_fmri value='svc:/milestone/network:default'/>
       </dependency>

       <!--
          Wait for all local filesystems to be mounted.
       -->
       <dependency name='filesystem-local'
           grouping='require_all'
           restart_on='none'
           type='service'>
           <service_fmri
               value='svc:/system/filesystem/local:default'/>
       </dependency>

       <!--
           Wait for automounting to be available, as we may be
           serving data from home directories or other remote
           filesystems.
       -->
       <dependency name='autofs'
           grouping='optional_all'
           restart_on='error'
           type='service'>
           <service_fmri
               value='svc:/system/filesystem/autofs:default'/>
       </dependency>

       <exec_method
           type='method'
           name='start'
           exec='/opt/coolstack/lib/svc/method/svc-cskapache2 start'
           timeout_seconds='60'>
           <method_context>
               <method_credential
                   user='webservd' group='webservd'
                   privileges='basic,!proc_session,!proc_info,!file_link_any,net_privaddr' />
           </method_context>
      </exec_method>

      <exec_method
           type='method'
           name='stop'
           exec='/opt/coolstack/lib/svc/method/svc-cskapache2 stop'
           timeout_seconds='60'>
           <method_context />
      </exec_method>

       <exec_method
           type='method'
           name='refresh'
           exec='/opt/coolstack/lib/svc/method/svc-cskapache2 refresh'
           timeout_seconds='60'>
           <method_context />
      </exec_method>

        <property_group name='httpd' type='application'>
            <stability value='Evolving' />
            <propval name='ssl' type='boolean' value='false' />
        </property_group>

        <property_group name='startd' type='framework'>
            <!-- sub-process core dumps shouldn't restart session -->
            <propval name='ignore_error' type='astring'
                     value='core,signal' />
        </property_group>

    </instance>

    <stability value='Evolving' />
    <template>
        <common_name>
            <loctext xml:lang='C'>
                Apache 2 HTTP server
            </loctext>
        </common_name>
        <documentation>
            <manpage title='httpd' section='8'
                manpath='/opt/coolstack/apache2/man' />
            <doc_link name='apache.org'
                uri='http://httpd.apache.org' />
        </documentation>
    </template>
</service>

</service_bundle>

Create the method

Create the file /opt/coolstack/lib/svc/method/svc-cskapache2 referenced in the manifest with the following contents and make it executable. You will have to create all directories below /opt/coolstack/lib first.This file assumes a certain name and location for the apache configuration and pid file. Edit it if you are not using the default settings from Cool Stack.

#!/sbin/sh
#
# Copyright 2004-2007 Sun Microsystems, Inc.  All rights reserved.
# Use is subject to license terms.
#
# ident "@(#)http-apache2       1.2     04/11/11 SMI"
# Modified for apache in CSKamp package of Cool Stack
# This file should reside in /opt/coolstack/lib/svc/method

. /lib/svc/share/smf_include.sh

APACHE_HOME=/opt/coolstack/apache2
CONF_FILE=$APACHE_HOME/conf/httpd.conf
PIDFILE=$APACHE_HOME/logs/httpd.pid

[ ! -f ${CONF_FILE} ] &&  exit $SMF_EXIT_ERR_CONFIG

case "$1" in
    start)
        /bin/rm -f ${PIDFILE}
        cmd="start"
        ;;
    refresh)
        cmd="graceful"
        ;;
    stop)
        cmd="stop"
        ;;
    \*)
        echo "Usage: $0 {start|stop|refresh}"
        exit 1
        ;;
esac

exec ${APACHE_HOME}/bin/apachectl $cmd 2>&1

Change file ownership

Since we want to start apache as user webservd (not root), we need to ensure that this user can write to the log directory and the pid file. All these files  reside in /opt/coolstack/apache2/logs by default.

# cd /opt/coolstack/apache2
# chown -R webservd logs
# chgrp -R webservd logs


Disable the Solaris http service

We do not want our service to conflict with the apache2 service that ships with Solaris. Note that by default, this service is disabled. You can check if it is enabled as follows :

# svcs |grep http

If no output is printed, then it is disabled. If you see something like :

maintenance    11:47:11 svc:/network/http:apache2

or

online    11:47:11 svc:/network/http:apache2

then, the service is up. Disable the service as follows : 

# svcadm -v disable http

svc:/network/http:apache2 disabled.
 

Start the csk-http service

Import the new service config as follows :

# svccfg -v import /var/svc/manifest/network/cskapache2.xml

We are now ready to start our service. Start it as follows :

# svcadm -v enable csk-http 

If the service starts successfully, you should see httpd processes running. A log of the service startup will be in /var/svc/log/network-csk-http:CSKapache2.log file. You can also get more detailed information for troubleshooting startup failures from the command svcs -x. More information on SMF can be found at  http://www.sun.com/bigadmin/content/selfheal/smf-quickstart.html

 

 


 

Wednesday Apr 04, 2007

Welcome

I finally caved in. After resisting the blogging world for so long, I have now decided to get on the bandwagon. The main reason for my resistance has been due to the fact that I didn't think I could keep it up - how in the world was I going to post entries consistently ? What interesting things did I have to say that others would find useful or insightful ?

 It's not that I think I have answers to these questions - rather, I've come to the conclusion that it's okay not to treat a blog as a diary and feel pressured to post something regularly. So, go ahead and subscribe to it - I promise you that you will not be inundated !

So, what am I going to focus on in this blog ? Primarily, Cool Stack and open source apps on Solaris for now.

 

Shanti

Cool Stack Lead

 

About

I'm a Senior Staff Engineer in the Performance & Applications Engineering Group (PAE). This blog focuses on tips to build, configure, tune and measure performance of popular open source web applications on Solaris.

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today