Monday Sep 09, 2013

Using PHP DTrace on Oracle Linux

This post shows PHP and DTrace "dynamic tracing" in action on Oracle Linux. It follows my previous post on recent PHP patches to stabilize DTrace support.

Install Oracle Linux and the UEK3 Kernel

  1. The starting point is to install Oracle Linux 6.4 from Oracle eDelivery. Wim Coekaerts blogged about the UEK3 release. I'm going to quote a paragraph from Wim here because it is fundamental to understanding Oracle Linux's direction:

    Oracle Linux is freely downloadable from http://edelivery.oracle.com/linux. Oracle Linux is free to use on as many systems as you want, is freely re-distributable without changing the CD/ISO content (so including our cute penguin), provides free security errata and bugfix errata updates. You only need to pay for a support subscription for those systems that you want/need support for, not for other systems. This allows our customers/users to run the exact same software on test and dev systems as well as production systems without having to maintain potentially two kinds of repositories. All systems can run the exact same software all the time.

  2. Once OL 6.4 is installed, add the Beta repo with:

    # cd /etc/yum.repos.d
    # mv public-yum-ol6.repo public-yum-ol6.repo.disabled
    # wget http://public-yum.oracle.com/beta/public-yum-ol6-beta.repo
    
  3. Enable the UEK3 Beta channel by editing public-yum-ol6-beta.repo and setting "enabled" to 1.

    enabled=1
  4. Install the UEK3 kernel, which supports DTrace:

    # yum update
  5. Install the DTrace utilities:

    # yum install dtrace-utils
  6. Reboot to the new UEK3 3.8.13 kernel

Install PHP

[Update: see DTrace PHP Using Oracle Linux 'playground' Pre-Built Packages for some pre-built PHP RPMs.]

  1. Download a PHP snapshot (or PHP 5.4.20 or PHP 5.5.4, when they become available) from snaps.php.net and extract it:

    $ tar -xJf php5.5-201309042230.tar.xz
    $ cd php5.5-201309042230
  2. Configure PHP:

    $ ./configure \
      --prefix=$HOME/p55 \
      --enable-dtrace \
      --disable-all --disable-cgi \
      --with-pear --enable-xml --enable-libxml --with-zlib

    This builds a minimal command line PHP with DTrace enabled. All unwanted extensions are disabled. You can include other extensions as needed. Currently PHP DTrace testing is limited to command-line use because a UEK3 DTrace fix for forked environments wasn't available at the time of the UEK3 Beta 1 release.

    The --prefix option puts the installation into a local directory, which makes it easy to see the files installed. It is easy to cleanup this directory when finished with the snapshot.

    The PEAR, XML and Zlib options allow the use of the 'pecl' command.

  3. Make the PHP binary and install it:

    $ make
    $ make install
  4. Copy php.ini-development to $HOME/p55/lib/php.ini and edit it to set the timezone, for example:

    date.timezone = America/Los_Angeles

Install PHP OCI8 for Oracle Database

To connect to Oracle Database, add PHP OCI8 as a "shared" extension:

  1. Download Oracle Instant Client "basic" and "devel" RPMs from ULN (for ULN subscribers) or OTN. You can use the 10g, 11g or 12c versions.

  2. Install Instant Client as root:

    # rpm -Uvh oracle-instantclient12.1-basic-12.1.0.1.0-1.x86_64.rpm
    # rmp -Uvh oracle-instantclient12.1-devel-12.1.0.1.0-1.x86_64.rpm
  3. As a normal user, set PATH so PHP is found:

    $ export PATH=$HOME/p55/bin:$PATH
  4. Set a PEAR proxy, if needed for access to http://pecl.php.net:

    $ pear config-set http_proxy http://myproxy.example.com:80/
  5. Set an environment variable PHP_DTRACE to enable DTrace, and install PHP OCI8:

    $ PHP_DTRACE=yes pecl install oci8-2.0.2

    The DTrace probes definitions used later in this article are based on PHP OCI8 2.0.2, so that explicit version is installed. If you install any future, later version review the probes and their arguments for differences. Note PHP OCI8 2.0 is in "development" status so changes are likely.

    When prompted for the ORACLE_HOME directory, hit return without entering text. The installation will autodetect the Instant Client RPMs. Configuration will continue and the output will contain something like:

    [ . . . ]
    checking for Oracle Database OCI8 support... yes, shared
    checking PHP version... 5.5.4, ok
    checking OCI8 DTrace support... yes
    [ . . . ]
    configure: WARNING: OCI8 extension: ORACLE_HOME is not set,
        looking for default Oracle Instant Client instead
    checking Oracle Instant Client directory...
        /usr/lib/oracle/12.1/client64/lib
    checking Oracle Instant Client SDK header directory...
       /usr/include/oracle/12.1/client64
    checking Oracle Instant Client library version compatibility... 12.1
    [ . . . ]
    
  6. Edit php.ini again and add PHP OCI8:

    extension=oci8.so
  7. Confirm the installation:

    $ php --ri oci8
    
    oci8
    
    OCI8 Support => enabled
    OCI8 DTrace Support => enabled
    OCI8 Version => 2.0.2-dev
    Revision => $Id: b30fb4bef45d9f5ce8a56b736f1546ea0cff08ef $
    Oracle Run-time Client Library Version => 12.1.0.1.0
    Oracle Compile-time Instant Client Version => 12.1
    
    Directive => Local Value => Master Value
    oci8.max_persistent => -1 => -1
    oci8.persistent_timeout => -1 => -1
    oci8.ping_interval => 60 => 60
    oci8.privileged_connect => Off => Off
    oci8.statement_cache_size => 20 => 20
    oci8.default_prefetch => 100 => 100
    oci8.old_oci_close_semantics => Off => Off
    oci8.connection_class => no value => no value
    oci8.events => Off => Off
    
    Statistics =>  
    Active Persistent Connections => 0
    Active Connections => 0

PHP OCI8 Installation Notes

For DTrace support, PHP OCI8 2.0 needs to be installed from PECL because PHP 5.4 and PHP 5.5 have PHP OCI8 1.4, which doesn't have DTrace probes. In future, when PHP 5.6 (or whatever comes after 5.5) is released, you will be able to configure a DTrace-enabled PHP OCI8 while building PHP.

You can, of course, install PHP OCI8 with Instant Client ZIP files, or simply use an existing ORACLE_HOME install.

You can DTrace-enable PHP OCI8 on a version of PHP that doesn't have DTrace available or configured. This includes older versions of PHP. You will be able to trace the PHP OCI8 probes but not any core PHP probes. Similarly you can install a DTrace-disabled PHP OCI8 on DTrace-enabled PHP.

If you install PHP OCI8 2.0 from PECL using 'phpize' and 'configure' (instead of 'pecl'), you will still need to set PHP_DTRACE=yes. This is because the --enable-dtrace option will be ignored by the limited 'configure' script of a PECL bundle.

The PHP OCI8 2.0 configuration script is suitable for "real" DTrace use but Linux SystemTap will not trace the extension.

Note that DTracing optimized binaries might give output that is not quite expected from code observation.

Verify the PHP DTrace Probes

  1. As root, enable DTrace and allow normal users to record trace information:

    # modprobe fasttrap
    # chmod 666 /dev/dtrace/helper

    Instead of the chmod, you could instead use an acl package rule to limit device access to a specific user.

  2. As a normal user, run php without any options. It will start and wait for input:

    $ php
  3. As root, list the DTrace probes that are available. Both PHP core and PHP OCI8 probes are listed:

    # dtrace -l -m php -m oci8.so
     4 php9559     php              dtrace_compile_file compile-file-entry
     5 php9559     php              dtrace_compile_file compile-file-return
     6 php9559     php                       zend_error error
     7 php9559     php ZEND_CATCH_SPEC_CONST_CV_HANDLER exception-caught
     8 php9559     php    zend_throw_exception_internal exception-thrown
     9 php9559     php                dtrace_execute_ex execute-entry
    10 php9559     php          dtrace_execute_internal execute-entry
    11 php9559     php                dtrace_execute_ex execute-return
    12 php9559     php          dtrace_execute_internal execute-return
    13 php9559     php                dtrace_execute_ex function-entry
    14 php9559     php                dtrace_execute_ex function-return
    15 php9559     php             php_request_shutdown request-shutdown
    16 php9559     php              php_request_startup request-startup
    17 php9559 oci8.so  php_oci_dtrace_check_connection oci8-check-connection
    18 php9559 oci8.so               php_oci_do_connect oci8-connect-entry
    19 php9559 oci8.so        php_oci_persistent_helper oci8-connect-expiry
    20 php9559 oci8.so            php_oci_do_connect_ex oci8-connect-lookup
    21 php9559 oci8.so php_oci_pconnection_list_np_dtor oci8-connect-p-dtor-close
    22 php9559 oci8.so php_oci_pconnection_list_np_dtor oci8-connect-p-dtor-release
    23 php9559 oci8.so               php_oci_do_connect oci8-connect-return
    24 php9559 oci8.so            php_oci_do_connect_ex oci8-connect-type
    25 php9559 oci8.so                    php_oci_error oci8-error
    26 php9559 oci8.so        php_oci_statement_execute oci8-execute-mode
    27 php9559 oci8.so             php_oci_create_spool oci8-sesspool-create
    28 php9559 oci8.so           php_oci_create_session oci8-sesspool-stats
    29 php9559 oci8.so           php_oci_create_session oci8-sesspool-type
    30 php9559 oci8.so         php_oci_statement_create oci8-sqltext
    

    The core PHP probes are documented here. ThePHP OCI8 probes are described below.

  4. In your user terminal, stop the php executable with Ctrl-C.

    $ php
    ^C
    $

PHP OCI8 2.0 DTrace Probe Overview

The static PHP OCI8 2.0 probes can be categorized as "user" probes and "maintainer" probes. The latter that are more useful for PHP OCI8 maintainers to verify functionality during development of the extension itself. All the probes return data in arguments.

User Probes are:

  • oci8-connect-entry - initiated by oci_connect(), oci_pconnect() and oci_new_connect(). Fires before database connection is established.
    • char *username - the connection username
    • char *dbname - the database connection string
    • char *charset - the character set specified
    • long session_mode - A binary "or" of OCI_SYSDBA (0x2), OCI_SYSOPER (0x4) and OCI_CRED_EXT (1<<31, or -2147483648 on the platform I was using). Set to 0 by default.
    • int persistent - set to 1 if oci_pconnect() was called, 0 otherwise
    • int exclusive - set to 1 if oci_new_connect() was called, 0 otherwise
  • oci8-connect-return - fires at the end of connection.
    • void *connection - the address of the connection structure
  • oci8-check-connection - initiated if an Oracle error might have caused the connection to become invalid
    • void *connection - the address of the connection structure
    • int is_open - will be 0 if the errcode or server_status indicate the connection is invalid and must be recreated.
    • long errcode - the Oracle error number
    • unsigned long server_status - an indicator from the Oracle library if the connection is considered invalid. If is_open is 0 because errcode indicated the connection was invalid, then server_status will be its default of 1.
  • oci8-sqltext - initiated when oci_parse() is executed
    • void *connection - the address of the connection structure
    • char *sql - text of the SQL statement executed
  • oci8-error - initiated if an Oracle error occurs
    • int status - the Oracle return status of the failing Oracle library call, such as -1 for Oracle's OCI_ERROR or 1 for Oracle's OCI_SUCCESS_WITH_INFO. See Oracle's oci.h for all definitions.
    • long errcode - the Oracle error number
  • oci8-execute-mode - indicates the commit state of an oci_execute() call
    • void *connection - the address of the connection structure
    • unsigned int mode - the mode passed to the Oracle library such as OCI_NO_AUTO_COMMIT (0x00), OCI_DESCRIBE_ONLY (0x10) or OCI_COMMIT_ON_SUCCESS (0x20)

Maintainer probes are below. Refer to the PHP OCI8 source code for the argument descriptions:

  • oci8-connect-p-dtor-close
    • void *connection
  • oci8-connect-p-dtor-release
    • void *connection
  • oci8-connect-lookup
    • void *connection
    • int is_stub
  • oci8-connect-expiry
    • void *connection
    • int is_stub
    • long idle_expiry
    • long timestamp
  • oci8-connect-type
    • int persistent
    • int exclusive
    • void *connection
    • long num_persistent
    • long num_connections
  • oci8-sesspool-create
    • void *session_pool
  • oci8-sesspool-stats
    • unsigned long free
    • unsigned long busy
    • unsigned long open
  • oci8-sesspool-type
    • int type
    • void *session_pool

The PHP OCI8 probes are highly likely to be extended prior to PHP OCI8 2.0 being marked "production". The PHP OCI8 documentation will be updated only at that time, but you can check the oci8_dtrace.d file in the PHP OCI8 source code to see the probe arguments for your version. (Update: The documentation is here).

The probes in PHP OCI8 2.0 replace PHP OCI8 1.4's use of oci_internal_debug() tracing. This function has become a no-op.

Using PHP OCI8 and DTrace

Follow these steps.

  1. Create a simple PHP file, oci8.php, to query the database:

    <?php
    
    error_reporting(0);
    ini_set('display_errors', 'Off');
    
    function do_query($c, $sql)
    {
        $s = oci_parse($c, $sql);
        if (!$s)
            return;
        $r = oci_execute($s);
        if (!$r)
            return;
        while (($row = oci_fetch_row($s)) != false) {
            foreach ($row as $item) {
                echo $item . " ";
            }
            echo "\n";
        }
    }
    
    $c = oci_new_connect('hr', 'welcome', 'localhost/pdborcl');
    
    do_query($c, "select city from locations where rownum < 5 order by 1");
    do_query($c, "select something from does_not_exist");
    
    ?>
  2. Create a D script, user_oci8.d, to probe the execution of oci8.php:

    #!/usr/sbin/dtrace -Zs
    
    php*:::oci8-connect-entry
    {
        printf("PHP connect-entry\n");
        printf("\t   username      %s\n", arg0 ? copyinstr(arg0) : "");
        printf("\t   dbname        %s\n", arg1 ? copyinstr(arg1) : "");
        printf("\t   charset       %s\n", arg2 ? copyinstr(arg2) : "");
        printf("\t   session_mode  %ld\n", (long)arg3);
        printf("\t   persistent    %d\n", (int)arg4);
        printf("\t   exclusive     %d\n", (int)arg5);
    }
    
    php*:::oci8-connect-return
    {
        printf("PHP oci8-connect-return\n");
        printf("\t   connection    0x%p\n", (void *)arg0);
    }
    
    php*:::oci8-connection-close
    {
        printf("PHP oci8-connect-close\n");
        printf("\t   connection    0x%p\n", (void *)arg0);
    }
    
    php*:::oci8-error
    {
        printf("PHP oci8-error\n");
        printf("\t   status        %d\n", (int)arg0);
        printf("\t   errcode       %ld\n", (long)arg1);
    }
    
    php*:::oci8-check-connection
    {
        printf("PHP oci8-check-connection\n");
        printf("\t   connection    0x%p\n", (void *)arg0);
        printf("\t   is_open       %d\n", arg1);
        printf("\t   errcode       %ld\n", (long)arg2);
        printf("\t   server_status %lu\n", (unsigned long)arg3);
    }
    
    php*:::oci8-sqltext
    {
        printf("PHP oci8-sqltext\n");
        printf("\t   connection    0x%p\n", (void *)arg0);
        printf("\t   sql           %s\n", arg0 ? copyinstr(arg1) : "");
    }
    
    php*:::oci8-execute-mode
    {
        printf("PHP oci8-execute-mode\n");
        printf("\t   connection    0x%p\n", (void *)arg0);
        printf("\t   mode          0x%x\n", arg1);
    }
    
  3. As root, start the D script. It will pause, waiting for probes to be fired:

    # chmod +x user_oci8.d
    # ./user_oci8.d

    (Later, this terminal can be Ctrl-C'd when you have finished experimenting with PHP)

  4. Run command-line PHP in another window. The output from the successful query is displayed:

    $ php oci8.php 
    Beijing 
    Bern 
    Bombay 
    Geneva 
    
  5. In the root terminal running the D script, the probes firing during execution of PHP will be displayed:

    # ./user_oci8.d
    dtrace: script 'user_oci8.d' matched 0 probes
    CPU  ID                    FUNCTION:NAME
    1    18 php_oci_do_connect:oci8-connect-entry PHP connect-entry
            username      hr
            dbname        localhost/pdborcl
            charset       
            session_mode  0
            persistent    0
            exclusive     0
    
    0    23 php_oci_do_connect:oci8-connect-return PHP oci8-connect-return
            connection    0x7f64e112cff0
    
    0    31 php_oci_statement_create:oci8-sqltext PHP oci8-sqltext
            connection    0x7f64e112cff0
            sql           select city from locations where rownum < 5 order by 1
    
    0    27 php_oci_statement_execute:oci8-execute-mode PHP oci8-execute-mode
            connection    0x7f64e112cff0
            mode          0x20
    
    0    31 php_oci_statement_create:oci8-sqltext PHP oci8-sqltext
            connection    0x7f64e112cff0
            sql           select something from does_not_exist
    
    0    27 php_oci_statement_execute:oci8-execute-mode PHP oci8-execute-mode
            connection    0x7f64e112cff0
            mode          0x20
    
    0    26 php_oci_error:oci8-error PHP oci8-error
            status        -1
            errcode       942
    
    0    17 php_oci_dtrace_check_connection:oci8-check-connection PHP oci8-check-connection
            connection    0x7f64e112cff0
            is_open       1
            errcode       942
            server_status 1
    
    0    25 php_oci_connection_close:oci8-connection-close PHP oci8-connect-close
             connection    0x7f64e112cff0

    (Adding "-q" to the /usr/sbin/dtrace arguments in user_oci8.d will suppress the CPU and ID details.)

    On multi-CPU machines the probe ordering might not appear sequential, depending on which CPU was processing the probes. Displaying probe timestamps will help reduce confusion, for example:

    php*:::oci8-connect-entry
    {
        printf("PHP connect-entry at %lld\n", walltimestamp);
    }

    From the user_oci8.d DTrace output, you can see

    • The connection being initiated (oci8-connect-entry). The user 'hr' connected to the 'localhost/pdborcl' database. It was an oci_connect() call because both 'exclusive' and 'persistent' were 0. No explicit character set was requested. The default session mode (the optional fifth parameter to oci_connect) was requested.

    • Two SQL statements being parsed (oci8-sqltext) and executed (oci-execute-mode) with mode 0x20 aka OCI_COMMIT_ON_SUCCESS.

    • An Oracle error ORA-942 "table or view does not exist" was generated (oci8-error)

    • The error causing the connection status to be verified (oci8-check-connection). The value of is_open is 1, indicating that the connection is OK.

    With this information you can trace problematic statement execution and connection issues.

Conclusion

This is just a morsel about using DTrace, which is a very powerful utility. Following on from the example above, you could integrate PHP OCI8 tracing with core PHP tracing. Bryan Cantrill posted some examples of core PHP tracing in DTrace and PHP, demonstrated (Note that blog platform upgrades have caused single backslashes to display as double backslashes in his post. Also you no longer need the separate PHP DTrace extension). To explore more DTrace power look on the web for example scripts. There are various blogs too.

Remember that the intent of DTrace is that its functionality is enabled all the time, suitable for development and ready for when you need it most: in production. The design of DTrace means that the probes have zero overhead when nothing is monitoring them.

Finally, as I write this article, I can already see how the PHP OCI8 probes can be enhanced (perhaps to display the connection client identifier to aid end-to-end tracing through the Oracle stack.) Oracle Linux support for DTrace should be improving all the time, too. The power of DTrace on Linux is growing and it's time to think about incorporating it into your application life cycle.

Updated 26 Sep 2013 to mention using an ACL rule for /dev/dtrace/helper

Wednesday Sep 04, 2013

DTrace with PHP Update

I've recently been working with the in-built DTrace code in PHP 5.4 and PHP 5.5, specifically stabilizing its configuration on Solaris and Oracle Linux.

DTrace is an always-available, low overhead, tracing framework that has just celebrated its 10th Birthday. DTrace support in PHP was originally via a separate PECL extension. In PHP 5.4 David Soria Parra (then at Sun) merged DTrace functionality to core PHP making it more accessible.

At the end of last year, I blogged about Adding DTrace Probes to PHP Extensions and how Linux's SystemTap could be used to trace the DTrace probe points. Since then, Oracle's Linux's DTrace project has been making great strides. The latest Oracle Linux UEK3 Beta kernel was just released. It comes with DTrace 0.4 and also supports "User-Level Statically Defined Tracing" (USDT) for the first time. This motivated me to make sure PHP DTrace worked well with "real" DTrace, not just with SystemTap's wrappers.

Some of the recent PHP DTrace changes were:

  • DTrace build script changes from PHP 5.5 were merged back to PHP 5.4 so both branches are now in sync.

  • A 'make install' recursive dependency issue that caused Zend/zend_dtrace.d to be deleted was fixed.

  • PHP DTrace configuration now uses the correct PIC or non-PIC objects. This also fixed building PHP when any extensions were built 'shared'.

  • A segmentation fault in the "error" probe was fixed.

  • PHP's OCI8 2.0 extension now builds correctly with "real" DTrace.

The changes were tested on Solaris and Oracle Linux. The core PHP changes should be available in the next monthly releases, PHP 5.4.20 and PHP 5.5.4. The PHP OCI8 2.0 changes are already available from PECL.

I'll cover more detail about using PHP and PHP OCI8 with DTrace on Oracle Linux in a later blog post, but the basic steps are:

  • Install the UEK3 kernel, currently in Beta from public-yum.oracle.com.

  • Install the dtrace-utils package with 'yum install dtrace-utils'

  • Download a PHP 5.4 or 5.5 snapshot from snaps.php.net (Or download PHP 5.4.20, PHP 5.5.4, or later, once they are available)

  • Configure PHP with the --enable-dtrace option

  • Load the DTrace module: modprobe fasttrap

  • Change the helper device to allow writing: chmod 666 /dev/dtrace/helper

  • Run a DTrace profiling script and your PHP application

Update: the detail is in a new blog post Using PHP DTrace on Oracle Linux

Thursday Jul 25, 2013

Using PHP and Oracle Database 12c Implicit Result Sets

Note: This post describes functionality in PHP OCI8 2.0.0-devel. Functionality and naming is subject to change.

The new Oracle Database 12c "Implicit Result Sets" (IRS) feature allows query results to be returned from a stored PL/SQL procedure (or a PL/SQL anonymous block) without requiring special PHP code. Support for IRS is available in PHP's OCI8 2.0.0-devel extension when it is compiled and used with Oracle Database 12c. (OCI8 2.0 can be compiled and used with other versions of Oracle Database but the available feature set is reduced).

Recall that a normal Oracle query can be performed in PHP with a parse-execute-fetch loop like:

<?php
$c = oci_connect('hr', 'welcome', 'localhost/pdborcl');
$sql = "select city from locations where rownum < 4";
$s = oci_parse($c, $sql);
oci_execute($s);
while (($row = oci_fetch_row($s)) != false) {
    foreach ($row as $item) {
        echo $item . " ";
    }
    echo "\n";
}
?>

The output is:

Beijing
Bern
Bombay

With PHP OCI8 2.0.0-devel and Oracle Database 12c, the same results can be obtained by changing $sql to an anonymous PL/SQL block (or by calling a previously created stored PL/SQL procedure) that uses DBMS_SQL.RETURN_RESULT like:

$sql =
 "declare
    c1 sys_refcursor;
  begin
    open c1 for select city from locations where rownum < 4;
    dbms_sql.return_result(c1);
  end;";

With this statement change, the previous PHP code fetches the query results without needing any logic alterations. In older versions without IRS, the PL/SQL and PHP code would have to handle a REF CURSOR parameter.

The real fun begins when you have multiple DBMS_SQL.RETURN_RESULT calls in the same PL/SQL block:

$sql =
 "declare
    c1 sys_refcursor;
  begin
    open c1 for select city from locations where rownum < 4;
    dbms_sql.return_result(c1);
    open c1 for select first_name, last_name from employees where rownum < 4;
    dbms_sql.return_result(c1);
  end;";

The PHP fetch loop handles this nicely and sequentially fetches rows from both queries:

Beijing  
Bern  
Bombay  
Ellen Abel  
Sundar Ande  
Mozhe Atkinson  

Only oci_fetch_array(), oci_fetch_assoc(), oci_fetch_object() and oci_fetch_row() (but not oci_fetch() or oci_fetch_all()) will automatically fetch IRS data like this. [This is a semi-arbitrary decision trying to balance the increased amount of code complexity and testing versus the expected use of the feature. If there is strong demand this decision can be revisited.]

To process each of the query results independently in PHP, use the newly introduced oci_get_implicit_resultset() function. This takes the parent statement resource, e.g. $s, and returns a PHP statement resource corresponding to the first result set. Each time oci_get_implicit_resultset() is subsequently called, it returns the next result set. When there are no more result sets, it returns false. Because oci_get_implicit_resultset() returns a statement resource, you can use any of the oci_fetch_* functions to get rows.

For example, to print appropriate column names above each row's items:

<?php
$c = oci_connect('hr', 'welcome', 'localhost/pdborcl');
$sql =
 "declare
    c1 sys_refcursor;
  begin
    open c1 for select city from locations where rownum < 4;
    dbms_sql.return_result(c1);
    open c1 for select first_name, last_name from employees where rownum < 4;
    dbms_sql.return_result(c1);
  end;";
$s = oci_parse($c, $sql);
oci_execute($s);
while (($s2 = oci_get_implicit_resultset($s))) {
    // Now treat $s2 as a distinct query statement resource
    $ncols = oci_num_fields($s2);
    for ($i = 1; $i <= $ncols; ++$i) {
        $colname = oci_field_name($s2, $i);
        echo $colname . " ";
    }
    echo "\n";
    while (($row = oci_fetch_row($s2)) != false) {
        foreach ($row as $item) {
            echo $item . " ";
        }
        echo "\n";
    }
}
?>

The output is:

CITY  
Beijing  
Bern  
Bombay  
FIRST_NAME LAST_NAME  
Ellen Abel  
Sundar Ande  
Mozhe Atkinson  

You can also do things like calling oci_set_prefetch() on the IRS statement resources, or calling oci_get_implicit_resultset() multiple times before beginning to fetch row data:

$sql =
 "declare
    c1 sys_refcursor;
  begin
    open c1 for select city from locations where rownum < 4;
    dbms_sql.return_result(c1);
    open c1 for select first_name, last_name from employees where rownum < 4;
    dbms_sql.return_result(c1);
  end;";
$s = oci_parse($c, $sql);
oci_execute($s);
$s1 = oci_get_implicit_resultset($s);
$s2 = oci_get_implicit_resultset($s);
$row1_1 = oci_fetch_row($s1);
$row2_1 = oci_fetch_row($s2);
$row1_2 = oci_fetch_row($s1);
$row2_2 = oci_fetch_row($s2);
. . .

There are more examples in the OCI8 test suite (see the tests sub-directory of the source bundle).

When would you used an IRS? Some cases are:

  • Migrating an application to Oracle Database, where the application currently makes use of stored procedures returning result sets. The IRS support in OCI8 will make migration easier.
  • Instead of calling a PL/SQL procedure and then executing a separate SELECT in PHP, complex processing can be performed in a stored PL/SQL procedure before that procedure returns results to PHP. This architecture might help system scalability since it reduces the number of calls made to the database.
  • PHP Frameworks typically use a basic oci_fetch_* call, so they will will automatically be able fetch IRS results.

I can't suggest turning every individual SELECT statement into an IRS since this will add the overhead of the PL/SQL-to-SQL calls. However there are legitimate use cases for Implicit Result Sets that open some interesting possibilities.

PHP OCI8 2.0.0-devel for Oracle Database is on PECL

PHP OCI8 2.0.0-devel is now available on PECL (and in the PHP 'master' branch). This is a development release, so changes may occur. I'm looking for feedback, particularly on:

  • Feature design, including naming choices
  • Install feedback from different platforms with different build tool versions
  • Whether to drop support for installing on PHP 4. I'm likely to do this. (The OCI8 1.4.10 release is usable for installing on PHP 4)

OCI8 2.0 can be installed as a shared extension from PECL with:

pecl install oci8-devel

Alternatively, if you have OCI8 1.x currently installed as a static extension, then you will need to build PHP again. This is easy using a trunk snapshot from snaps.php.net. Instructions for building are in various places on the web, including in The Underground PHP and Oracle Manual.

The OCI8 2.0.0-devel release contains a bunch of clean ups, and some new and changed features. The full list is found in the package.xml file:

    - NEW FUNCTIONALITY:
 
      - Added Implicit Result Set support for Oracle Database 12c.
	Streaming of all IRS's returned from a PL/SQL block is available
	via oci_fetch_array, oci_fetch_assoc, oci_fetch_object and
	oci_fetch_row (but not oci_fetch or oci_fetch_all).
	Alternatively individual IRS statement resources can be obtained
	with the new function 'oci_get_implicit_resultset' and passed to
	any oci_fetch_* function.

      - Added DTrace probes enabled with PHP's generic --enable-dtrace

    - IMPROVED FUNCTIONALITY:
 
      - Using 'oci_execute($s, OCI_NO_AUTO_COMMIT)' for a SELECT no
	longer unnecessarily initiates an internal ROLLBACK during
	connection close.  This can improve overall scalability by
	reducing "round trips" between PHP and the database.
 
    - CHANGED FUNCTIONALITY:
 
      - PHPINFO() CHANGES:
 
        - The oci8.event and oci8.connection_class values are now shown
          only when the Oracle client libraries support the respective
          functionality.
 
        - Connection statistics are now in a separate phpinfo() table.
 
        - Temporary LOB and Collection support status lines in
          phpinfo() were removed.  These features have always been
          enabled since 2007.
 
      - OCI_INTERNAL_DEBUG() CHANGES:
 
        - The oci_internal_debug() function is now a no-op.  Use PHP's
          --enable-dtrace functionality with DTrace or SystemTap instead.
 
    - INTERNAL CHANGES:
 
      - Fixed a potential NULL pointer dereference flagged by Parfait
        static code analysis.
 
      - Extended testing of existing OCI8 functionality.
 
      - Improved test output portability when using the PHP development
        web server to run tests.
 
      - Removed no-longer necessary Unicode patterns from tests
        (vestiges of PHP's previous PHP 6 project)
        
      - Improved build portability by removing compilation type cast
        warnings with some compilers.
 
      - Fixed compilation warnings when building with Oracle 9.2
        client libraries.
 
      - Updated code to use internal macro PHP_OCI_REGISTER_RESOURCE.
 
      - Regularized code prototypes and fixed some in-line documentation
        prototypes.
 
      - Fixed code folding. 

Subsequent blog posts will talk about the features in OCI8 2.0.0.

Thursday May 16, 2013

Offline Processing in PHP with Advanced Queuing

Offloading slow batch tasks to an external process is a common method of improving website responsiveness. One great way to initiate such background tasks in PHP is to use Oracle Streams Advanced Queuing in a producer-consumer message passing fashion. Oracle AQ is highly configurable. Messages can queued by multiple producers. Different consumers can filter messages. From PHP, the PL/SQL interface to AQ is used. There are also Java, C and HTTPS interfaces, allowing wide architectural freedom. Oracle Advanced Queuing is included in all editions of the database.

The following example simulates an application user registration system where the PHP application queues each new user's street address. An external system monitoring the queue can then fetch and process that address. In real life the external system might initiate a snail-mail welcome letter, or do further, slower automated validation on the address.

The following SQL*Plus script qcreate.sql creates a new Oracle user demoqueue with permission to create and use queues. A payload type for the address is created and a queue is set up for this payload.

-- qcreate.sql

connect / as sysdba
drop user demoqueue cascade;

create user demoqueue identified by welcome;
grant connect, resource to demoqueue;
grant aq_administrator_role, aq_user_role to demoqueue;
grant execute on dbms_aq to demoqueue;
grant create type to demoqueue;

connect demoqueue/welcome@localhost/orcl

-- The data we want to queue
create or replace type user_address_type as object (
  name        varchar2(10),
  address     varchar2(50)
);
/

-- Create and start the queue
begin
 dbms_aqadm.create_queue_table(
   queue_table        =>  'demoqueue.addr_queue_tab',
   queue_payload_type =>  'demoqueue.user_address_type');
end;
/

begin
 dbms_aqadm.create_queue(
   queue_name         =>  'demoqueue.addr_queue',
   queue_table        =>  'demoqueue.addr_queue_tab');
end;
/

begin
 dbms_aqadm.start_queue(
   queue_name         => 'demoqueue.addr_queue',
   enqueue            => true);
end;
/

The script qhelper.sql creates two useful helper functions to enqueue and dequeue messages:

-- qhelper.sql
-- Helpful address enqueue/dequeue procedures

connect demoqueue/welcome@localhost/orcl

-- Put an address in the queue
create or replace procedure my_enq(name_p in varchar2, address_p in varchar2) as
  user_address       user_address_type;
  enqueue_options    dbms_aq.enqueue_options_t;
  message_properties dbms_aq.message_properties_t;
  enq_id             raw(16);
begin
  user_address := user_address_type(name_p, address_p);
  dbms_aq.enqueue(queue_name         => 'demoqueue.addr_queue',
                  enqueue_options    => enqueue_options,
                  message_properties => message_properties,
                  payload            => user_address,
                  msgid              => enq_id);
  commit;
end;
/
show errors

-- Get an address from the queue
create or replace procedure my_deq(name_p out varchar2, address_p out varchar2) as
  dequeue_options    dbms_aq.dequeue_options_t;
  message_properties dbms_aq.message_properties_t;
  user_address       user_address_type;
  enq_id             raw(16);
begin
  dbms_aq.dequeue(queue_name         => 'demoqueue.addr_queue',
                  dequeue_options    => dequeue_options,
                  message_properties => message_properties,
                  payload            => user_address,
                  msgid              => enq_id);
  name_p    := user_address.name;
  address_p := user_address.address;
  commit;
end;
/
show errors

The script newuser.php is the part of the PHP application that handles site registration for a new user. It queues a message containing their address and continues executing:

<?php
// newuser.php

$c = oci_connect("demoqueue", "welcome", "localhost/orcl");

// The new user details
$username = 'Fred';
$address  = '500 Oracle Parkway';

// Enqueue the address for later offline handling
$s = oci_parse($c, "begin my_enq(:username, :address); end;");
oci_bind_by_name($s, ":username", $username, 10);
oci_bind_by_name($s, ":address",  $address,  50);
$r = oci_execute($s);

// Continue executing
echo "Welcome $username\n";

?>

It executes an anonymous PL/SQL block to create and enqueue the address message. The immediate script output is simply the echoed welcome message:

Welcome Fred

Once this PHP script is executed, any application can dequeue the new message at its leisure. For example, the following SQL*Plus commands call the helper my_deq() dequeue function and displays the user details:

-- getuser.sql

connect demoqueue/welcome@localhost/orcl

set serveroutput on
declare
  name varchar2(10);
  address varchar2(50);
begin
  my_deq(name, address);
  dbms_output.put_line('Name     : ' || name);
  dbms_output.put_line('Address  : ' || address);
end;
/

The output is:

Name     : Fred
Address  : 500 Oracle Parkway

If you instead want to check the queue from PHP, use getuser.php:

<?php
// getuser.php

$c = oci_connect("demoqueue", "welcome", "localhost/orcl");

// dequeue the message
$sql = "begin my_deq(:username, :address); end;";
$s = oci_parse($c, $sql);
oci_bind_by_name($s, ":username", $username, 10);
oci_bind_by_name($s, ":address", $address, 50);
$r = oci_execute($s);

echo "Name     : $username\n";
echo "Address  : $address\n";

?>

If the dequeue operation is called without anything in the queue, it will block waiting for a message until the queue wait time expires. This is configurable by setting a zero wait time dequeue_options.wait := 0; before calling dbms_aq.dequeue.

The PL/SQL API has much more functionality than shown in this overview. For example you can enqueue an array of messages, or listen to more than one queue. Queuing is highly configurable and scalable, providing a great way to distribute workload for web or mobile applications. More information about AQ is in the Oracle Streams Advanced Queuing User's Guide.

Bootnote: The basis for this blog post comes from the Underground PHP and Oracle Manual

This post was updated to show setting a zero wait time.

Tuesday May 14, 2013

Getting Started with PHP Zend Framework 2 for Oracle DB

This post shows the changes to the ZF2 tutorial application to allow it to run with Oracle Database 11gR2.

Oracle Database SQL identifiers are case insensitive by default so "select Abc from Xyz" is the same as "select abc from xyz". However the identifier metadata returned to programs like PHP is standardized to uppercase by default. After executing either query PHP knows that column "ABC" was selected from table "XYZ".

In PHP code, array indices and object attributes need to match the schema identifier case that is returned by the database. This is either done by using uppercase indices and attributes in the PHP code, or by forcing the SQL schema to case-sensitively use lower-case names.

The former approach is more common, and is shown here.

The instructions for creating the sample ZF2 application are here. Follow those steps as written, making the substitutions shown below.

Schema

In Oracle 11gR2, the schema can be created like:

DROP USER ZF2 CASCADE;

CREATE USER ZF2 IDENTIFIED BY WELCOME
    DEFAULT TABLESPACE USERS QUOTA UNLIMITED ON USERS
    TEMPORARY TABLESPACE TEMP;

GRANT CREATE SESSION
    , CREATE TABLE
    , CREATE PROCEDURE
    , CREATE SEQUENCE
    , CREATE TRIGGER
    , CREATE VIEW
    , CREATE SYNONYM
    , ALTER SESSION
TO ZF2;

CONNECT ZF2/WELCOME

CREATE TABLE ALBUM (
  ID NUMBER NOT NULL,
  ARTIST VARCHAR2(100) NOT NULL,
  TITLE VARCHAR2(100) NOT NULL,
  PRIMARY KEY (ID)
);

CREATE SEQUENCE ALBUMSEQ;

CREATE TRIGGER ALBUMTRIGGER BEFORE INSERT ON ALBUM FOR EACH ROW
BEGIN
  :NEW.ID := ALBUMSEQ.NEXTVAL;
END;
/

INSERT INTO ALBUM (ARTIST, TITLE)
    VALUES ('The  Military  Wives', 'In  My  Dreams');
INSERT INTO ALBUM (ARTIST, TITLE)
    VALUES ('Adele', '21');
INSERT INTO ALBUM (ARTIST, TITLE)
    VALUES ('Bruce  Springsteen', 'Wrecking Ball (Deluxe)');
INSERT INTO ALBUM (ARTIST, TITLE)
    VALUES ('Lana  Del  Rey', 'Born  To  Die');
INSERT INTO ALBUM (ARTIST, TITLE)
    VALUES ('Gotye', 'Making  Mirrors');

COMMIT;

Driver and Credentials

The driver and credentials are Oracle-specific. Always use the OCI8 adapter in ZF, since it is more stable and has better scalability. Specifying a character set will make connection faster.

zf2-tutorial/config/autoload/global.php:
 return array(
     'db' => array(
-        'driver'         => 'Pdo',
-        'dsn'            => 'mysql:dbname=zf2tutorial;host=localhost',
-        'driver_options' => array(
-            PDO::MYSQL_ATTR_INIT_COMMAND => 'SET NAMES \'UTF8\''
-        ),
+        'driver'         => 'OCI8',
+        'connection_string' => 'localhost/orcl',
+        'character_set' => 'AL32UTF8',
     ),
     'service_manager' => array(
         'factories' => array(
zf2-tutorial/config/autoload/local.php:
 return array(
     'db' => array(
-        'username' => 'YOUR USERNAME HERE',
-        'password' => 'YOUR USERNAME HERE',
+        'username' => 'ZF2',
+        'password' => 'WELCOME',
     ),
     // Whether or not to enable a configuration cache.
     // If enabled, the merged configuration will be cached and used in

Attribute & Index Changes

The rest of the application changes are just to handle the case of the Oracle identifiers correctly.

zf2-tutorial/module/Album/Module.php
                     $dbAdapter = $sm->get('Zend\Db\Adapter\Adapter');
                     $resultSetPrototype = new ResultSet();
                     $resultSetPrototype->setArrayObjectPrototype(new Album());
-                    return new TableGateway('album', $dbAdapter, null, $resultSetPrototype);
+                    return new TableGateway('ALBUM', $dbAdapter, null, $resultSetPrototype);
                 },
             ),
         );
zf2-tutorial/module/Album/view/album/album/add.phtml
 $form->prepare();
 
 echo $this->form()->openTag($form);
-echo $this->formHidden($form->get('id'));
-echo $this->formRow($form->get('title'));
-echo $this->formRow($form->get('artist'));
+echo $this->formHidden($form->get('ID'));
+echo $this->formRow($form->get('TITLE'));
+echo $this->formRow($form->get('ARTIST'));
 echo $this->formSubmit($form->get('submit'));
 echo $this->form()->closeTag();
zf2-tutorial/module/Album/view/album/album/delete.phtml
 <h1><?php echo $this->escapeHtml($title); ?></h1>
 
 <p>Are you sure that you want to delete
-'<?php echo $this->escapeHtml($album->title); ?>' by
-'<?php echo $this->escapeHtml($album->artist); ?>'?
+'<?php echo $this->escapeHtml($album->TITLE); ?>' by
+'<?php echo $this->escapeHtml($album->ARTIST); ?>'?
 </p>
 <?php
 $url = $this->url('album', array(
            'action' => 'delete',
-           'id'     => $this->id,
+           'id'     => $this->ID,
        ));
 ?>
 <form action="<?php echo $url; ?>" method="post">
     <div>
-    <input type="hidden" name="id" value="<?php echo (int) $album->id; ?>" />
+    <input type="hidden" name="id" value="<?php echo (int) $album->ID; ?>" />
     <input type="submit" name="del" value="Yes" />
     <input type="submit" name="del" value="No" />
     </div>
zf2-tutorial/module/Album/view/album/album/edit.phtml
         'album',
         array(
             'action' => 'edit',
-            'id'     => $this->id,
+            'id'     => $this->ID,
         )
     ));
 $form->prepare();
 
 echo $this->form()->openTag($form);
-echo $this->formHidden($form->get('id'));
-echo $this->formRow($form->get('title'));
-echo $this->formRow($form->get('artist'));
+echo $this->formHidden($form->get('ID'));
+echo $this->formRow($form->get('TITLE'));
+echo $this->formRow($form->get('ARTIST'));
 echo $this->formSubmit($form->get('submit'));
 echo $this->form()->closeTag();
zf2-tutorial/module/Album/view/album/album/index.phtml
</tr>
 <?php foreach ($albums as $album) : ?>
 <tr>
-<td><?php echo $this->escapeHtml($album->title);?></td>
-<td><?php echo $this->escapeHtml($album->artist);?></td>
+<td><?php echo $this->escapeHtml($album->TITLE);?></td>
+<td><?php echo $this->escapeHtml($album->ARTIST);?></td>
 <td>
 <a href="<?php echo $this->url('album',
-            array('action'=>'edit', 'id' => $album->id));?>">Edit</a>
+            array('action'=>'edit', 'id' => $album->ID));?>">Edit</a>
     <a href="<?php echo $this->url('album',
-            array('action'=>'delete', 'id' => $album->id));?>">Delete</a>
+            array('action'=>'delete', 'id' => $album->ID));?>">Delete</a>
     </td>
     </tr>
 <?php endforeach; ?>
zf2-tutorial/module/Album/src/Album/Model/Album.php
 class Album
 {
-    public $id;
-    public $artist;
-    public $title;
+    public $ID;
+    public $ARTIST;
+    public $TITLE;
     protected $inputFilter;
 
     public function exchangeArray($data)
     {
-        $this->id     = (!empty($data['id'])) ? $data['id'] : null;
-        $this->artist = (!empty($data['artist'])) ? $data['artist'] : null;
-        $this->title  = (!empty($data['title'])) ? $data['title'] : null;
+        $this->ID     = (!empty($data['ID'])) ? $data['ID'] : null;
+        $this->ARTIST = (!empty($data['ARTIST'])) ? $data['ARTIST'] : null;
+        $this->TITLE  = (!empty($data['TITLE'])) ? $data['TITLE'] : null;
     }
 
     public function getArrayCopy()

and

             $factory     = new InputFactory();
 
             $inputFilter->add($factory->createInput(array(
-                        'name'     => 'id',
+                        'name'     => 'ID',
                         'required' => true,
                         'filters'  => array(
                             array('name' => 'Int'),

and

                     )));
 
             $inputFilter->add($factory->createInput(array(
-                        'name'     => 'artist',
+                        'name'     => 'ARTIST',
                         'required' => true,
                         'filters'  => array(
                             array('name' => 'StripTags'),

and

                     )));
 
             $inputFilter->add($factory->createInput(array(
-                        'name'     => 'title',
+                        'name'     => 'TITLE',
                         'required' => true,
                         'filters'  => array(
                             array('name' => 'StripTags'),
zf2-tutorial/module/Album/src/Album/Model/AlbumTable.php
     public function getAlbum($id)
     {
         $id  = (int) $id;
-        $rowset = $this->tableGateway->select(array('id' => $id));
+        $rowset = $this->tableGateway->select(array('ID' => $id));
         $row = $rowset->current();
         if (!$row) {
             throw new \Exception("Could not find row $id");

and

     public function saveAlbum(Album $album)
     {
         $data = array(
-            'artist' => $album->artist,
-            'title'  => $album->title,
+            'ARTIST' => $album->ARTIST,
+            'TITLE'  => $album->TITLE,
         );
 
-        $id = (int)$album->id;
+        $id = (int)$album->ID;
         if ($id == 0) {
             $this->tableGateway->insert($data);
         } else {
             if ($this->getAlbum($id)) {
-                $this->tableGateway->update($data, array('id' => $id));
+                $this->tableGateway->update($data, array('ID' => $id));
             } else {
                 throw new \Exception('Form id does not exist');
             }

and

     public function deleteAlbum($id)
     {
-        $this->tableGateway->delete(array('id' => $id));
+        $this->tableGateway->delete(array('ID' => $id));
     }
 }
zf2-tutorial/module/Album/src/Album/Form/AlbumForm.php
         parent::__construct('album');
         $this->setAttribute('method', 'post');
         $this->add(array(
-                'name' => 'id',
+                'name' => 'ID',
                 'type' => 'Hidden',
             ));
         $this->add(array(
-                'name' => 'title',
+                'name' => 'TITLE',
                 'type' => 'Text',
                 'options' => array(
                     'label' => 'Title',
                 ),
             ));
         $this->add(array(
-                'name' => 'artist',
+                'name' => 'ARTIST',
                 'type' => 'Text',
                 'options' => array(
                     'label' => 'Artist',
zf2-tutorial/module/Album/src/Album/Controller/AlbumController.php
         }
 
         return array(
-            'id' => $id,
+            'ID' => $id,
             'form' => $form,
         );
     }

and

         }
 
         return array(
-            'id'    => $id,
+            'ID'    => $id,
             'album' => $this->getAlbumTable()->getAlbum($id)
         );
     }

When you create applications from scratch it will be straightforward to get it all working.

Tuesday Apr 30, 2013

OTN Forum software is going to be upgraded in May 2013

Update: the migration has been delayed so they can "establish thread level redirects across the site". Yay!

In case you missed the notifications, the Oracle Technology Network forum software is going to be upgraded this weekend. This is great, since the old software is getting long-in-the-tooth and doesn't allow a bunch of useful features. The current forums will be in read-only mode over the weekend.

On launch of the new version, a minimal set of features will be supported. Once the upgrade is stable, then additional features will be turned on.

As a side part to the migration project, some little used and obsolete forums will be removed. Some other categories will be reworked.

You can get more information about the migration here. One thing to note is that the forum software is powered by Jive; i.e it is a packaged application so not all features you (and I) have requested will magically become feasible. And also, for better or worse, Jive has renamed "forums" as "spaces" - apparently we can't change this.

Thursday Mar 21, 2013

Python cx_Oracle and Oracle 11g DRCP Connection Pooling

The topic of Oracle 11g DRCP connection pooling in Python cx_Oracle came up twice this week for me. DRCP is a database tier connection pooling solution which is great for applications run in multiple processes. There is a whitepaper on DRCP that covers a lot of background and talks about configuration. This whitepaper is ostensibly about PHP but is good reading for all DRCP users.

The first DRCP and cx_Oracle scenario I dealt with was a question about mod_python.

To cut a long story short, I created a handler and requested it 1000 times via Apache's 'ab' tool. In my first script, and despite having increased the default pool parameters, there were a high number of NUM_WAITS. Also NUM_AUTHENTICATIONS was high. Performance wasn't the best. Querying V$CPOOL_CC_STATS showed:

CCLASS_NAME  NUM_REQUESTS   NUM_HITS NUM_MISSES  NUM_WAITS NUM_AUTHENTICATIONS
------------ ------------ ---------- ---------- ---------- -------------------
HR.CJDEMO1           1000        992          8        478                1000

At least the session information in each DRCP server was reused (shown via a high NUM_HITS).

Results were better after fixing the script mptest.py to look like:

from mod_python import apache
import cx_Oracle
import datetime

# mptest.py
# Example: Oracle 11g DRCP with cx_Oracle and mod_python

# These pool params are suitable for Apache Pre-fork MPM
mypool = cx_Oracle.SessionPool(user='hr', password='welcome',
         dsn='localhost/orcl:pooled', min=1, max=2, increment=1)

def handler(req):
    global mypool

    req.content_type = 'text/html'
    n = datetime.datetime.now()
    req.write (str(n) + "<br>");

    db = cx_Oracle.connect(user='hr', password='welcome',
            dsn='localhost/orcl:pooled', pool=mypool, cclass="CJDEMO1",
            purity=cx_Oracle.ATTR_PURITY_SELF)

    cur = db.cursor()
    cur.execute('select * from locations')
    resultset = cur.fetchall()
    for result in resultset:
        for item in result:
            req.write (str(item) + " ")
        req.write ("<br>")
    cur.close()
    mypool.release(db)

    return apache.OK

The 'ab' benchmark on this script ran much faster and the stats from V$CPOOL_CC_STATS looked much better. The number of authentications was right down about to about 1 per Apache (ie. mod_python) process:

CCLASS_NAME  NUM_REQUESTS   NUM_HITS NUM_MISSES  NUM_WAITS NUM_AUTHENTICATIONS
------------ ------------ ---------- ---------- ---------- -------------------
HR.CJDEMO1           1000        977         23         13                  26

The NUM_HITS was high again, because the DRCP purity was ATTR_PURITY_SELF. If I hadn't wanted session information to be reused each time the handler was executed, I could have set the purity to ATTR_PURITY_NEW. If I'd done this then NUM_HITS would have been low and NUM_MISSES would have been high.

If you're testing this yourself, before restarting the DRCP pool don't forget to shutdown Apache to close all DB connections. Otherwise restarting the pool will block. Also, if you're interpreting your own V$CPOOL_CC_STATS stats don't forget to account for the DRCP "dedicated optimization" that retains an association between clients (mod_python processes) and the DB. The whitepaper previously mentioned discusses this.

The second place where DRCP and python came up this week was on the cx_Oracle mail list. David Stanek posed a question. He was seeing application processes blocking while waiting for a DRCP pooled server to execute a query. My variant of David's script is drcp2.py:

import os
import time
import cx_Oracle

# drcp2.py
# Example: Sub-optimal connection pooling with Oracle 11g DRCP and cx_Oracle

def do_connection():
    print 'Starting do_connection ' + str(os.getpid())
    con = cx_Oracle.connect(user=user, password=pw, dsn=dsn, cclass="CJDEMO2",
           purity=cx_Oracle.ATTR_PURITY_SELF)
    cur = con.cursor()
    print 'Querying ' + str(os.getpid())
    cur.execute("select to_char(systimestamp) from dual")
    print cur.fetchall()
    cur.close()
    con.close()
    print 'Sleeping ' + str(os.getpid())
    time.sleep(30)
    print 'Finishing do_connection ' + str(os.getpid())
 
user = 'hr'
pw = 'welcome'
dsn = 'localhost/orcl:pooled'
for x in range(100):
    pid = os.fork()
    if not pid:
        do_connection()
        os._exit(0)

This script forks off a bunch of processes - more than the number of pooled DRCP servers (see MAXSIZE in the DBA_CPOOL_INFO view). The first few processes grab a DRCP server from the pool and do their query. But they don't release the DRCP server back to the DRCP pool until after the sleep() when the process ends. The other forked processes are blocked waiting for those DRCP servers to become available. This isn't optimal pool sharing.

My suggestion was to use an explicit cx_Oracle session pool like drcp3.py:

import os
import time
import cx_Oracle

# drcp3.py
# Example: Connection pooling with Oracle 11g DRCP and cx_Oracle
 
def do_connection():
    print 'Starting do_connection ' + str(os.getpid())
    mypool = cx_Oracle.SessionPool(user=user,password=pw,dsn=dsn,min=1,max=2,increment=1)
    con = cx_Oracle.connect(user=user, password=pw,
          dsn=dsn, pool = mypool, cclass="CJDEMO3", purity=cx_Oracle.ATTR_PURITY_SELF)
    cur = con.cursor()
    print 'Querying ' + str(os.getpid())
    cur.execute("select to_char(systimestamp) from dual")
    print cur.fetchall()
    cur.close()
    mypool.release(con)
    print 'Sleeping ' + str(os.getpid())
    time.sleep(30)
    print 'Finishing do_connection ' + str(os.getpid())

user = 'hr'
pw = 'welcome'
dsn = 'localhost/orcl:pooled'
for x in range(100):
    pid = os.fork()
    if not pid:
        do_connection()
        os._exit(0)

The mypool.release(con) call releases the DRCP server back to the DRCP pool prior to the sleep. When this second script is run, there is a smoothness to the output. The queries happen sequentially without noticeably being blocked.

Like with any shared resource, it is recommended to release DRCP pooled servers back to the pool when they are no longer needed by the application.

Update August 2014: Oracle submitted a cx_Oracle patch which Anthony has merged but not yet released. This patch now allows drcp2.py to work efficiently. For the fork scenario the non-explicit session pool solution in drcp2.py is likely to be better than drcp3.py, since it does fewer DRCP requests. For the mod_python example, continuing to use a session pool as shown in mptest.py can minimize the number of authentications, which helps scalability.

Friday Mar 15, 2013

Using PHP 5.5's New "OPcache" Opcode Cache

Zend have contributed their Zend Optimizer+ opcode cache to PHP - thanks Zend!!! (Update 19 March 2013: the renaming to "Zend OPcache" is complete)

"The Zend OPcache provides faster PHP execution through opcode caching and optimization."

The new OPcache extension can be seen as substitute for the venerable APC cache, the maintenance of which had become an issue. Note: although OPcache is now readily available, there is currently nothing preventing you from using any available (working!) opcode cache in PHP 5.5.

A few minutes ago Dmitry Stogov did the physical merge to the PHP 5.5 source's ext/opcache directory. The current PHP 5.5 snapshot has the code. Future Alpha or Beta (and Production) releases will include it too.

Please test OPcache. It is not a panacea for all performance problems. There are a lot of settings which may need adjusting. Understanding how it works and identifying issues during the stabilization phase of PHP 5.5 release process will greatly help.

To build Zend OPcache for PHP 5.5:

When you configure PHP, add the option --enable-opcache like:

./configure ... --enable-opcache

Then make and make install, as normal. This will build OPcache shared extension. It's not possible to build it statically.

Find the shared library in your installation directory with a command like find /home/cjones/php55 -name opcache.so

Edit php.ini and add the extension with its full path:

zend_extension=/home/cjones/php55/lib/php/extension/debug-non-zts-20121212/opcache.so

Update (25 March 2013): Dmitry merged a PHP 5.5 change so that the full path is not required for zend_extension libraries in the extension_dir directory. You can now simply do zend_extension=opcache.so.

You'll want to enable OPcache too:

opcache.enable=On

The ext/opcache/README is the current source of documentation, and lists all the other php.ini parameters.

Problems can be reported in the Github issue tracker

Update (18 March 2013): In a commit over the weekend, the build option --enable-opcache is On by default. You will still need to update php.ini.

To build Zend OPcache for older versions of PHP:

You should be able to build OPcache with PHP 5.2 onwards

Install it by getting the source from Github. There is also a PECL repository; this is slightly out of date so I recommend using Github. Follow the README instructions to install it.

User-data cache:

The new opcode cache does not include a user-data cache. Joe Watkins recently started the APCu project to extract the user-data cache code from APC. Test this too!

Update 19 March 2013: Xinchen Hui is working on a lockless user-data cache, see https://github.com/laruence/yac and his blog about it here (in Chinese).

Tuesday Feb 12, 2013

The Mysterious PHP RFC Process and How You Can Change the Web

The PHP RFC process has been in place for a while, and users new to core PHP development are starting to use RFCs to propose desirable features.

Here are some personal observations and suggestions that show how I have seen feature acceptance and the RFC process work in practice. These notes augment the steps in How To Create an RFC. I hope they help set expectations about the PHP RFC process and feature acceptance in the PHP language.

  1. Before starting an RFC, review existing RFCs and search the mail list archives for similar ideas. Use this information to explain what your RFC will bring to the language.

  2. If you're new to PHP core development, mail the "internals" mail list to judge interest before you start your RFC. If you get no reaction (this is likely) or positive reaction then create the RFC. Core PHP developers with karma will know when they can skip this step.

  3. Avoid presenting an RFC idea to the "internals" mail list with email that begins "I don't know much about ... but ...". Do some research first.

  4. There are many really good ideas for improving PHP, however some of them are really tedious or technically risky or hard. If you are about to email the "internals" mail list saying "someone should do ...", then don't hit "Send". Work out how you could do it, and then send a patch.

  5. If the overall goals of PHP are not clear to you, you may not be alone. However, as an RFC writer you need to learn the general trajectory of the language and where web development is heading. You then need to enthuse everyone with your feature and show what it brings to the language.

  6. Don't start an RFC (or mail list discussion) about standardizing PHP function names and function argument orders. There are several historical reasons why the functions are what they are (including compatibility with underlying libraries), and many good reasons why the change would be counter-productive causing more end-user confusion, would lead to unmaintainable PHP engine code, and generally be a waste of everyone's time when they could be doing more interesting projects. This has been discussed ad infinitum. Review previous discussions and feel free to fork PHP on github.

  7. Your RFC should talk about all PHP areas that will be affected: php.ini, different SAPIs, engine, extensions, etc. List similar features. List similar features in other languages. Link to references. Give an estimate of the actual positive impact to user code. Put the proposed voting options in the RFC so they can be included in its discussion. Include the proposed acceptance criteria (2/3 or 50%+1 majority - see the original Voting RFC) to avoid later argument.

  8. Your RFC will be used to create the PHP documentation, so make its sections re-usable. Explain concepts and details. Keep the RFC technical and have plenty of examples.

  9. Many current PHP RFCs don't contain enough detail, nor do they capture the pros & cons that were discussed on the mail list. You can, and should, do better than these RFCs.

  10. An implementation is not mandatory while proposing an RFC but it can help persuade people and help fine-tune the design. Note that if you are unable to implement the RFC, and no one else volunteers during the discussion period, then your RFC is unlikely ever to be implemented.

  11. Don't start an implementation too early. Get support for the feature concept from the "internals" mail list before spending time on coding - unless you want a learning experience.

  12. If you do have an implementation, make it clear whether the implementation is a simple prototype or is expected to be the final code. This is specially important during the vote.

  13. It's easy to get sidetracked by mail list trolling or well intentioned but over-eager discussion. Take advice on board but make sure your feature doesn't end up being designed by a committee. Don't blame the "PHP core community" for derailing discussions when the issues of non- code-contributors on the mail list are at fault.

  14. For every user who wants a backwardly incompatible change to PHP, there is a user who will scream that this will be disastrous to them. As the RFC writer you need to walk the line between these groups. At voting time, other people may see the line in a different place than you do.

  15. There is no need to respond to every discussion email individually. You should batch up your responses and manage the discussion intelligently.

  16. Don't get side tracked by how long it will take a feature to reach end users. It will reach them eventually. You can always supply an unofficial patch or maybe create a PECL extension for users of current versions of PHP.

  17. Don't let mail list discussion drag on too long. High levels of discussion can be a symptom that an RFC is contentious. Having an early vote about only the feature's concept can prevent over-investment in an RFC and implementation.

  18. With long, fragmented discussions, not everyone will read every email. Update the RFC at regular intervals, and let people know what has changed. Include the RFC URL in all your emails so anyone dipping into a mail thread has a better chance of finding the RFC.

  19. PHP is popular so you can often find support from "many" people for cool language features. This doesn't mean those features are a "good thing" for the language or its implementation.

  20. There are multiple PHP communities. Don't interpret support in one community as universal support. Listen carefully to the key PHP developers on the "internals" mail list (they may not be the ones doing the most discussion). They are the people who will undoubtedly end up doing final integration of the feature (e.g. with the op-code cache), fixing edge cases, and maintaining it in the long term. They are the ones that have seen the PHP code base complexity grow and become more difficult to maintain and extend. Even if they like your idea, they may not be able to contribute time and effort to make it stable.

  21. The PHP core development community needs more end-users to become core language developers but the barrier to entry is high because of the complexity of the API, and because new features need very careful integration with a complex language. See the point above about existing developers not having enough time to support every good idea. This just means you need perseverance to become a core developer yourself. You can gain karma by being a regular code contributor before starting your magnum opus RFC.

  22. Some areas of PHP are complex or niche. Sometimes feature suggestions will be greeted by an apparent lack of interest. Don't be discouraged. This just means you need to take a stronger leadership role, and also prove your credentials by first working on the existing code base.

  23. Keep your RFC updated with a summary of all the positive and negative mail list discussion points and examples.

    • This prevents arguments being rehashed on the mail list each year.
    • Documentation will have clear use cases and show what does and doesn't work.
    • Future RFC writers can see trends and build on previous ideas.
  24. Before initiating a vote on your RFC, make sure the RFC contains enough detail so that the vote is solely on the RFC contents, not on any half-remembered or misinterpretable mail list discussions. Un- (or badly) expressed intentions will only cause you frustration when your RFC is rejected.

  25. Don't leave any ambiguity in the RFC. As well as stating what will be changed, also state what won't be changed. Ambiguity will hurt the chances of a successful outcome because voters will be unsure that the feature has been fully thought through. Make sure there are no "Open Issues" in the RFC when voting. Make a decision about the choices before opening the vote, or mark the issues clearly as "Future topics for exploration".

  26. Make sure any vote is clear about what is being voted on: is it the idea, the specification, or the implementation? Is a particular version of PHP being targeted? Is the implementation going to be merged to the core language or made as a PECL extension?

  27. Warn the "internals" mail list before starting the vote. This notification gives you a chance to fine tune the wording of your poll; this wording has caused contention in the past. The approaching deadline may also cause last-minute RFC responses. These may be significant enough to require further discussion.

  28. Set a deadline for the vote (sadly the current voting RFC doesn't require this). Having a deadline will forestall suggestions that a vote was deliberately left open until it succeeded, and then closed before the "negative" side could rally support.

  29. During the voting period, it is common for people to continue mail list discussion. You may need to halt the vote and address any issues.

  30. Prior to the end of the voting period, remind the mail list that the vote is about to close.

  31. Positive voting poll outcomes for RFs that are just "concepts" can be interpreted as support for "good ideas" rather than being a definitive guarantee that a feature will appear in a future version of PHP. As a feature is later investigated and further discussed, the early vote may become irrelevant. Similarly, where there is no chance of an implementation being written, a positive poll outcome is just an indicator of desire.

  32. If your RFC is rejected, it is not the end of the world. Sometimes ideas are submitted "before their time", and it takes an acclimatization period for other developers to see their value. As the core PHP developer base goes through natural turn-over, revisiting an idea after a (long) period may result in different appreciation. (This does not apply to some ideas, for example standardizing function names). See the previous points about becoming a core contributor - having karma goes a long way towards getting an RFC accepted, not only because experienced contributors know which RFCs are worth suggesting, and know how to make proposals understandable. When your RFC is rejected, continue working with the PHP core development community and revisit the RFC later.

  33. You can always fork PHP from github.

In summary, the PHP development process is an organic process and subject to flexibility. It is based heavily around users contributing features that they require. This therefore requires high investment from users who want change. Very rarely do PHP core developers have bandwidth to take on external ideas, no matter how good they are. Feature acceptance has to be pragmatic. The core PHP contributors would love to see more people get commit karma and contribute to PHP.

This post has gone on long enough. My notes are current for the time being. I'm sure there are observations or clarifications you can make. If you want to add anything, please post a comment.

(Note: this page has been augmented and re-ordered since first being published)

Thursday Jan 31, 2013

OS X Users! 11gR2 Oracle Instant Client 32 & 64-bit is now available

The Oracle 11g Release 2 (11.2.0.3) Database Instant Client for Apple OS X on Intel x86-64 is now available for download from OTN. It is supported on the two latest OS X releases: Lion (10.7) and Mountain Lion (10.8). It provides both 32-bit and 64-bit client support.

Oracle Instant Client is a simple bundle of libraries that client tools and programs (like PHP and Ruby) can link with. This allows those tools to access Oracle Databases.

Any issues with Instant Client can be posted to the OTN Instant Client Forum.

Tuesday Jan 29, 2013

MySQL Sessions at the ConFoo.ca conference

Who says direct marketing doesn't work? A personal request to blog the upcoming ConFoo conference (25 February - 1 March 2013 in Montreal, Canada) has, as you can see, been successful. Although it's been a few years since I spoke there, I recall how impressive the organization was. The diversity and continual growth trajectory of the conference over the years is a very good indicator that you should be involved.

While you're there, say Hi to Oracle Ace Director Sheeri Cabral who is giving a couple of MySQL talks.

Wednesday Jan 23, 2013

Quick Debugging of PHP Scripts in Emacs with Geben and Xdebug

When you want to test a PHP code snippet quickly, it's handy to do it within your day-to-day environment. For me, this environment is Emacs. The geben package for Emacs provides an interface to the DBGp protocol, such as used by Derick Rethans's standard Xdebug extension for PHP.

With the combination of geben and Xdebug, I can quickly and efficiently step through execution of local files, examining code flow and data values.

Working steps to install and use the debugger on command line PHP scripts are shown below. They are standard enough that they can be customized to your actual environment.

You probably already have Xdebug installed, since it is the common debugger and profiler used by most IDEs. Installing geben in Emacs is straight forward.

1. Download PHP 5.4 from http://php.net/downloads.php

2. Install PHP with:

$ tar -jxf php-5.4.11.tar.bz2
$ cd php-5.4.11
$ ./configure --prefix=/home/cjones/php54
$ make install
$ cp php.ini-development /home/cjones/php54/lib/php.ini

3. Download the source code for the Xdebug Debugger extension for PHP from http://xdebug.org/download.php

4. Install Xdebug into PHP with:

$ cd /home/cjones
$ tar -zxf xdebug-2.2.1.tgz
$ cd xdebug-2.2.1
$ export PATH=/home/cjones/php54/bin:$PATH
$ phpize
$ ./configure --enable-xdebug --with-php-config=/home/cjones/php54/bin/php-config
$ make install

More information is shown in http://xdebug.org/docs/install

5. Edit /home/cjones/php54/lib/php.ini and enable Xdebug by adding these lines:

zend_extension=/home/cjones/php54/lib/php/extensions/no-debug-non-zts-20100525/xdebug.so
xdebug.remote_enable=on
xdebug.remote_host=127.0.0.1

6. Install geben manually or use the Emacs 24 package repository (M-x package-list-packages)

To manually install geben, download it from http://code.google.com/p/geben-on-emacs/downloads/list and install it with:

$ cd /home/cjones
$ tar -zxf geben-0.26.tar.gz

8. Add the following to the Emacs initialization file /home/cjones/.emacs. When testing this blog post, my .emacs file only contained this code:

(setq load-path (cons "/home/cjones/geben-0.26" load-path))

(autoload 'geben "geben" "DBGp protocol frontend, a script debugger" t)

;; Debug a simple PHP script.
;; Change the session key my-php-54 to any session key text you like
(defun my-php-debug ()
  "Run current PHP script for debugging with geben"
  (interactive)
  (call-interactively 'geben)
  (shell-command
    (concat "XDEBUG_CONFIG='idekey=my-php-54' /home/cjones/php54/bin/php "
    (buffer-file-name) " &"))
  )

(global-set-key [f5] 'my-php-debug)

Experienced Emacs users will most likely use local-set-key in a php-mode hook to set a key mapping.

9. Start Emacs and load a PHP file:

$ emacs my.php

10. Press F5 to start the debugger. The script will open in geben mode and can be stepped through with the space bar.

Geben mode commands can be shown with '?'. They include 'v' for showing variables, 'c' for run-to-cursor, and 'g' for completing the program.

Geben can also be used for scripts called via a browser. Start the geben listener in Emacs with M-x geben and load a script in the browser, passing an Xdebug URL parameter setting the session name. This initiates the debug session in Emacs. E.g. Load http://localhost/my.php?XDEBUG_SESSION_START=my-php-54

11. When you are finished debugging, stop the debug server listener inside Emacs:

M-x geben-end

Tuesday Dec 11, 2012

Excitement! Updated Underground PHP and Oracle Manual is Available for Download

We're thrilled to have a major update of the free Underground PHP and Oracle Manual released on OTN.

The Underground PHP and Oracle Manual is designed to bridge the gap between the many PHP scripting language and the many Oracle Database books available. It contains unique material about PHP's OCI8 extension for Oracle Database, and about other components in the PHP-Oracle ecosystem. It shows PHP developers how to use PHP and Oracle together, efficiently and easily.

The book has been completely refreshed. It has been updated for Oracle XE 11g and the latest PHP OCI8 extension. There are new chapters about using PHP with Oracle TimesTen, NetBeans and Oracle Tuxedo. There is also a new chapter about installing PHP on Oracle Solaris. The book now clocks in at 347 pages of great content.

Acknowledgements are due to all those who have helped with this and previous editions of the book. Thanks to the product teams that assisted with brand new content. In particular Craig Mohrman contributed the chapter about PHP on Solaris. Jeffry Rubinoff contributed the base text for the chapter on PHP and NetBeans.

Thursday Dec 06, 2012

Adding DTrace Probes to PHP Extensions

The powerful DTrace tracing facility has some PHP-specific probes that can be enabled with --enable-dtrace.

DTrace for Linux is being created by Oracle and is currently in tech preview. Currently it doesn't support userspace tracing so, in the meantime, Systemtap can be used to monitor the probes implemented in PHP. This was recently outlined in David Soria Parra's post Probing PHP with Systemtap on Linux.

My post shows how DTrace probes can be added to PHP extensions and traced on Linux. I was using Oracle Linux 6.3.

Not all Linux kernels are built with Systemtap, since this can impact stability. Check whether your running kernel (or others installed) have Systemtap enabled, and reboot with such a kernel:

# grep CONFIG_UTRACE /boot/config-`uname -r`
# grep CONFIG_UTRACE /boot/config-*

When you install Systemtap itself, the package systemtap-sdt-devel is needed since it provides the sdt.h header file:

# yum install systemtap-sdt-devel

You can now install and build PHP as shown in David's article. Basically the build is with:

$ cd ~/php-src
$ ./configure --disable-all --enable-dtrace
$ make

(For me, running 'make' a second time failed with an error. The workaround is to do 'git checkout Zend/zend_dtrace.d' and then rerun 'make'. See PHP Bug 63704)

David's article shows how to trace the probes already implemented in PHP. You can also use Systemtap to trace things like userspace PHP function calls. For example, create test.php:

<?php

$c = oci_connect('hr', 'welcome', 'localhost/orcl');
$s = oci_parse($c, "select dbms_xmlgen.getxml('select * from dual') xml from dual");
$r = oci_execute($s);
$row = oci_fetch_array($s, OCI_NUM);
$x = $row[0]->load();
$row[0]->free();
echo $x;

?>

The normal output of this file is the XML form of Oracle's DUAL table:

$ ./sapi/cli/php ~/test.php
<?xml version="1.0"?>
<ROWSET>
 <ROW>
  <DUMMY>X</DUMMY>
 </ROW>
</ROWSET>

To trace the PHP function calls, create the tracing file functrace.stp:

probe process("sapi/cli/php").function("zif_*") {
    printf("Started function %s\n", probefunc());
}

probe process("sapi/cli/php").function("zif_*").return {
    printf("Ended function %s\n", probefunc());
}

This makes use of the way PHP userspace functions (not builtins) like oci_connect() map to C functions with a "zif_" prefix.

Login as root, and run System tap on the PHP script:

# cd ~cjones/php-src
# stap -c 'sapi/cli/php ~cjones/test.php' ~cjones/functrace.stp
Started function zif_oci_connect
Ended function zif_oci_connect
Started function zif_oci_parse
Ended function zif_oci_parse
Started function zif_oci_execute
Ended function zif_oci_execute
Started function zif_oci_fetch_array
Ended function zif_oci_fetch_array
Started function zif_oci_lob_load
<?xml version="1.0"?>
<ROWSET>
 <ROW>
  <DUMMY>X</DUMMY>
 </ROW>
</ROWSET>
Ended function zif_oci_lob_load
Started function zif_oci_free_descriptor
Ended function zif_oci_free_descriptor

Each call and return is logged. The Systemtap scripting language allows complex scripts to be built. There are many examples on the web.

To augment this generic capability and the PHP probes in PHP, other extensions can have probes too. Below are the steps I used to add probes to OCI8:

  1. I created a provider file ext/oci8/oci8_dtrace.d, enabling three probes. The first one will accept a parameter that runtime tracing can later display:

    provider php {
    	probe oci8__connect(char *username);
    	probe oci8__nls_start();
    	probe oci8__nls_done();
    };
    
  2. I updated ext/oci8/config.m4 with the PHP_INIT_DTRACE macro. The patch is at the end of config.m4. The macro takes the provider prototype file, a name of the header file that 'dtrace' will generate, and a list of sources files with probes. When --enable-dtrace is used during PHP configuration, then the outer $PHP_DTRACE check is true and my new probes will be enabled. I've chosen to define an OCI8 specific macro, HAVE_OCI8_DTRACE, which can be used in the OCI8 source code:

    diff --git a/ext/oci8/config.m4 b/ext/oci8/config.m4
    index 34ae76c..f3e583d 100644
    --- a/ext/oci8/config.m4
    +++ b/ext/oci8/config.m4
    @@ -341,4 +341,17 @@ if test "$PHP_OCI8" != "no"; then
         PHP_SUBST_OLD(OCI8_ORACLE_VERSION)
     
       fi
    +
    +  if test "$PHP_DTRACE" = "yes"; then
    +     AC_CHECK_HEADERS([sys/sdt.h], [
    +       PHP_INIT_DTRACE([ext/oci8/oci8_dtrace.d],
    +                       [ext/oci8/oci8_dtrace_gen.h],[ext/oci8/oci8.c])
    +         AC_DEFINE(HAVE_OCI8_DTRACE,1,
    +         [Whether to enable DTrace support for OCI8 ])
    +     ], [
    +       AC_MSG_ERROR(
    +         [Cannot find sys/sdt.h which is required for DTrace support])
    +     ])
    +   fi
    +
     fi
    
  3. In ext/oci8/oci8.c, I added the probes at, for this example, semi-arbitrary places:

    diff --git a/ext/oci8/oci8.c b/ext/oci8/oci8.c
    index e2241cf..ffa0168 100644
    --- a/ext/oci8/oci8.c
    +++ b/ext/oci8/oci8.c
    @@ -1811,6 +1811,12 @@ php_oci_connection *php_oci_do_connect_ex(char *username, int username_len, char
     		}
     	}
     
    +#ifdef HAVE_OCI8_DTRACE
    +    if (DTRACE_OCI8_CONNECT_ENABLED()) {
    +		DTRACE_OCI8_CONNECT(username);
    +	}
    +#endif
    +
     	/* Initialize global handles if they weren't initialized before */
     	if (OCI_G(env) == NULL) {
     		php_oci_init_global_handles(TSRMLS_C);
    @@ -1870,11 +1876,22 @@ php_oci_connection *php_oci_do_connect_ex(char *username, int username_len, char
     		size_t rsize = 0;
     		sword result;
     
    +#ifdef HAVE_OCI8_DTRACE
    +		if (DTRACE_OCI8_NLS_START_ENABLED()) {
    +			DTRACE_OCI8_NLS_START();
    +		}
    +#endif
     		PHP_OCI_CALL_RETURN(result, OCINlsEnvironmentVariableGet, (&charsetid_nls_lang, 0, OCI_NLS_CHARSET_ID, 0, &rsize));
     		if (result != OCI_SUCCESS) {
     			charsetid_nls_lang = 0;
     		}
     		smart_str_append_unsigned_ex(&hashed_details, charsetid_nls_lang, 0);
    +
    +#ifdef HAVE_OCI8_DTRACE
    +		if (DTRACE_OCI8_NLS_DONE_ENABLED()) {
    +			DTRACE_OCI8_NLS_DONE();
    +		}
    +#endif
     	}
     
     	timestamp = time(NULL);
    

    The oci_connect(), oci_pconnect() and oci_new_connect() calls all use php_oci_do_connect_ex() internally. The first probe simply records that the PHP application made a connection call. I already showed a way to do this without needing a probe, but adding a specific probe lets me record the username. The other two probes can be used to time how long the globalization initialization takes.

    The relationships between the oci8_dtrace.d names like oci8__connect, the probe guards like DTRACE_OCI8_CONNECT_ENABLED() and probe names like DTRACE_OCI8_CONNECT() are obvious after seeing the pattern of all three probes.

    I included the new header that will be automatically created by the dtrace tool when PHP is built. I did this in ext/oci8/php_oci8_int.h:

    diff --git a/ext/oci8/php_oci8_int.h b/ext/oci8/php_oci8_int.h
    index b0d6516..c81fc5a 100644
    --- a/ext/oci8/php_oci8_int.h
    +++ b/ext/oci8/php_oci8_int.h
    @@ -44,6 +44,10 @@
     #  endif
     # endif /* osf alpha */
     
    +#ifdef HAVE_OCI8_DTRACE
    +#include "oci8_dtrace_gen.h"
    +#endif
    +
     #if defined(min)
     #undef min
     #endif
    
  4. Now PHP can be rebuilt:

    $ cd ~/php-src
    $ rm configure && ./buildconf --force
    $ ./configure --disable-all --enable-dtrace \
                  --with-oci8=instantclient,/home/cjones/instantclient
    $ make
    

    If 'make' fails, do the 'git checkout Zend/zend_dtrace.d' trick I mentioned.

  5. The new probes can be seen by logging in as root and running:

    # stap -l 'process.provider("php").mark("oci8*")' -c 'sapi/cli/php -i'
    process("sapi/cli/php").provider("php").mark("oci8__connect")
    process("sapi/cli/php").provider("php").mark("oci8__nls_done")
    process("sapi/cli/php").provider("php").mark("oci8__nls_start")
    
  6. To test them out, create a new trace file, oci.stp:

    global numconnects;
    global start;
    global numcharlookups = 0;
    global tottime = 0;
    probe process.provider("php").mark("oci8-connect") {
        printf("Connected as %s\n", user_string($arg1));
        numconnects += 1;
    }
    probe process.provider("php").mark("oci8-nls_start") {
        start = gettimeofday_us();
        numcharlookups++;
    }
    probe process.provider("php").mark("oci8-nls_done") {
        tottime += gettimeofday_us() - start;
    }
    probe end {
        printf("Connects: %d, Charset lookups: %ld\n", numconnects, numcharlookups);
        printf("Total NLS charset initalization time: %ld usecs/connect\n", 
                            (numcharlookups > 0 ? tottime/numcharlookups : 0));
    }
    

    This calculates the average time that the NLS character set lookup takes. It also prints out the username of each connection, as an example of using parameters.

  7. Login as root and run Systemtap over the PHP script:

    # cd ~cjones/php-src
    # stap -c 'sapi/cli/php ~cjones/test.php' ~cjones/oci.stp
    Connected as cj
    <?xml version="1.0"?>
    <ROWSET>
     <ROW>
      <DUMMY>X</DUMMY>
     </ROW>
    </ROWSET>
    Connects: 1, Charset lookups: 1
    Total NLS charset initalization time: 164 usecs/connect
    

    This shows the time penalty of making OCI8 look up the default character set. This time would be zero if a character set had been passed as the fourth argument to oci_connect() in test.php.

Update: To use real (non-SystemTap) DTrace, the extension binary needs to be built slightly differently than shown above in step 2. Instead of modifying config.m4 to reuse PHP_INIT_DTRACE, an OCI8-specific variant of that macro is created. To see how this is done, look at the new OCI8_INIT_DTRACE macro definition and way it is used in PHP OCI8 2.0's ext/oci8/config.m4.

About

Tourists looking out over an Opal mine
I'm a Product Manager in Server Technologies, working on scripting languages and developer-access.
Email: christopher.jones@oracle.com
Twitter: http://twitter.com/ghrd
Links: OTN Node.js Developer Center
OTN PHP Developer Center
Book: Free PHP Oracle book

Search

Archives
« May 2015
SunMonTueWedThuFriSat
     
1
2
3
4
5
6
7
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
      
Today