Wednesday Sep 04, 2013

DTrace with PHP Update

I've recently been working with the in-built DTrace code in PHP 5.4 and PHP 5.5, specifically stabilizing its configuration on Solaris and Oracle Linux.

DTrace is an always-available, low overhead, tracing framework that has just celebrated its 10th Birthday. DTrace support in PHP was originally via a separate PECL extension. In PHP 5.4 David Soria Parra (then at Sun) merged DTrace functionality to core PHP making it more accessible.

At the end of last year, I blogged about Adding DTrace Probes to PHP Extensions and how Linux's SystemTap could be used to trace the DTrace probe points. Since then, Oracle's Linux's DTrace project has been making great strides. The latest Oracle Linux UEK3 Beta kernel was just released. It comes with DTrace 0.4 and also supports "User-Level Statically Defined Tracing" (USDT) for the first time. This motivated me to make sure PHP DTrace worked well with "real" DTrace, not just with SystemTap's wrappers.

Some of the recent PHP DTrace changes were:

  • DTrace build script changes from PHP 5.5 were merged back to PHP 5.4 so both branches are now in sync.

  • A 'make install' recursive dependency issue that caused Zend/zend_dtrace.d to be deleted was fixed.

  • PHP DTrace configuration now uses the correct PIC or non-PIC objects. This also fixed building PHP when any extensions were built 'shared'.

  • A segmentation fault in the "error" probe was fixed.

  • PHP's OCI8 2.0 extension now builds correctly with "real" DTrace.

The changes were tested on Solaris and Oracle Linux. The core PHP changes should be available in the next monthly releases, PHP 5.4.20 and PHP 5.5.4. The PHP OCI8 2.0 changes are already available from PECL.

I'll cover more detail about using PHP and PHP OCI8 with DTrace on Oracle Linux in a later blog post, but the basic steps are:

  • Install the UEK3 kernel, currently in Beta from public-yum.oracle.com.

  • Install the dtrace-utils package with 'yum install dtrace-utils'

  • Download a PHP 5.4 or 5.5 snapshot from snaps.php.net (Or download PHP 5.4.20, PHP 5.5.4, or later, once they are available)

  • Configure PHP with the --enable-dtrace option

  • Load the DTrace module: modprobe fasttrap

  • Change the helper device to allow writing: chmod 666 /dev/dtrace/helper

  • Run a DTrace profiling script and your PHP application

Update: the detail is in a new blog post Using PHP DTrace on Oracle Linux

Thursday Jul 25, 2013

Using PHP and Oracle Database 12c Implicit Result Sets

Note: This post describes functionality in PHP OCI8 2.0.0-devel. Functionality and naming is subject to change.

The new Oracle Database 12c "Implicit Result Sets" (IRS) feature allows query results to be returned from a stored PL/SQL procedure (or a PL/SQL anonymous block) without requiring special PHP code. Support for IRS is available in PHP's OCI8 2.0.0-devel extension when it is compiled and used with Oracle Database 12c. (OCI8 2.0 can be compiled and used with other versions of Oracle Database but the available feature set is reduced).

Recall that a normal Oracle query can be performed in PHP with a parse-execute-fetch loop like:

<?php
$c = oci_connect('hr', 'welcome', 'localhost/pdborcl');
$sql = "select city from locations where rownum < 4";
$s = oci_parse($c, $sql);
oci_execute($s);
while (($row = oci_fetch_row($s)) != false) {
    foreach ($row as $item) {
        echo $item . " ";
    }
    echo "\n";
}
?>

The output is:

Beijing
Bern
Bombay

With PHP OCI8 2.0.0-devel and Oracle Database 12c, the same results can be obtained by changing $sql to an anonymous PL/SQL block (or by calling a previously created stored PL/SQL procedure) that uses DBMS_SQL.RETURN_RESULT like:

$sql =
 "declare
    c1 sys_refcursor;
  begin
    open c1 for select city from locations where rownum < 4;
    dbms_sql.return_result(c1);
  end;";

With this statement change, the previous PHP code fetches the query results without needing any logic alterations. In older versions without IRS, the PL/SQL and PHP code would have to handle a REF CURSOR parameter.

The real fun begins when you have multiple DBMS_SQL.RETURN_RESULT calls in the same PL/SQL block:

$sql =
 "declare
    c1 sys_refcursor;
  begin
    open c1 for select city from locations where rownum < 4;
    dbms_sql.return_result(c1);
    open c1 for select first_name, last_name from employees where rownum < 4;
    dbms_sql.return_result(c1);
  end;";

The PHP fetch loop handles this nicely and sequentially fetches rows from both queries:

Beijing  
Bern  
Bombay  
Ellen Abel  
Sundar Ande  
Mozhe Atkinson  

Only oci_fetch_array(), oci_fetch_assoc(), oci_fetch_object() and oci_fetch_row() (but not oci_fetch() or oci_fetch_all()) will automatically fetch IRS data like this. [This is a semi-arbitrary decision trying to balance the increased amount of code complexity and testing versus the expected use of the feature. If there is strong demand this decision can be revisited.]

To process each of the query results independently in PHP, use the newly introduced oci_get_implicit_resultset() function. This takes the parent statement resource, e.g. $s, and returns a PHP statement resource corresponding to the first result set. Each time oci_get_implicit_resultset() is subsequently called, it returns the next result set. When there are no more result sets, it returns false. Because oci_get_implicit_resultset() returns a statement resource, you can use any of the oci_fetch_* functions to get rows.

For example, to print appropriate column names above each row's items:

<?php
$c = oci_connect('hr', 'welcome', 'localhost/pdborcl');
$sql =
 "declare
    c1 sys_refcursor;
  begin
    open c1 for select city from locations where rownum < 4;
    dbms_sql.return_result(c1);
    open c1 for select first_name, last_name from employees where rownum < 4;
    dbms_sql.return_result(c1);
  end;";
$s = oci_parse($c, $sql);
oci_execute($s);
while (($s2 = oci_get_implicit_resultset($s))) {
    // Now treat $s2 as a distinct query statement resource
    $ncols = oci_num_fields($s2);
    for ($i = 1; $i <= $ncols; ++$i) {
        $colname = oci_field_name($s2, $i);
        echo $colname . " ";
    }
    echo "\n";
    while (($row = oci_fetch_row($s2)) != false) {
        foreach ($row as $item) {
            echo $item . " ";
        }
        echo "\n";
    }
}
?>

The output is:

CITY  
Beijing  
Bern  
Bombay  
FIRST_NAME LAST_NAME  
Ellen Abel  
Sundar Ande  
Mozhe Atkinson  

You can also do things like calling oci_set_prefetch() on the IRS statement resources, or calling oci_get_implicit_resultset() multiple times before beginning to fetch row data:

$sql =
 "declare
    c1 sys_refcursor;
  begin
    open c1 for select city from locations where rownum < 4;
    dbms_sql.return_result(c1);
    open c1 for select first_name, last_name from employees where rownum < 4;
    dbms_sql.return_result(c1);
  end;";
$s = oci_parse($c, $sql);
oci_execute($s);
$s1 = oci_get_implicit_resultset($s);
$s2 = oci_get_implicit_resultset($s);
$row1_1 = oci_fetch_row($s1);
$row2_1 = oci_fetch_row($s2);
$row1_2 = oci_fetch_row($s1);
$row2_2 = oci_fetch_row($s2);
. . .

There are more examples in the OCI8 test suite (see the tests sub-directory of the source bundle).

When would you used an IRS? Some cases are:

  • Migrating an application to Oracle Database, where the application currently makes use of stored procedures returning result sets. The IRS support in OCI8 will make migration easier.
  • Instead of calling a PL/SQL procedure and then executing a separate SELECT in PHP, complex processing can be performed in a stored PL/SQL procedure before that procedure returns results to PHP. This architecture might help system scalability since it reduces the number of calls made to the database.
  • PHP Frameworks typically use a basic oci_fetch_* call, so they will will automatically be able fetch IRS results.

I can't suggest turning every individual SELECT statement into an IRS since this will add the overhead of the PL/SQL-to-SQL calls. However there are legitimate use cases for Implicit Result Sets that open some interesting possibilities.

PHP OCI8 2.0.0-devel for Oracle Database is on PECL

PHP OCI8 2.0.0-devel is now available on PECL (and in the PHP 'master' branch). This is a development release, so changes may occur. I'm looking for feedback, particularly on:

  • Feature design, including naming choices
  • Install feedback from different platforms with different build tool versions
  • Whether to drop support for installing on PHP 4. I'm likely to do this. (The OCI8 1.4.10 release is usable for installing on PHP 4)

OCI8 2.0 can be installed as a shared extension from PECL with:

pecl install oci8-devel

Alternatively, if you have OCI8 1.x currently installed as a static extension, then you will need to build PHP again. This is easy using a trunk snapshot from snaps.php.net. Instructions for building are in various places on the web, including in The Underground PHP and Oracle Manual.

The OCI8 2.0.0-devel release contains a bunch of clean ups, and some new and changed features. The full list is found in the package.xml file:

    - NEW FUNCTIONALITY:
 
      - Added Implicit Result Set support for Oracle Database 12c.
	Streaming of all IRS's returned from a PL/SQL block is available
	via oci_fetch_array, oci_fetch_assoc, oci_fetch_object and
	oci_fetch_row (but not oci_fetch or oci_fetch_all).
	Alternatively individual IRS statement resources can be obtained
	with the new function 'oci_get_implicit_resultset' and passed to
	any oci_fetch_* function.

      - Added DTrace probes enabled with PHP's generic --enable-dtrace

    - IMPROVED FUNCTIONALITY:
 
      - Using 'oci_execute($s, OCI_NO_AUTO_COMMIT)' for a SELECT no
	longer unnecessarily initiates an internal ROLLBACK during
	connection close.  This can improve overall scalability by
	reducing "round trips" between PHP and the database.
 
    - CHANGED FUNCTIONALITY:
 
      - PHPINFO() CHANGES:
 
        - The oci8.event and oci8.connection_class values are now shown
          only when the Oracle client libraries support the respective
          functionality.
 
        - Connection statistics are now in a separate phpinfo() table.
 
        - Temporary LOB and Collection support status lines in
          phpinfo() were removed.  These features have always been
          enabled since 2007.
 
      - OCI_INTERNAL_DEBUG() CHANGES:
 
        - The oci_internal_debug() function is now a no-op.  Use PHP's
          --enable-dtrace functionality with DTrace or SystemTap instead.
 
    - INTERNAL CHANGES:
 
      - Fixed a potential NULL pointer dereference flagged by Parfait
        static code analysis.
 
      - Extended testing of existing OCI8 functionality.
 
      - Improved test output portability when using the PHP development
        web server to run tests.
 
      - Removed no-longer necessary Unicode patterns from tests
        (vestiges of PHP's previous PHP 6 project)
        
      - Improved build portability by removing compilation type cast
        warnings with some compilers.
 
      - Fixed compilation warnings when building with Oracle 9.2
        client libraries.
 
      - Updated code to use internal macro PHP_OCI_REGISTER_RESOURCE.
 
      - Regularized code prototypes and fixed some in-line documentation
        prototypes.
 
      - Fixed code folding. 

Subsequent blog posts will talk about the features in OCI8 2.0.0.

Thursday May 16, 2013

Offline Processing in PHP with Advanced Queuing

Offloading slow batch tasks to an external process is a common method of improving website responsiveness. One great way to initiate such background tasks in PHP is to use Oracle Streams Advanced Queuing in a producer-consumer message passing fashion. Oracle AQ is highly configurable. Messages can queued by multiple producers. Different consumers can filter messages. From PHP, the PL/SQL interface to AQ is used. There are also Java, C and HTTPS interfaces, allowing wide architectural freedom. Oracle Advanced Queuing is included in all editions of the database.

The following example simulates an application user registration system where the PHP application queues each new user's street address. An external system monitoring the queue can then fetch and process that address. In real life the external system might initiate a snail-mail welcome letter, or do further, slower automated validation on the address.

The following SQL*Plus script qcreate.sql creates a new Oracle user demoqueue with permission to create and use queues. A payload type for the address is created and a queue is set up for this payload.

-- qcreate.sql

connect / as sysdba
drop user demoqueue cascade;

create user demoqueue identified by welcome;
grant connect, resource to demoqueue;
grant aq_administrator_role, aq_user_role to demoqueue;
grant execute on dbms_aq to demoqueue;
grant create type to demoqueue;

connect demoqueue/welcome@localhost/orcl

-- The data we want to queue
create or replace type user_address_type as object (
  name        varchar2(10),
  address     varchar2(50)
);
/

-- Create and start the queue
begin
 dbms_aqadm.create_queue_table(
   queue_table        =>  'demoqueue.addr_queue_tab',
   queue_payload_type =>  'demoqueue.user_address_type');
end;
/

begin
 dbms_aqadm.create_queue(
   queue_name         =>  'demoqueue.addr_queue',
   queue_table        =>  'demoqueue.addr_queue_tab');
end;
/

begin
 dbms_aqadm.start_queue(
   queue_name         => 'demoqueue.addr_queue',
   enqueue            => true);
end;
/

The script qhelper.sql creates two useful helper functions to enqueue and dequeue messages:

-- qhelper.sql
-- Helpful address enqueue/dequeue procedures

connect demoqueue/welcome@localhost/orcl

-- Put an address in the queue
create or replace procedure my_enq(name_p in varchar2, address_p in varchar2) as
  user_address       user_address_type;
  enqueue_options    dbms_aq.enqueue_options_t;
  message_properties dbms_aq.message_properties_t;
  enq_id             raw(16);
begin
  user_address := user_address_type(name_p, address_p);
  dbms_aq.enqueue(queue_name         => 'demoqueue.addr_queue',
                  enqueue_options    => enqueue_options,
                  message_properties => message_properties,
                  payload            => user_address,
                  msgid              => enq_id);
  commit;
end;
/
show errors

-- Get an address from the queue
create or replace procedure my_deq(name_p out varchar2, address_p out varchar2) as
  dequeue_options    dbms_aq.dequeue_options_t;
  message_properties dbms_aq.message_properties_t;
  user_address       user_address_type;
  enq_id             raw(16);
begin
  dbms_aq.dequeue(queue_name         => 'demoqueue.addr_queue',
                  dequeue_options    => dequeue_options,
                  message_properties => message_properties,
                  payload            => user_address,
                  msgid              => enq_id);
  name_p    := user_address.name;
  address_p := user_address.address;
  commit;
end;
/
show errors

The script newuser.php is the part of the PHP application that handles site registration for a new user. It queues a message containing their address and continues executing:

<?php
// newuser.php

$c = oci_connect("demoqueue", "welcome", "localhost/orcl");

// The new user details
$username = 'Fred';
$address  = '500 Oracle Parkway';

// Enqueue the address for later offline handling
$s = oci_parse($c, "begin my_enq(:username, :address); end;");
oci_bind_by_name($s, ":username", $username, 10);
oci_bind_by_name($s, ":address",  $address,  50);
$r = oci_execute($s);

// Continue executing
echo "Welcome $username\n";

?>

It executes an anonymous PL/SQL block to create and enqueue the address message. The immediate script output is simply the echoed welcome message:

Welcome Fred

Once this PHP script is executed, any application can dequeue the new message at its leisure. For example, the following SQL*Plus commands call the helper my_deq() dequeue function and displays the user details:

-- getuser.sql

connect demoqueue/welcome@localhost/orcl

set serveroutput on
declare
  name varchar2(10);
  address varchar2(50);
begin
  my_deq(name, address);
  dbms_output.put_line('Name     : ' || name);
  dbms_output.put_line('Address  : ' || address);
end;
/

The output is:

Name     : Fred
Address  : 500 Oracle Parkway

If you instead want to check the queue from PHP, use getuser.php:

<?php
// getuser.php

$c = oci_connect("demoqueue", "welcome", "localhost/orcl");

// dequeue the message
$sql = "begin my_deq(:username, :address); end;";
$s = oci_parse($c, $sql);
oci_bind_by_name($s, ":username", $username, 10);
oci_bind_by_name($s, ":address", $address, 50);
$r = oci_execute($s);

echo "Name     : $username\n";
echo "Address  : $address\n";

?>

If the dequeue operation is called without anything in the queue, it will block waiting for a message until the queue wait time expires. This is configurable by setting a zero wait time dequeue_options.wait := 0; before calling dbms_aq.dequeue.

The PL/SQL API has much more functionality than shown in this overview. For example you can enqueue an array of messages, or listen to more than one queue. Queuing is highly configurable and scalable, providing a great way to distribute workload for web or mobile applications. More information about AQ is in the Oracle Streams Advanced Queuing User's Guide.

Bootnote: The basis for this blog post comes from the Underground PHP and Oracle Manual

This post was updated to show setting a zero wait time.

Tuesday May 14, 2013

Getting Started with PHP Zend Framework 2 for Oracle DB

This post shows the changes to the ZF2 tutorial application to allow it to run with Oracle Database 11gR2.

Oracle Database SQL identifiers are case insensitive by default so "select Abc from Xyz" is the same as "select abc from xyz". However the identifier metadata returned to programs like PHP is standardized to uppercase by default. After executing either query PHP knows that column "ABC" was selected from table "XYZ".

In PHP code, array indices and object attributes need to match the schema identifier case that is returned by the database. This is either done by using uppercase indices and attributes in the PHP code, or by forcing the SQL schema to case-sensitively use lower-case names.

The former approach is more common, and is shown here.

The instructions for creating the sample ZF2 application are here. Follow those steps as written, making the substitutions shown below.

Schema

In Oracle 11gR2, the schema can be created like:

DROP USER ZF2 CASCADE;

CREATE USER ZF2 IDENTIFIED BY WELCOME
    DEFAULT TABLESPACE USERS QUOTA UNLIMITED ON USERS
    TEMPORARY TABLESPACE TEMP;

GRANT CREATE SESSION
    , CREATE TABLE
    , CREATE PROCEDURE
    , CREATE SEQUENCE
    , CREATE TRIGGER
    , CREATE VIEW
    , CREATE SYNONYM
    , ALTER SESSION
TO ZF2;

CONNECT ZF2/WELCOME

CREATE TABLE ALBUM (
  ID NUMBER NOT NULL,
  ARTIST VARCHAR2(100) NOT NULL,
  TITLE VARCHAR2(100) NOT NULL,
  PRIMARY KEY (ID)
);

CREATE SEQUENCE ALBUMSEQ;

CREATE TRIGGER ALBUMTRIGGER BEFORE INSERT ON ALBUM FOR EACH ROW
BEGIN
  :NEW.ID := ALBUMSEQ.NEXTVAL;
END;
/

INSERT INTO ALBUM (ARTIST, TITLE)
    VALUES ('The  Military  Wives', 'In  My  Dreams');
INSERT INTO ALBUM (ARTIST, TITLE)
    VALUES ('Adele', '21');
INSERT INTO ALBUM (ARTIST, TITLE)
    VALUES ('Bruce  Springsteen', 'Wrecking Ball (Deluxe)');
INSERT INTO ALBUM (ARTIST, TITLE)
    VALUES ('Lana  Del  Rey', 'Born  To  Die');
INSERT INTO ALBUM (ARTIST, TITLE)
    VALUES ('Gotye', 'Making  Mirrors');

COMMIT;

Driver and Credentials

The driver and credentials are Oracle-specific. Always use the OCI8 adapter in ZF, since it is more stable and has better scalability. Specifying a character set will make connection faster.

zf2-tutorial/config/autoload/global.php:
 return array(
     'db' => array(
-        'driver'         => 'Pdo',
-        'dsn'            => 'mysql:dbname=zf2tutorial;host=localhost',
-        'driver_options' => array(
-            PDO::MYSQL_ATTR_INIT_COMMAND => 'SET NAMES \'UTF8\''
-        ),
+        'driver'         => 'OCI8',
+        'connection_string' => 'localhost/orcl',
+        'character_set' => 'AL32UTF8',
     ),
     'service_manager' => array(
         'factories' => array(
zf2-tutorial/config/autoload/local.php:
 return array(
     'db' => array(
-        'username' => 'YOUR USERNAME HERE',
-        'password' => 'YOUR USERNAME HERE',
+        'username' => 'ZF2',
+        'password' => 'WELCOME',
     ),
     // Whether or not to enable a configuration cache.
     // If enabled, the merged configuration will be cached and used in

Attribute & Index Changes

The rest of the application changes are just to handle the case of the Oracle identifiers correctly.

zf2-tutorial/module/Album/Module.php
                     $dbAdapter = $sm->get('Zend\Db\Adapter\Adapter');
                     $resultSetPrototype = new ResultSet();
                     $resultSetPrototype->setArrayObjectPrototype(new Album());
-                    return new TableGateway('album', $dbAdapter, null, $resultSetPrototype);
+                    return new TableGateway('ALBUM', $dbAdapter, null, $resultSetPrototype);
                 },
             ),
         );
zf2-tutorial/module/Album/view/album/album/add.phtml
 $form->prepare();
 
 echo $this->form()->openTag($form);
-echo $this->formHidden($form->get('id'));
-echo $this->formRow($form->get('title'));
-echo $this->formRow($form->get('artist'));
+echo $this->formHidden($form->get('ID'));
+echo $this->formRow($form->get('TITLE'));
+echo $this->formRow($form->get('ARTIST'));
 echo $this->formSubmit($form->get('submit'));
 echo $this->form()->closeTag();
zf2-tutorial/module/Album/view/album/album/delete.phtml
 <h1><?php echo $this->escapeHtml($title); ?></h1>
 
 <p>Are you sure that you want to delete
-'<?php echo $this->escapeHtml($album->title); ?>' by
-'<?php echo $this->escapeHtml($album->artist); ?>'?
+'<?php echo $this->escapeHtml($album->TITLE); ?>' by
+'<?php echo $this->escapeHtml($album->ARTIST); ?>'?
 </p>
 <?php
 $url = $this->url('album', array(
            'action' => 'delete',
-           'id'     => $this->id,
+           'id'     => $this->ID,
        ));
 ?>
 <form action="<?php echo $url; ?>" method="post">
     <div>
-    <input type="hidden" name="id" value="<?php echo (int) $album->id; ?>" />
+    <input type="hidden" name="id" value="<?php echo (int) $album->ID; ?>" />
     <input type="submit" name="del" value="Yes" />
     <input type="submit" name="del" value="No" />
     </div>
zf2-tutorial/module/Album/view/album/album/edit.phtml
         'album',
         array(
             'action' => 'edit',
-            'id'     => $this->id,
+            'id'     => $this->ID,
         )
     ));
 $form->prepare();
 
 echo $this->form()->openTag($form);
-echo $this->formHidden($form->get('id'));
-echo $this->formRow($form->get('title'));
-echo $this->formRow($form->get('artist'));
+echo $this->formHidden($form->get('ID'));
+echo $this->formRow($form->get('TITLE'));
+echo $this->formRow($form->get('ARTIST'));
 echo $this->formSubmit($form->get('submit'));
 echo $this->form()->closeTag();
zf2-tutorial/module/Album/view/album/album/index.phtml
</tr>
 <?php foreach ($albums as $album) : ?>
 <tr>
-<td><?php echo $this->escapeHtml($album->title);?></td>
-<td><?php echo $this->escapeHtml($album->artist);?></td>
+<td><?php echo $this->escapeHtml($album->TITLE);?></td>
+<td><?php echo $this->escapeHtml($album->ARTIST);?></td>
 <td>
 <a href="<?php echo $this->url('album',
-            array('action'=>'edit', 'id' => $album->id));?>">Edit</a>
+            array('action'=>'edit', 'id' => $album->ID));?>">Edit</a>
     <a href="<?php echo $this->url('album',
-            array('action'=>'delete', 'id' => $album->id));?>">Delete</a>
+            array('action'=>'delete', 'id' => $album->ID));?>">Delete</a>
     </td>
     </tr>
 <?php endforeach; ?>
zf2-tutorial/module/Album/src/Album/Model/Album.php
 class Album
 {
-    public $id;
-    public $artist;
-    public $title;
+    public $ID;
+    public $ARTIST;
+    public $TITLE;
     protected $inputFilter;
 
     public function exchangeArray($data)
     {
-        $this->id     = (!empty($data['id'])) ? $data['id'] : null;
-        $this->artist = (!empty($data['artist'])) ? $data['artist'] : null;
-        $this->title  = (!empty($data['title'])) ? $data['title'] : null;
+        $this->ID     = (!empty($data['ID'])) ? $data['ID'] : null;
+        $this->ARTIST = (!empty($data['ARTIST'])) ? $data['ARTIST'] : null;
+        $this->TITLE  = (!empty($data['TITLE'])) ? $data['TITLE'] : null;
     }
 
     public function getArrayCopy()

and

             $factory     = new InputFactory();
 
             $inputFilter->add($factory->createInput(array(
-                        'name'     => 'id',
+                        'name'     => 'ID',
                         'required' => true,
                         'filters'  => array(
                             array('name' => 'Int'),

and

                     )));
 
             $inputFilter->add($factory->createInput(array(
-                        'name'     => 'artist',
+                        'name'     => 'ARTIST',
                         'required' => true,
                         'filters'  => array(
                             array('name' => 'StripTags'),

and

                     )));
 
             $inputFilter->add($factory->createInput(array(
-                        'name'     => 'title',
+                        'name'     => 'TITLE',
                         'required' => true,
                         'filters'  => array(
                             array('name' => 'StripTags'),
zf2-tutorial/module/Album/src/Album/Model/AlbumTable.php
     public function getAlbum($id)
     {
         $id  = (int) $id;
-        $rowset = $this->tableGateway->select(array('id' => $id));
+        $rowset = $this->tableGateway->select(array('ID' => $id));
         $row = $rowset->current();
         if (!$row) {
             throw new \Exception("Could not find row $id");

and

     public function saveAlbum(Album $album)
     {
         $data = array(
-            'artist' => $album->artist,
-            'title'  => $album->title,
+            'ARTIST' => $album->ARTIST,
+            'TITLE'  => $album->TITLE,
         );
 
-        $id = (int)$album->id;
+        $id = (int)$album->ID;
         if ($id == 0) {
             $this->tableGateway->insert($data);
         } else {
             if ($this->getAlbum($id)) {
-                $this->tableGateway->update($data, array('id' => $id));
+                $this->tableGateway->update($data, array('ID' => $id));
             } else {
                 throw new \Exception('Form id does not exist');
             }

and

     public function deleteAlbum($id)
     {
-        $this->tableGateway->delete(array('id' => $id));
+        $this->tableGateway->delete(array('ID' => $id));
     }
 }
zf2-tutorial/module/Album/src/Album/Form/AlbumForm.php
         parent::__construct('album');
         $this->setAttribute('method', 'post');
         $this->add(array(
-                'name' => 'id',
+                'name' => 'ID',
                 'type' => 'Hidden',
             ));
         $this->add(array(
-                'name' => 'title',
+                'name' => 'TITLE',
                 'type' => 'Text',
                 'options' => array(
                     'label' => 'Title',
                 ),
             ));
         $this->add(array(
-                'name' => 'artist',
+                'name' => 'ARTIST',
                 'type' => 'Text',
                 'options' => array(
                     'label' => 'Artist',
zf2-tutorial/module/Album/src/Album/Controller/AlbumController.php
         }
 
         return array(
-            'id' => $id,
+            'ID' => $id,
             'form' => $form,
         );
     }

and

         }
 
         return array(
-            'id'    => $id,
+            'ID'    => $id,
             'album' => $this->getAlbumTable()->getAlbum($id)
         );
     }

When you create applications from scratch it will be straightforward to get it all working.

Friday Mar 15, 2013

Using PHP 5.5's New "OPcache" Opcode Cache

Zend have contributed their Zend Optimizer+ opcode cache to PHP - thanks Zend!!! (Update 19 March 2013: the renaming to "Zend OPcache" is complete)

"The Zend OPcache provides faster PHP execution through opcode caching and optimization."

The new OPcache extension can be seen as substitute for the venerable APC cache, the maintenance of which had become an issue. Note: although OPcache is now readily available, there is currently nothing preventing you from using any available (working!) opcode cache in PHP 5.5.

A few minutes ago Dmitry Stogov did the physical merge to the PHP 5.5 source's ext/opcache directory. The current PHP 5.5 snapshot has the code. Future Alpha or Beta (and Production) releases will include it too.

Please test OPcache. It is not a panacea for all performance problems. There are a lot of settings which may need adjusting. Understanding how it works and identifying issues during the stabilization phase of PHP 5.5 release process will greatly help.

To build Zend OPcache for PHP 5.5:

When you configure PHP, add the option --enable-opcache like:

./configure ... --enable-opcache

Then make and make install, as normal. This will build OPcache shared extension. It's not possible to build it statically.

Find the shared library in your installation directory with a command like find /home/cjones/php55 -name opcache.so

Edit php.ini and add the extension with its full path:

zend_extension=/home/cjones/php55/lib/php/extension/debug-non-zts-20121212/opcache.so

Update (25 March 2013): Dmitry merged a PHP 5.5 change so that the full path is not required for zend_extension libraries in the extension_dir directory. You can now simply do zend_extension=opcache.so.

You'll want to enable OPcache too:

opcache.enable=On

The ext/opcache/README is the current source of documentation, and lists all the other php.ini parameters.

Problems can be reported in the Github issue tracker

Update (18 March 2013): In a commit over the weekend, the build option --enable-opcache is On by default. You will still need to update php.ini.

To build Zend OPcache for older versions of PHP:

You should be able to build OPcache with PHP 5.2 onwards

Install it by getting the source from Github. There is also a PECL repository; this is slightly out of date so I recommend using Github. Follow the README instructions to install it.

User-data cache:

The new opcode cache does not include a user-data cache. Joe Watkins recently started the APCu project to extract the user-data cache code from APC. Test this too!

Update 19 March 2013: Xinchen Hui is working on a lockless user-data cache, see https://github.com/laruence/yac and his blog about it here (in Chinese).

Tuesday Feb 12, 2013

The Mysterious PHP RFC Process and How You Can Change the Web

The PHP RFC process has been in place for a while, and users new to core PHP development are starting to use RFCs to propose desirable features.

Here are some personal observations and suggestions that show how I have seen feature acceptance and the RFC process work in practice. These notes augment the steps in How To Create an RFC. I hope they help set expectations about the PHP RFC process and feature acceptance in the PHP language.

  1. Before starting an RFC, review existing RFCs and search the mail list archives for similar ideas. Use this information to explain what your RFC will bring to the language.

  2. If you're new to PHP core development, mail the "internals" mail list to judge interest before you start your RFC. If you get no reaction (this is likely) or positive reaction then create the RFC. Core PHP developers with karma will know when they can skip this step.

  3. Avoid presenting an RFC idea to the "internals" mail list with email that begins "I don't know much about ... but ...". Do some research first.

  4. There are many really good ideas for improving PHP, however some of them are really tedious or technically risky or hard. If you are about to email the "internals" mail list saying "someone should do ...", then don't hit "Send". Work out how you could do it, and then send a patch.

  5. If the overall goals of PHP are not clear to you, you may not be alone. However, as an RFC writer you need to learn the general trajectory of the language and where web development is heading. You then need to enthuse everyone with your feature and show what it brings to the language.

  6. Don't start an RFC (or mail list discussion) about standardizing PHP function names and function argument orders. There are several historical reasons why the functions are what they are (including compatibility with underlying libraries), and many good reasons why the change would be counter-productive causing more end-user confusion, would lead to unmaintainable PHP engine code, and generally be a waste of everyone's time when they could be doing more interesting projects. This has been discussed ad infinitum. Review previous discussions and feel free to fork PHP on github.

  7. Your RFC should talk about all PHP areas that will be affected: php.ini, different SAPIs, engine, extensions, etc. List similar features. List similar features in other languages. Link to references. Give an estimate of the actual positive impact to user code. Put the proposed voting options in the RFC so they can be included in its discussion. Include the proposed acceptance criteria (2/3 or 50%+1 majority - see the original Voting RFC) to avoid later argument.

  8. Your RFC will be used to create the PHP documentation, so make its sections re-usable. Explain concepts and details. Keep the RFC technical and have plenty of examples.

  9. Many current PHP RFCs don't contain enough detail, nor do they capture the pros & cons that were discussed on the mail list. You can, and should, do better than these RFCs.

  10. An implementation is not mandatory while proposing an RFC but it can help persuade people and help fine-tune the design. Note that if you are unable to implement the RFC, and no one else volunteers during the discussion period, then your RFC is unlikely ever to be implemented.

  11. Don't start an implementation too early. Get support for the feature concept from the "internals" mail list before spending time on coding - unless you want a learning experience.

  12. If you do have an implementation, make it clear whether the implementation is a simple prototype or is expected to be the final code. This is specially important during the vote.

  13. It's easy to get sidetracked by mail list trolling or well intentioned but over-eager discussion. Take advice on board but make sure your feature doesn't end up being designed by a committee. Don't blame the "PHP core community" for derailing discussions when the issues of non- code-contributors on the mail list are at fault.

  14. For every user who wants a backwardly incompatible change to PHP, there is a user who will scream that this will be disastrous to them. As the RFC writer you need to walk the line between these groups. At voting time, other people may see the line in a different place than you do.

  15. There is no need to respond to every discussion email individually. You should batch up your responses and manage the discussion intelligently.

  16. Don't get side tracked by how long it will take a feature to reach end users. It will reach them eventually. You can always supply an unofficial patch or maybe create a PECL extension for users of current versions of PHP.

  17. Don't let mail list discussion drag on too long. High levels of discussion can be a symptom that an RFC is contentious. Having an early vote about only the feature's concept can prevent over-investment in an RFC and implementation.

  18. With long, fragmented discussions, not everyone will read every email. Update the RFC at regular intervals, and let people know what has changed. Include the RFC URL in all your emails so anyone dipping into a mail thread has a better chance of finding the RFC.

  19. PHP is popular so you can often find support from "many" people for cool language features. This doesn't mean those features are a "good thing" for the language or its implementation.

  20. There are multiple PHP communities. Don't interpret support in one community as universal support. Listen carefully to the key PHP developers on the "internals" mail list (they may not be the ones doing the most discussion). They are the people who will undoubtedly end up doing final integration of the feature (e.g. with the op-code cache), fixing edge cases, and maintaining it in the long term. They are the ones that have seen the PHP code base complexity grow and become more difficult to maintain and extend. Even if they like your idea, they may not be able to contribute time and effort to make it stable.

  21. The PHP core development community needs more end-users to become core language developers but the barrier to entry is high because of the complexity of the API, and because new features need very careful integration with a complex language. See the point above about existing developers not having enough time to support every good idea. This just means you need perseverance to become a core developer yourself. You can gain karma by being a regular code contributor before starting your magnum opus RFC.

  22. Some areas of PHP are complex or niche. Sometimes feature suggestions will be greeted by an apparent lack of interest. Don't be discouraged. This just means you need to take a stronger leadership role, and also prove your credentials by first working on the existing code base.

  23. Keep your RFC updated with a summary of all the positive and negative mail list discussion points and examples.

    • This prevents arguments being rehashed on the mail list each year.
    • Documentation will have clear use cases and show what does and doesn't work.
    • Future RFC writers can see trends and build on previous ideas.
  24. Before initiating a vote on your RFC, make sure the RFC contains enough detail so that the vote is solely on the RFC contents, not on any half-remembered or misinterpretable mail list discussions. Un- (or badly) expressed intentions will only cause you frustration when your RFC is rejected.

  25. Don't leave any ambiguity in the RFC. As well as stating what will be changed, also state what won't be changed. Ambiguity will hurt the chances of a successful outcome because voters will be unsure that the feature has been fully thought through. Make sure there are no "Open Issues" in the RFC when voting. Make a decision about the choices before opening the vote, or mark the issues clearly as "Future topics for exploration".

  26. Make sure any vote is clear about what is being voted on: is it the idea, the specification, or the implementation? Is a particular version of PHP being targeted? Is the implementation going to be merged to the core language or made as a PECL extension?

  27. Warn the "internals" mail list before starting the vote. This notification gives you a chance to fine tune the wording of your poll; this wording has caused contention in the past. The approaching deadline may also cause last-minute RFC responses. These may be significant enough to require further discussion.

  28. Set a deadline for the vote (sadly the current voting RFC doesn't require this). Having a deadline will forestall suggestions that a vote was deliberately left open until it succeeded, and then closed before the "negative" side could rally support.

  29. During the voting period, it is common for people to continue mail list discussion. You may need to halt the vote and address any issues.

  30. Prior to the end of the voting period, remind the mail list that the vote is about to close.

  31. Positive voting poll outcomes for RFs that are just "concepts" can be interpreted as support for "good ideas" rather than being a definitive guarantee that a feature will appear in a future version of PHP. As a feature is later investigated and further discussed, the early vote may become irrelevant. Similarly, where there is no chance of an implementation being written, a positive poll outcome is just an indicator of desire.

  32. If your RFC is rejected, it is not the end of the world. Sometimes ideas are submitted "before their time", and it takes an acclimatization period for other developers to see their value. As the core PHP developer base goes through natural turn-over, revisiting an idea after a (long) period may result in different appreciation. (This does not apply to some ideas, for example standardizing function names). See the previous points about becoming a core contributor - having karma goes a long way towards getting an RFC accepted, not only because experienced contributors know which RFCs are worth suggesting, and know how to make proposals understandable. When your RFC is rejected, continue working with the PHP core development community and revisit the RFC later.

  33. You can always fork PHP from github.

In summary, the PHP development process is an organic process and subject to flexibility. It is based heavily around users contributing features that they require. This therefore requires high investment from users who want change. Very rarely do PHP core developers have bandwidth to take on external ideas, no matter how good they are. Feature acceptance has to be pragmatic. The core PHP contributors would love to see more people get commit karma and contribute to PHP.

This post has gone on long enough. My notes are current for the time being. I'm sure there are observations or clarifications you can make. If you want to add anything, please post a comment.

(Note: this page has been augmented and re-ordered since first being published)

Tuesday Jan 29, 2013

MySQL Sessions at the ConFoo.ca conference

Who says direct marketing doesn't work? A personal request to blog the upcoming ConFoo conference (25 February - 1 March 2013 in Montreal, Canada) has, as you can see, been successful. Although it's been a few years since I spoke there, I recall how impressive the organization was. The diversity and continual growth trajectory of the conference over the years is a very good indicator that you should be involved.

While you're there, say Hi to Oracle Ace Director Sheeri Cabral who is giving a couple of MySQL talks.

Wednesday Jan 23, 2013

Quick Debugging of PHP Scripts in Emacs with Geben and Xdebug

When you want to test a PHP code snippet quickly, it's handy to do it within your day-to-day environment. For me, this environment is Emacs. The geben package for Emacs provides an interface to the DBGp protocol, such as used by Derick Rethans's standard Xdebug extension for PHP.

With the combination of geben and Xdebug, I can quickly and efficiently step through execution of local files, examining code flow and data values.

Working steps to install and use the debugger on command line PHP scripts are shown below. They are standard enough that they can be customized to your actual environment.

You probably already have Xdebug installed, since it is the common debugger and profiler used by most IDEs. Installing geben in Emacs is straight forward.

1. Download PHP 5.4 from http://php.net/downloads.php

2. Install PHP with:

$ tar -jxf php-5.4.11.tar.bz2
$ cd php-5.4.11
$ ./configure --prefix=/home/cjones/php54
$ make install
$ cp php.ini-development /home/cjones/php54/lib/php.ini

3. Download the source code for the Xdebug Debugger extension for PHP from http://xdebug.org/download.php

4. Install Xdebug into PHP with:

$ cd /home/cjones
$ tar -zxf xdebug-2.2.1.tgz
$ cd xdebug-2.2.1
$ export PATH=/home/cjones/php54/bin:$PATH
$ phpize
$ ./configure --enable-xdebug --with-php-config=/home/cjones/php54/bin/php-config
$ make install

More information is shown in http://xdebug.org/docs/install

5. Edit /home/cjones/php54/lib/php.ini and enable Xdebug by adding these lines:

zend_extension=/home/cjones/php54/lib/php/extensions/no-debug-non-zts-20100525/xdebug.so
xdebug.remote_enable=on
xdebug.remote_host=127.0.0.1

6. Install geben manually or use the Emacs 24 package repository (M-x package-list-packages)

To manually install geben, download it from http://code.google.com/p/geben-on-emacs/downloads/list and install it with:

$ cd /home/cjones
$ tar -zxf geben-0.26.tar.gz

8. Add the following to the Emacs initialization file /home/cjones/.emacs. When testing this blog post, my .emacs file only contained this code:

(setq load-path (cons "/home/cjones/geben-0.26" load-path))

(autoload 'geben "geben" "DBGp protocol frontend, a script debugger" t)

;; Debug a simple PHP script.
;; Change the session key my-php-54 to any session key text you like
(defun my-php-debug ()
  "Run current PHP script for debugging with geben"
  (interactive)
  (call-interactively 'geben)
  (shell-command
    (concat "XDEBUG_CONFIG='idekey=my-php-54' /home/cjones/php54/bin/php "
    (buffer-file-name) " &"))
  )

(global-set-key [f5] 'my-php-debug)

Experienced Emacs users will most likely use local-set-key in a php-mode hook to set a key mapping.

9. Start Emacs and load a PHP file:

$ emacs my.php

10. Press F5 to start the debugger. The script will open in geben mode and can be stepped through with the space bar.

Geben mode commands can be shown with '?'. They include 'v' for showing variables, 'c' for run-to-cursor, and 'g' for completing the program.

Geben can also be used for scripts called via a browser. Start the geben listener in Emacs with M-x geben and load a script in the browser, passing an Xdebug URL parameter setting the session name. This initiates the debug session in Emacs. E.g. Load http://localhost/my.php?XDEBUG_SESSION_START=my-php-54

11. When you are finished debugging, stop the debug server listener inside Emacs:

M-x geben-end

Tuesday Dec 11, 2012

Excitement! Updated Underground PHP and Oracle Manual is Available for Download

We're thrilled to have a major update of the free Underground PHP and Oracle Manual released on OTN.

The Underground PHP and Oracle Manual is designed to bridge the gap between the many PHP scripting language and the many Oracle Database books available. It contains unique material about PHP's OCI8 extension for Oracle Database, and about other components in the PHP-Oracle ecosystem. It shows PHP developers how to use PHP and Oracle together, efficiently and easily.

The book has been completely refreshed. It has been updated for Oracle XE 11g and the latest PHP OCI8 extension. There are new chapters about using PHP with Oracle TimesTen, NetBeans and Oracle Tuxedo. There is also a new chapter about installing PHP on Oracle Solaris. The book now clocks in at 347 pages of great content.

Acknowledgements are due to all those who have helped with this and previous editions of the book. Thanks to the product teams that assisted with brand new content. In particular Craig Mohrman contributed the chapter about PHP on Solaris. Jeffry Rubinoff contributed the base text for the chapter on PHP and NetBeans.

Thursday Dec 06, 2012

Adding DTrace Probes to PHP Extensions

The powerful DTrace tracing facility has some PHP-specific probes that can be enabled with --enable-dtrace.

DTrace for Linux is being created by Oracle and is currently in tech preview. Currently it doesn't support userspace tracing so, in the meantime, Systemtap can be used to monitor the probes implemented in PHP. This was recently outlined in David Soria Parra's post Probing PHP with Systemtap on Linux.

My post shows how DTrace probes can be added to PHP extensions and traced on Linux. I was using Oracle Linux 6.3.

Not all Linux kernels are built with Systemtap, since this can impact stability. Check whether your running kernel (or others installed) have Systemtap enabled, and reboot with such a kernel:

# grep CONFIG_UTRACE /boot/config-`uname -r`
# grep CONFIG_UTRACE /boot/config-*

When you install Systemtap itself, the package systemtap-sdt-devel is needed since it provides the sdt.h header file:

# yum install systemtap-sdt-devel

You can now install and build PHP as shown in David's article. Basically the build is with:

$ cd ~/php-src
$ ./configure --disable-all --enable-dtrace
$ make

(For me, running 'make' a second time failed with an error. The workaround is to do 'git checkout Zend/zend_dtrace.d' and then rerun 'make'. See PHP Bug 63704)

David's article shows how to trace the probes already implemented in PHP. You can also use Systemtap to trace things like userspace PHP function calls. For example, create test.php:

<?php

$c = oci_connect('hr', 'welcome', 'localhost/orcl');
$s = oci_parse($c, "select dbms_xmlgen.getxml('select * from dual') xml from dual");
$r = oci_execute($s);
$row = oci_fetch_array($s, OCI_NUM);
$x = $row[0]->load();
$row[0]->free();
echo $x;

?>

The normal output of this file is the XML form of Oracle's DUAL table:

$ ./sapi/cli/php ~/test.php
<?xml version="1.0"?>
<ROWSET>
 <ROW>
  <DUMMY>X</DUMMY>
 </ROW>
</ROWSET>

To trace the PHP function calls, create the tracing file functrace.stp:

probe process("sapi/cli/php").function("zif_*") {
    printf("Started function %s\n", probefunc());
}

probe process("sapi/cli/php").function("zif_*").return {
    printf("Ended function %s\n", probefunc());
}

This makes use of the way PHP userspace functions (not builtins) like oci_connect() map to C functions with a "zif_" prefix.

Login as root, and run System tap on the PHP script:

# cd ~cjones/php-src
# stap -c 'sapi/cli/php ~cjones/test.php' ~cjones/functrace.stp
Started function zif_oci_connect
Ended function zif_oci_connect
Started function zif_oci_parse
Ended function zif_oci_parse
Started function zif_oci_execute
Ended function zif_oci_execute
Started function zif_oci_fetch_array
Ended function zif_oci_fetch_array
Started function zif_oci_lob_load
<?xml version="1.0"?>
<ROWSET>
 <ROW>
  <DUMMY>X</DUMMY>
 </ROW>
</ROWSET>
Ended function zif_oci_lob_load
Started function zif_oci_free_descriptor
Ended function zif_oci_free_descriptor

Each call and return is logged. The Systemtap scripting language allows complex scripts to be built. There are many examples on the web.

To augment this generic capability and the PHP probes in PHP, other extensions can have probes too. Below are the steps I used to add probes to OCI8:

  1. I created a provider file ext/oci8/oci8_dtrace.d, enabling three probes. The first one will accept a parameter that runtime tracing can later display:

    provider php {
    	probe oci8__connect(char *username);
    	probe oci8__nls_start();
    	probe oci8__nls_done();
    };
    
  2. I updated ext/oci8/config.m4 with the PHP_INIT_DTRACE macro. The patch is at the end of config.m4. The macro takes the provider prototype file, a name of the header file that 'dtrace' will generate, and a list of sources files with probes. When --enable-dtrace is used during PHP configuration, then the outer $PHP_DTRACE check is true and my new probes will be enabled. I've chosen to define an OCI8 specific macro, HAVE_OCI8_DTRACE, which can be used in the OCI8 source code:

    diff --git a/ext/oci8/config.m4 b/ext/oci8/config.m4
    index 34ae76c..f3e583d 100644
    --- a/ext/oci8/config.m4
    +++ b/ext/oci8/config.m4
    @@ -341,4 +341,17 @@ if test "$PHP_OCI8" != "no"; then
         PHP_SUBST_OLD(OCI8_ORACLE_VERSION)
     
       fi
    +
    +  if test "$PHP_DTRACE" = "yes"; then
    +     AC_CHECK_HEADERS([sys/sdt.h], [
    +       PHP_INIT_DTRACE([ext/oci8/oci8_dtrace.d],
    +                       [ext/oci8/oci8_dtrace_gen.h],[ext/oci8/oci8.c])
    +         AC_DEFINE(HAVE_OCI8_DTRACE,1,
    +         [Whether to enable DTrace support for OCI8 ])
    +     ], [
    +       AC_MSG_ERROR(
    +         [Cannot find sys/sdt.h which is required for DTrace support])
    +     ])
    +   fi
    +
     fi
    
  3. In ext/oci8/oci8.c, I added the probes at, for this example, semi-arbitrary places:

    diff --git a/ext/oci8/oci8.c b/ext/oci8/oci8.c
    index e2241cf..ffa0168 100644
    --- a/ext/oci8/oci8.c
    +++ b/ext/oci8/oci8.c
    @@ -1811,6 +1811,12 @@ php_oci_connection *php_oci_do_connect_ex(char *username, int username_len, char
     		}
     	}
     
    +#ifdef HAVE_OCI8_DTRACE
    +    if (DTRACE_OCI8_CONNECT_ENABLED()) {
    +		DTRACE_OCI8_CONNECT(username);
    +	}
    +#endif
    +
     	/* Initialize global handles if they weren't initialized before */
     	if (OCI_G(env) == NULL) {
     		php_oci_init_global_handles(TSRMLS_C);
    @@ -1870,11 +1876,22 @@ php_oci_connection *php_oci_do_connect_ex(char *username, int username_len, char
     		size_t rsize = 0;
     		sword result;
     
    +#ifdef HAVE_OCI8_DTRACE
    +		if (DTRACE_OCI8_NLS_START_ENABLED()) {
    +			DTRACE_OCI8_NLS_START();
    +		}
    +#endif
     		PHP_OCI_CALL_RETURN(result, OCINlsEnvironmentVariableGet, (&charsetid_nls_lang, 0, OCI_NLS_CHARSET_ID, 0, &rsize));
     		if (result != OCI_SUCCESS) {
     			charsetid_nls_lang = 0;
     		}
     		smart_str_append_unsigned_ex(&hashed_details, charsetid_nls_lang, 0);
    +
    +#ifdef HAVE_OCI8_DTRACE
    +		if (DTRACE_OCI8_NLS_DONE_ENABLED()) {
    +			DTRACE_OCI8_NLS_DONE();
    +		}
    +#endif
     	}
     
     	timestamp = time(NULL);
    

    The oci_connect(), oci_pconnect() and oci_new_connect() calls all use php_oci_do_connect_ex() internally. The first probe simply records that the PHP application made a connection call. I already showed a way to do this without needing a probe, but adding a specific probe lets me record the username. The other two probes can be used to time how long the globalization initialization takes.

    The relationships between the oci8_dtrace.d names like oci8__connect, the probe guards like DTRACE_OCI8_CONNECT_ENABLED() and probe names like DTRACE_OCI8_CONNECT() are obvious after seeing the pattern of all three probes.

    I included the new header that will be automatically created by the dtrace tool when PHP is built. I did this in ext/oci8/php_oci8_int.h:

    diff --git a/ext/oci8/php_oci8_int.h b/ext/oci8/php_oci8_int.h
    index b0d6516..c81fc5a 100644
    --- a/ext/oci8/php_oci8_int.h
    +++ b/ext/oci8/php_oci8_int.h
    @@ -44,6 +44,10 @@
     #  endif
     # endif /* osf alpha */
     
    +#ifdef HAVE_OCI8_DTRACE
    +#include "oci8_dtrace_gen.h"
    +#endif
    +
     #if defined(min)
     #undef min
     #endif
    
  4. Now PHP can be rebuilt:

    $ cd ~/php-src
    $ rm configure && ./buildconf --force
    $ ./configure --disable-all --enable-dtrace \
                  --with-oci8=instantclient,/home/cjones/instantclient
    $ make
    

    If 'make' fails, do the 'git checkout Zend/zend_dtrace.d' trick I mentioned.

  5. The new probes can be seen by logging in as root and running:

    # stap -l 'process.provider("php").mark("oci8*")' -c 'sapi/cli/php -i'
    process("sapi/cli/php").provider("php").mark("oci8__connect")
    process("sapi/cli/php").provider("php").mark("oci8__nls_done")
    process("sapi/cli/php").provider("php").mark("oci8__nls_start")
    
  6. To test them out, create a new trace file, oci.stp:

    global numconnects;
    global start;
    global numcharlookups = 0;
    global tottime = 0;
    probe process.provider("php").mark("oci8-connect") {
        printf("Connected as %s\n", user_string($arg1));
        numconnects += 1;
    }
    probe process.provider("php").mark("oci8-nls_start") {
        start = gettimeofday_us();
        numcharlookups++;
    }
    probe process.provider("php").mark("oci8-nls_done") {
        tottime += gettimeofday_us() - start;
    }
    probe end {
        printf("Connects: %d, Charset lookups: %ld\n", numconnects, numcharlookups);
        printf("Total NLS charset initalization time: %ld usecs/connect\n", 
                            (numcharlookups > 0 ? tottime/numcharlookups : 0));
    }
    

    This calculates the average time that the NLS character set lookup takes. It also prints out the username of each connection, as an example of using parameters.

  7. Login as root and run Systemtap over the PHP script:

    # cd ~cjones/php-src
    # stap -c 'sapi/cli/php ~cjones/test.php' ~cjones/oci.stp
    Connected as cj
    <?xml version="1.0"?>
    <ROWSET>
     <ROW>
      <DUMMY>X</DUMMY>
     </ROW>
    </ROWSET>
    Connects: 1, Charset lookups: 1
    Total NLS charset initalization time: 164 usecs/connect
    

    This shows the time penalty of making OCI8 look up the default character set. This time would be zero if a character set had been passed as the fourth argument to oci_connect() in test.php.

Update: To use real (non-SystemTap) DTrace, the extension binary needs to be built slightly differently than shown above in step 2. Instead of modifying config.m4 to reuse PHP_INIT_DTRACE, an OCI8-specific variant of that macro is created. To see how this is done, look at the new OCI8_INIT_DTRACE macro definition and way it is used in PHP OCI8 2.0's ext/oci8/config.m4.

Sunday Nov 25, 2012

How (and when) to move users to mysqli and PDO_MYSQL?

An important discussion on the PHP "internals" development mailing list is taking place. It's one that you should take some note of. It concerns the next step in transitioning PHP applications away from the very old mysql extension and towards adopting the much better mysqli extension or PDO_MYSQL driver for PDO. This would allow the mysql extension to, at some as-yet undetermined time in the future, be removed. Both mysqli and PDO_MYSQL have been around for many years, and have various advantages: http://php.net/manual/en/mysqlinfo.api.choosing.php

The initial RFC for this next step is at https://wiki.php.net/rfc/mysql_deprecation I would expect the RFC to change substantially based on current discussion. The crux of that discussion is the timing of the next step of deprecation. There is also discussion of the carrot approach (showing users the benfits of moving), and stick approach (displaying warnings when the mysql extension is used). As always, there is a lot of guesswork going on as to what MySQL APIs are in current use by PHP applications, how those applications are deployed, and what their upgrade cycle is. This is where you can add your weight to the discussion - and also help by spreading the word to move to mysqli or PDO_MYSQL. An example of such a 'carrot' is the excellent summary at Ulf Wendel's blog: http://blog.ulf-wendel.de/2012/php-mysql-why-to-upgrade-extmysql/

I want to repeat that no time frame for the eventual removal of the mysql extension is set. I expect it to be some years away.

Tuesday Mar 13, 2012

PHP-FPM FastCGI Process Manager with Apache 2

I've published some vanilla PHP 5.4.0 RPMs to make new feature testing easier.

Along with all the PHP 5.4 goodies, the php-fpm "FastCGI Process Manager" is available for the first time on oss.oracle.com. Php-fpm is an alternative FastCGI interface to PHP with various extra features such as load dependent spawning of processes. (For other features, see php-fpm.org). Php-fpm has been getting more and more traction in the PHP community and the EXPERIMENTAL flag was removed in PHP 5.4. You might want to test it out.

To use php-fpm with the default Apache web server, first install Oracle Linux 5.8 (64bit) using Oracle's free, public yum repository public-yum.oracle.com.

Install Apache, if not already installed:

  yum install httpd

Download and install the PHP 5.4 RPMs from oss.oracle.com/projects/php:

  rpm -i php54-common-5.4.0-1.el5.x86_64.rpm php54-fpm-5.4.0-1.el5.x86_64.rpm

Other extensions can also be installed, depending on the functionality you want to test.

Download and build FastCGI for Apache:

  wget http://www.fastcgi.com/dist/mod_fastcgi-current.tar.gz
  tar -zxf mod_fastcgi-current.tar.gz
  cd mod_fastcgi-2.4.6
  cp Makefile.AP2 Makefile
  make top_dir=/usr/lib64/httpd

Install FastCGI as root:

  make top_dir=/usr/lib64/httpd install

Edit /etc/httpd/conf/httpd.conf and comment out any existing references to PHP you might previously have added for testing:

# LoadModule php5_module        modules/libphp5.so
# AddType application/x-httpd-php .php

Add the php-fpm configuration to httpd.conf:

  LoadModule fastcgi_module modules/mod_fastcgi.so

  <IfModule mod_fastcgi.c>  
    FastCGIExternalServer /usr/sbin/php-fpm -host 127.0.0.1:9000
    AddHandler php-fastcgi .php  

    #<LocationMatch "/status">
    #  SetHandler php-fastcgi-virt
    #  Action php-fastcgi-virt /usr/sbin/php-fpm.fcgi virtual
    #</LocationMatch>

    Action php-fastcgi /usr/sbin/php-fpm.fcgi  
    ScriptAlias /usr/sbin/php-fpm.fcgi /usr/sbin/php-fpm  

    <Directory /usr/sbin>  
      Options ExecCGI FollowSymLinks  
      SetHandler fastcgi-script  
      Order allow,deny  
      Allow from all  
    </Directory>  
  </IfModule> 

Start php-fpm and Apache:

  service php-fpm start
  service httpd start

Test it out with your favorite script or create a file pi.php:

  <?php
    phpinfo();
  ?>

Save it in /var/www/html/pi.php or in $HOME/public_html/pi.php, if you have configured UserDir in httpd.conf.

In a browser load http://localhost/pi.php or http://localhost/~yourname/pi.php.

This will show the PHP configuration values.

To test php-fpm's built-in statistics, edit httpd.conf and uncomment the four lines of the LocationMatch section:

  <LocationMatch "/status">
    SetHandler php-fastcgi-virt
    Action php-fastcgi-virt /usr/sbin/php-fpm.fcgi virtual
  </LocationMatch>

Edit /etc/php-fpm.conf and uncomment the line:

  pm.status_path = /status

Restart php-fpm and Apache:

  service php-fpm restart
  service httpd restart

Run some load on the system:

  ab -c 10 -t 60 http://localhost/pi.php  

Now http://localhost/status gives you the status of the server:

  pool:                 www
  process manager:      dynamic
  start time:           13/Mar/2012:14:25:53 -0700
  start since:          26
  accepted conn:        50001
  listen queue:         0
  max listen queue:     6
  listen queue len:     128
  idle processes:       2
  active processes:     1
  total processes:      3
  max active processes: 5
  max children reached: 1

The php-fpm.conf file documents other output formats for the statistics. It also shows the extensive functionality available with php-fpm.

Documentation on php-fpm is not ideal, but you can see some at http://php.net/manual/en/install.fpm.php. The php-fpm.org site has more, including a forum and wiki.

Php-fpm is commonly used in conjunction with the nginx webserver. For this, you will need to compile nginx yourself. Because php-fpm is installed and managed separately from the webserver, the php-fpm RPM will still be usable. Now that Apache 2.4 has been released with claims of improved performance, it will be interesting to see if web server popularity swings back towards Apache.

Finally, remember that the PHP RPMs on oss.oracle.com are for testing purposes only. They are not supported.

Update: In the Underground PHP & Oracle Manual, I simplified the httpd.conf options to this (note PHP was installed in a different location):

<IfModule mod_fastcgi.c>
    FastCGIExternalServer /opt/php544/sbin/php-fpm -host 127.0.0.1:9000
    AddHandler php-fastcgi .php
    Action php-fastcgi /mycgi
    ScriptAlias /mycgi /opt/php544/sbin/php-fpm
</IfModule>

PHP 5.4.0 RPMs for 64bit Oracle Linux 5.x are available

I've published some vanilla PHP 5.4.0 RPMs to make new feature testing easier for Oracle Linux 5.x 64 bit users. The standard set of RPMs is at oss.oracle.com/projects/php. The OCI8 extension is also available (this requires the free Oracle Instant Client 11.2 from ULN or OTN.)

Some of the features of PHP 5.4 are:

  • Improved memory usage and performance. Some impressive preliminary reports of the benefits include: http://news.php.net/php.internals/57760 and http://news.php.net/php.internals/57747.

  • File Upload progress support is natively implemented.

  • Support for Traits now allows code reuse:

        trait t1 {
    	function m1() { echo "hi"; }
    	function m2() { echo "bye"; }
        }
    
        class my_c1 {
    	use t1;
    	/*...*/
        }
    
        class my_c2 extends c2 {
    	use t1;
    	/*...*/
        }
    
  • A built-in HTTP server for development is included:

      php -S 127.0.0.1:8888
    
  • Improvements were made to the interactive PHP shell (when PHP is compiled with readline).

  • A shortened array syntax was introduced: $a = [1,2,3];

  • The default character set for several internal functions was changed to UTF-8.

  • Support for multibyte languages is now configurable at run-time instead of compile-time.

  • The value echo tag "<?=" is now always on.

  • Binary number support was added.

  • DTrace support was added.

  • A new typehint indicates a function argument must be callable.

  • Session entropy uses /dev/urandom or /dev/arandom by default for extra security if either is present at compile time.

  • Function call results can now be immediately dereferenced as arrays: foo()[0]

  • Class members can be accessed on instantiation: (new foo)->method()

For more changes see the migration documentation.

Thursday Jan 19, 2012

PHP VirtualBox VM has been refreshed

While I was recently on summer vacation (yes, it is summer in half the world), the VirtualBox VM for PHP was refreshed to a more recent Zend Server/Oracle Linux/Oracle Database XE stack.

I just pulled the VM down, imported it and had it booting and serving PHP pages in a very few minutes. VirtualBox really is marvelous. There is a link to download the VM near the foot of the Oracle Technology Network Developer VM page.

About

Tourists looking out over an Opal mine
I'm a Product Manager in Server Technologies, working on scripting languages and developer-access.
Email: christopher.jones@oracle.com
Twitter: http://twitter.com/ghrd
Links: OTN Node.js Developer Center
OTN PHP Developer Center
Book: Free PHP Oracle book

Follow:
Blaine Carter
Dan McGhan

Search

Archives
« July 2015
SunMonTueWedThuFriSat
   
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
21
22
23
24
25
27
28
29
30
31
 
       
Today