Tuesday Feb 12, 2013

The Mysterious PHP RFC Process and How You Can Change the Web

The PHP RFC process has been in place for a while, and users new to core PHP development are starting to use RFCs to propose desirable features.

Here are some personal observations and suggestions that show how I have seen feature acceptance and the RFC process work in practice. These notes augment the steps in How To Create an RFC. I hope they help set expectations about the PHP RFC process and feature acceptance in the PHP language.

  1. Before starting an RFC, review existing RFCs and search the mail list archives for similar ideas. Use this information to explain what your RFC will bring to the language.

  2. If you're new to PHP core development, mail the "internals" mail list to judge interest before you start your RFC. If you get no reaction (this is likely) or positive reaction then create the RFC. Core PHP developers with karma will know when they can skip this step.

  3. Avoid presenting an RFC idea to the "internals" mail list with email that begins "I don't know much about ... but ...". Do some research first.

  4. There are many really good ideas for improving PHP, however some of them are really tedious or technically risky or hard. If you are about to email the "internals" mail list saying "someone should do ...", then don't hit "Send". Work out how you could do it, and then send a patch.

  5. If the overall goals of PHP are not clear to you, you may not be alone. However, as an RFC writer you need to learn the general trajectory of the language and where web development is heading. You then need to enthuse everyone with your feature and show what it brings to the language.

  6. Don't start an RFC (or mail list discussion) about standardizing PHP function names and function argument orders. There are several historical reasons why the functions are what they are (including compatibility with underlying libraries), and many good reasons why the change would be counter-productive causing more end-user confusion, would lead to unmaintainable PHP engine code, and generally be a waste of everyone's time when they could be doing more interesting projects. This has been discussed ad infinitum. Review previous discussions and feel free to fork PHP on github.

  7. Your RFC should talk about all PHP areas that will be affected: php.ini, different SAPIs, engine, extensions, etc. List similar features. List similar features in other languages. Link to references. Give an estimate of the actual positive impact to user code. Put the proposed voting options in the RFC so they can be included in its discussion. Include the proposed acceptance criteria (2/3 or 50%+1 majority - see the original Voting RFC) to avoid later argument.

  8. Your RFC will be used to create the PHP documentation, so make its sections re-usable. Explain concepts and details. Keep the RFC technical and have plenty of examples.

  9. Many current PHP RFCs don't contain enough detail, nor do they capture the pros & cons that were discussed on the mail list. You can, and should, do better than these RFCs.

  10. An implementation is not mandatory while proposing an RFC but it can help persuade people and help fine-tune the design. Note that if you are unable to implement the RFC, and no one else volunteers during the discussion period, then your RFC is unlikely ever to be implemented.

  11. Don't start an implementation too early. Get support for the feature concept from the "internals" mail list before spending time on coding - unless you want a learning experience.

  12. If you do have an implementation, make it clear whether the implementation is a simple prototype or is expected to be the final code. This is specially important during the vote.

  13. It's easy to get sidetracked by mail list trolling or well intentioned but over-eager discussion. Take advice on board but make sure your feature doesn't end up being designed by a committee. Don't blame the "PHP core community" for derailing discussions when the issues of non- code-contributors on the mail list are at fault.

  14. For every user who wants a backwardly incompatible change to PHP, there is a user who will scream that this will be disastrous to them. As the RFC writer you need to walk the line between these groups. At voting time, other people may see the line in a different place than you do.

  15. There is no need to respond to every discussion email individually. You should batch up your responses and manage the discussion intelligently.

  16. Don't get side tracked by how long it will take a feature to reach end users. It will reach them eventually. You can always supply an unofficial patch or maybe create a PECL extension for users of current versions of PHP.

  17. Don't let mail list discussion drag on too long. High levels of discussion can be a symptom that an RFC is contentious. Having an early vote about only the feature's concept can prevent over-investment in an RFC and implementation.

  18. With long, fragmented discussions, not everyone will read every email. Update the RFC at regular intervals, and let people know what has changed. Include the RFC URL in all your emails so anyone dipping into a mail thread has a better chance of finding the RFC.

  19. PHP is popular so you can often find support from "many" people for cool language features. This doesn't mean those features are a "good thing" for the language or its implementation.

  20. There are multiple PHP communities. Don't interpret support in one community as universal support. Listen carefully to the key PHP developers on the "internals" mail list (they may not be the ones doing the most discussion). They are the people who will undoubtedly end up doing final integration of the feature (e.g. with the op-code cache), fixing edge cases, and maintaining it in the long term. They are the ones that have seen the PHP code base complexity grow and become more difficult to maintain and extend. Even if they like your idea, they may not be able to contribute time and effort to make it stable.

  21. The PHP core development community needs more end-users to become core language developers but the barrier to entry is high because of the complexity of the API, and because new features need very careful integration with a complex language. See the point above about existing developers not having enough time to support every good idea. This just means you need perseverance to become a core developer yourself. You can gain karma by being a regular code contributor before starting your magnum opus RFC.

  22. Some areas of PHP are complex or niche. Sometimes feature suggestions will be greeted by an apparent lack of interest. Don't be discouraged. This just means you need to take a stronger leadership role, and also prove your credentials by first working on the existing code base.

  23. Keep your RFC updated with a summary of all the positive and negative mail list discussion points and examples.

    • This prevents arguments being rehashed on the mail list each year.
    • Documentation will have clear use cases and show what does and doesn't work.
    • Future RFC writers can see trends and build on previous ideas.
  24. Before initiating a vote on your RFC, make sure the RFC contains enough detail so that the vote is solely on the RFC contents, not on any half-remembered or misinterpretable mail list discussions. Un- (or badly) expressed intentions will only cause you frustration when your RFC is rejected.

  25. Don't leave any ambiguity in the RFC. As well as stating what will be changed, also state what won't be changed. Ambiguity will hurt the chances of a successful outcome because voters will be unsure that the feature has been fully thought through. Make sure there are no "Open Issues" in the RFC when voting. Make a decision about the choices before opening the vote, or mark the issues clearly as "Future topics for exploration".

  26. Make sure any vote is clear about what is being voted on: is it the idea, the specification, or the implementation? Is a particular version of PHP being targeted? Is the implementation going to be merged to the core language or made as a PECL extension?

  27. Warn the "internals" mail list before starting the vote. This notification gives you a chance to fine tune the wording of your poll; this wording has caused contention in the past. The approaching deadline may also cause last-minute RFC responses. These may be significant enough to require further discussion.

  28. Set a deadline for the vote (sadly the current voting RFC doesn't require this). Having a deadline will forestall suggestions that a vote was deliberately left open until it succeeded, and then closed before the "negative" side could rally support.

  29. During the voting period, it is common for people to continue mail list discussion. You may need to halt the vote and address any issues.

  30. Prior to the end of the voting period, remind the mail list that the vote is about to close.

  31. Positive voting poll outcomes for RFs that are just "concepts" can be interpreted as support for "good ideas" rather than being a definitive guarantee that a feature will appear in a future version of PHP. As a feature is later investigated and further discussed, the early vote may become irrelevant. Similarly, where there is no chance of an implementation being written, a positive poll outcome is just an indicator of desire.

  32. If your RFC is rejected, it is not the end of the world. Sometimes ideas are submitted "before their time", and it takes an acclimatization period for other developers to see their value. As the core PHP developer base goes through natural turn-over, revisiting an idea after a (long) period may result in different appreciation. (This does not apply to some ideas, for example standardizing function names). See the previous points about becoming a core contributor - having karma goes a long way towards getting an RFC accepted, not only because experienced contributors know which RFCs are worth suggesting, and know how to make proposals understandable. When your RFC is rejected, continue working with the PHP core development community and revisit the RFC later.

  33. You can always fork PHP from github.

In summary, the PHP development process is an organic process and subject to flexibility. It is based heavily around users contributing features that they require. This therefore requires high investment from users who want change. Very rarely do PHP core developers have bandwidth to take on external ideas, no matter how good they are. Feature acceptance has to be pragmatic. The core PHP contributors would love to see more people get commit karma and contribute to PHP.

This post has gone on long enough. My notes are current for the time being. I'm sure there are observations or clarifications you can make. If you want to add anything, please post a comment.

(Note: this page has been augmented and re-ordered since first being published)

Tuesday Jan 29, 2013

MySQL Sessions at the ConFoo.ca conference

Who says direct marketing doesn't work? A personal request to blog the upcoming ConFoo conference (25 February - 1 March 2013 in Montreal, Canada) has, as you can see, been successful. Although it's been a few years since I spoke there, I recall how impressive the organization was. The diversity and continual growth trajectory of the conference over the years is a very good indicator that you should be involved.

While you're there, say Hi to Oracle Ace Director Sheeri Cabral who is giving a couple of MySQL talks.

Wednesday Jan 23, 2013

Quick Debugging of PHP Scripts in Emacs with Geben and Xdebug

When you want to test a PHP code snippet quickly, it's handy to do it within your day-to-day environment. For me, this environment is Emacs. The geben package for Emacs provides an interface to the DBGp protocol, such as used by Derick Rethans's standard Xdebug extension for PHP.

With the combination of geben and Xdebug, I can quickly and efficiently step through execution of local files, examining code flow and data values.

Working steps to install and use the debugger on command line PHP scripts are shown below. They are standard enough that they can be customized to your actual environment.

You probably already have Xdebug installed, since it is the common debugger and profiler used by most IDEs. Installing geben in Emacs is straight forward.

1. Download PHP 5.4 from http://php.net/downloads.php

2. Install PHP with:

$ tar -jxf php-5.4.11.tar.bz2
$ cd php-5.4.11
$ ./configure --prefix=/home/cjones/php54
$ make install
$ cp php.ini-development /home/cjones/php54/lib/php.ini

3. Download the source code for the Xdebug Debugger extension for PHP from http://xdebug.org/download.php

4. Install Xdebug into PHP with:

$ cd /home/cjones
$ tar -zxf xdebug-2.2.1.tgz
$ cd xdebug-2.2.1
$ export PATH=/home/cjones/php54/bin:$PATH
$ phpize
$ ./configure --enable-xdebug --with-php-config=/home/cjones/php54/bin/php-config
$ make install

More information is shown in http://xdebug.org/docs/install

5. Edit /home/cjones/php54/lib/php.ini and enable Xdebug by adding these lines:

zend_extension=/home/cjones/php54/lib/php/extensions/no-debug-non-zts-20100525/xdebug.so
xdebug.remote_enable=on
xdebug.remote_host=127.0.0.1

6. Install geben manually or use the Emacs 24 package repository (M-x package-list-packages)

To manually install geben, download it from http://code.google.com/p/geben-on-emacs/downloads/list and install it with:

$ cd /home/cjones
$ tar -zxf geben-0.26.tar.gz

8. Add the following to the Emacs initialization file /home/cjones/.emacs. When testing this blog post, my .emacs file only contained this code:

(setq load-path (cons "/home/cjones/geben-0.26" load-path))

(autoload 'geben "geben" "DBGp protocol frontend, a script debugger" t)

;; Debug a simple PHP script.
;; Change the session key my-php-54 to any session key text you like
(defun my-php-debug ()
  "Run current PHP script for debugging with geben"
  (interactive)
  (call-interactively 'geben)
  (shell-command
    (concat "XDEBUG_CONFIG='idekey=my-php-54' /home/cjones/php54/bin/php "
    (buffer-file-name) " &"))
  )

(global-set-key [f5] 'my-php-debug)

Experienced Emacs users will most likely use local-set-key in a php-mode hook to set a key mapping.

9. Start Emacs and load a PHP file:

$ emacs my.php

10. Press F5 to start the debugger. The script will open in geben mode and can be stepped through with the space bar.

Geben mode commands can be shown with '?'. They include 'v' for showing variables, 'c' for run-to-cursor, and 'g' for completing the program.

Geben can also be used for scripts called via a browser. Start the geben listener in Emacs with M-x geben and load a script in the browser, passing an Xdebug URL parameter setting the session name. This initiates the debug session in Emacs. E.g. Load http://localhost/my.php?XDEBUG_SESSION_START=my-php-54

11. When you are finished debugging, stop the debug server listener inside Emacs:

M-x geben-end

Tuesday Dec 11, 2012

Excitement! Updated Underground PHP and Oracle Manual is Available for Download

We're thrilled to have a major update of the free Underground PHP and Oracle Manual released on OTN.

The Underground PHP and Oracle Manual is designed to bridge the gap between the many PHP scripting language and the many Oracle Database books available. It contains unique material about PHP's OCI8 extension for Oracle Database, and about other components in the PHP-Oracle ecosystem. It shows PHP developers how to use PHP and Oracle together, efficiently and easily.

The book has been completely refreshed. It has been updated for Oracle XE 11g and the latest PHP OCI8 extension. There are new chapters about using PHP with Oracle TimesTen, NetBeans and Oracle Tuxedo. There is also a new chapter about installing PHP on Oracle Solaris. The book now clocks in at 347 pages of great content.

Acknowledgements are due to all those who have helped with this and previous editions of the book. Thanks to the product teams that assisted with brand new content. In particular Craig Mohrman contributed the chapter about PHP on Solaris. Jeffry Rubinoff contributed the base text for the chapter on PHP and NetBeans.

Thursday Dec 06, 2012

Adding DTrace Probes to PHP Extensions

The powerful DTrace tracing facility has some PHP-specific probes that can be enabled with --enable-dtrace.

DTrace for Linux is being created by Oracle and is currently in tech preview. Currently it doesn't support userspace tracing so, in the meantime, Systemtap can be used to monitor the probes implemented in PHP. This was recently outlined in David Soria Parra's post Probing PHP with Systemtap on Linux.

My post shows how DTrace probes can be added to PHP extensions and traced on Linux. I was using Oracle Linux 6.3.

Not all Linux kernels are built with Systemtap, since this can impact stability. Check whether your running kernel (or others installed) have Systemtap enabled, and reboot with such a kernel:

# grep CONFIG_UTRACE /boot/config-`uname -r`
# grep CONFIG_UTRACE /boot/config-*

When you install Systemtap itself, the package systemtap-sdt-devel is needed since it provides the sdt.h header file:

# yum install systemtap-sdt-devel

You can now install and build PHP as shown in David's article. Basically the build is with:

$ cd ~/php-src
$ ./configure --disable-all --enable-dtrace
$ make

(For me, running 'make' a second time failed with an error. The workaround is to do 'git checkout Zend/zend_dtrace.d' and then rerun 'make'. See PHP Bug 63704)

David's article shows how to trace the probes already implemented in PHP. You can also use Systemtap to trace things like userspace PHP function calls. For example, create test.php:

<?php

$c = oci_connect('hr', 'welcome', 'localhost/orcl');
$s = oci_parse($c, "select dbms_xmlgen.getxml('select * from dual') xml from dual");
$r = oci_execute($s);
$row = oci_fetch_array($s, OCI_NUM);
$x = $row[0]->load();
$row[0]->free();
echo $x;

?>

The normal output of this file is the XML form of Oracle's DUAL table:

$ ./sapi/cli/php ~/test.php
<?xml version="1.0"?>
<ROWSET>
 <ROW>
  <DUMMY>X</DUMMY>
 </ROW>
</ROWSET>

To trace the PHP function calls, create the tracing file functrace.stp:

probe process("sapi/cli/php").function("zif_*") {
    printf("Started function %s\n", probefunc());
}

probe process("sapi/cli/php").function("zif_*").return {
    printf("Ended function %s\n", probefunc());
}

This makes use of the way PHP userspace functions (not builtins) like oci_connect() map to C functions with a "zif_" prefix.

Login as root, and run System tap on the PHP script:

# cd ~cjones/php-src
# stap -c 'sapi/cli/php ~cjones/test.php' ~cjones/functrace.stp
Started function zif_oci_connect
Ended function zif_oci_connect
Started function zif_oci_parse
Ended function zif_oci_parse
Started function zif_oci_execute
Ended function zif_oci_execute
Started function zif_oci_fetch_array
Ended function zif_oci_fetch_array
Started function zif_oci_lob_load
<?xml version="1.0"?>
<ROWSET>
 <ROW>
  <DUMMY>X</DUMMY>
 </ROW>
</ROWSET>
Ended function zif_oci_lob_load
Started function zif_oci_free_descriptor
Ended function zif_oci_free_descriptor

Each call and return is logged. The Systemtap scripting language allows complex scripts to be built. There are many examples on the web.

To augment this generic capability and the PHP probes in PHP, other extensions can have probes too. Below are the steps I used to add probes to OCI8:

  1. I created a provider file ext/oci8/oci8_dtrace.d, enabling three probes. The first one will accept a parameter that runtime tracing can later display:

    provider php {
    	probe oci8__connect(char *username);
    	probe oci8__nls_start();
    	probe oci8__nls_done();
    };
    
  2. I updated ext/oci8/config.m4 with the PHP_INIT_DTRACE macro. The patch is at the end of config.m4. The macro takes the provider prototype file, a name of the header file that 'dtrace' will generate, and a list of sources files with probes. When --enable-dtrace is used during PHP configuration, then the outer $PHP_DTRACE check is true and my new probes will be enabled. I've chosen to define an OCI8 specific macro, HAVE_OCI8_DTRACE, which can be used in the OCI8 source code:

    diff --git a/ext/oci8/config.m4 b/ext/oci8/config.m4
    index 34ae76c..f3e583d 100644
    --- a/ext/oci8/config.m4
    +++ b/ext/oci8/config.m4
    @@ -341,4 +341,17 @@ if test "$PHP_OCI8" != "no"; then
         PHP_SUBST_OLD(OCI8_ORACLE_VERSION)
     
       fi
    +
    +  if test "$PHP_DTRACE" = "yes"; then
    +     AC_CHECK_HEADERS([sys/sdt.h], [
    +       PHP_INIT_DTRACE([ext/oci8/oci8_dtrace.d],
    +                       [ext/oci8/oci8_dtrace_gen.h],[ext/oci8/oci8.c])
    +         AC_DEFINE(HAVE_OCI8_DTRACE,1,
    +         [Whether to enable DTrace support for OCI8 ])
    +     ], [
    +       AC_MSG_ERROR(
    +         [Cannot find sys/sdt.h which is required for DTrace support])
    +     ])
    +   fi
    +
     fi
    
  3. In ext/oci8/oci8.c, I added the probes at, for this example, semi-arbitrary places:

    diff --git a/ext/oci8/oci8.c b/ext/oci8/oci8.c
    index e2241cf..ffa0168 100644
    --- a/ext/oci8/oci8.c
    +++ b/ext/oci8/oci8.c
    @@ -1811,6 +1811,12 @@ php_oci_connection *php_oci_do_connect_ex(char *username, int username_len, char
     		}
     	}
     
    +#ifdef HAVE_OCI8_DTRACE
    +    if (DTRACE_OCI8_CONNECT_ENABLED()) {
    +		DTRACE_OCI8_CONNECT(username);
    +	}
    +#endif
    +
     	/* Initialize global handles if they weren't initialized before */
     	if (OCI_G(env) == NULL) {
     		php_oci_init_global_handles(TSRMLS_C);
    @@ -1870,11 +1876,22 @@ php_oci_connection *php_oci_do_connect_ex(char *username, int username_len, char
     		size_t rsize = 0;
     		sword result;
     
    +#ifdef HAVE_OCI8_DTRACE
    +		if (DTRACE_OCI8_NLS_START_ENABLED()) {
    +			DTRACE_OCI8_NLS_START();
    +		}
    +#endif
     		PHP_OCI_CALL_RETURN(result, OCINlsEnvironmentVariableGet, (&charsetid_nls_lang, 0, OCI_NLS_CHARSET_ID, 0, &rsize));
     		if (result != OCI_SUCCESS) {
     			charsetid_nls_lang = 0;
     		}
     		smart_str_append_unsigned_ex(&hashed_details, charsetid_nls_lang, 0);
    +
    +#ifdef HAVE_OCI8_DTRACE
    +		if (DTRACE_OCI8_NLS_DONE_ENABLED()) {
    +			DTRACE_OCI8_NLS_DONE();
    +		}
    +#endif
     	}
     
     	timestamp = time(NULL);
    

    The oci_connect(), oci_pconnect() and oci_new_connect() calls all use php_oci_do_connect_ex() internally. The first probe simply records that the PHP application made a connection call. I already showed a way to do this without needing a probe, but adding a specific probe lets me record the username. The other two probes can be used to time how long the globalization initialization takes.

    The relationships between the oci8_dtrace.d names like oci8__connect, the probe guards like DTRACE_OCI8_CONNECT_ENABLED() and probe names like DTRACE_OCI8_CONNECT() are obvious after seeing the pattern of all three probes.

    I included the new header that will be automatically created by the dtrace tool when PHP is built. I did this in ext/oci8/php_oci8_int.h:

    diff --git a/ext/oci8/php_oci8_int.h b/ext/oci8/php_oci8_int.h
    index b0d6516..c81fc5a 100644
    --- a/ext/oci8/php_oci8_int.h
    +++ b/ext/oci8/php_oci8_int.h
    @@ -44,6 +44,10 @@
     #  endif
     # endif /* osf alpha */
     
    +#ifdef HAVE_OCI8_DTRACE
    +#include "oci8_dtrace_gen.h"
    +#endif
    +
     #if defined(min)
     #undef min
     #endif
    
  4. Now PHP can be rebuilt:

    $ cd ~/php-src
    $ rm configure && ./buildconf --force
    $ ./configure --disable-all --enable-dtrace \
                  --with-oci8=instantclient,/home/cjones/instantclient
    $ make
    

    If 'make' fails, do the 'git checkout Zend/zend_dtrace.d' trick I mentioned.

  5. The new probes can be seen by logging in as root and running:

    # stap -l 'process.provider("php").mark("oci8*")' -c 'sapi/cli/php -i'
    process("sapi/cli/php").provider("php").mark("oci8__connect")
    process("sapi/cli/php").provider("php").mark("oci8__nls_done")
    process("sapi/cli/php").provider("php").mark("oci8__nls_start")
    
  6. To test them out, create a new trace file, oci.stp:

    global numconnects;
    global start;
    global numcharlookups = 0;
    global tottime = 0;
    probe process.provider("php").mark("oci8-connect") {
        printf("Connected as %s\n", user_string($arg1));
        numconnects += 1;
    }
    probe process.provider("php").mark("oci8-nls_start") {
        start = gettimeofday_us();
        numcharlookups++;
    }
    probe process.provider("php").mark("oci8-nls_done") {
        tottime += gettimeofday_us() - start;
    }
    probe end {
        printf("Connects: %d, Charset lookups: %ld\n", numconnects, numcharlookups);
        printf("Total NLS charset initalization time: %ld usecs/connect\n", 
                            (numcharlookups > 0 ? tottime/numcharlookups : 0));
    }
    

    This calculates the average time that the NLS character set lookup takes. It also prints out the username of each connection, as an example of using parameters.

  7. Login as root and run Systemtap over the PHP script:

    # cd ~cjones/php-src
    # stap -c 'sapi/cli/php ~cjones/test.php' ~cjones/oci.stp
    Connected as cj
    <?xml version="1.0"?>
    <ROWSET>
     <ROW>
      <DUMMY>X</DUMMY>
     </ROW>
    </ROWSET>
    Connects: 1, Charset lookups: 1
    Total NLS charset initalization time: 164 usecs/connect
    

    This shows the time penalty of making OCI8 look up the default character set. This time would be zero if a character set had been passed as the fourth argument to oci_connect() in test.php.

Update: To use real (non-SystemTap) DTrace, the extension binary needs to be built slightly differently than shown above in step 2. Instead of modifying config.m4 to reuse PHP_INIT_DTRACE, an OCI8-specific variant of that macro is created. To see how this is done, look at the new OCI8_INIT_DTRACE macro definition and way it is used in PHP OCI8 2.0's ext/oci8/config.m4.

Sunday Nov 25, 2012

How (and when) to move users to mysqli and PDO_MYSQL?

An important discussion on the PHP "internals" development mailing list is taking place. It's one that you should take some note of. It concerns the next step in transitioning PHP applications away from the very old mysql extension and towards adopting the much better mysqli extension or PDO_MYSQL driver for PDO. This would allow the mysql extension to, at some as-yet undetermined time in the future, be removed. Both mysqli and PDO_MYSQL have been around for many years, and have various advantages: http://php.net/manual/en/mysqlinfo.api.choosing.php

The initial RFC for this next step is at https://wiki.php.net/rfc/mysql_deprecation I would expect the RFC to change substantially based on current discussion. The crux of that discussion is the timing of the next step of deprecation. There is also discussion of the carrot approach (showing users the benfits of moving), and stick approach (displaying warnings when the mysql extension is used). As always, there is a lot of guesswork going on as to what MySQL APIs are in current use by PHP applications, how those applications are deployed, and what their upgrade cycle is. This is where you can add your weight to the discussion - and also help by spreading the word to move to mysqli or PDO_MYSQL. An example of such a 'carrot' is the excellent summary at Ulf Wendel's blog: http://blog.ulf-wendel.de/2012/php-mysql-why-to-upgrade-extmysql/

I want to repeat that no time frame for the eventual removal of the mysql extension is set. I expect it to be some years away.

Tuesday Mar 13, 2012

PHP-FPM FastCGI Process Manager with Apache 2

I've published some vanilla PHP 5.4.0 RPMs to make new feature testing easier.

Along with all the PHP 5.4 goodies, the php-fpm "FastCGI Process Manager" is available for the first time on oss.oracle.com. Php-fpm is an alternative FastCGI interface to PHP with various extra features such as load dependent spawning of processes. (For other features, see php-fpm.org). Php-fpm has been getting more and more traction in the PHP community and the EXPERIMENTAL flag was removed in PHP 5.4. You might want to test it out.

To use php-fpm with the default Apache web server, first install Oracle Linux 5.8 (64bit) using Oracle's free, public yum repository public-yum.oracle.com.

Install Apache, if not already installed:

  yum install httpd

Download and install the PHP 5.4 RPMs from oss.oracle.com/projects/php:

  rpm -i php54-common-5.4.0-1.el5.x86_64.rpm php54-fpm-5.4.0-1.el5.x86_64.rpm

Other extensions can also be installed, depending on the functionality you want to test.

Download and build FastCGI for Apache:

  wget http://www.fastcgi.com/dist/mod_fastcgi-current.tar.gz
  tar -zxf mod_fastcgi-current.tar.gz
  cd mod_fastcgi-2.4.6
  cp Makefile.AP2 Makefile
  make top_dir=/usr/lib64/httpd

Install FastCGI as root:

  make top_dir=/usr/lib64/httpd install

Edit /etc/httpd/conf/httpd.conf and comment out any existing references to PHP you might previously have added for testing:

# LoadModule php5_module        modules/libphp5.so
# AddType application/x-httpd-php .php

Add the php-fpm configuration to httpd.conf:

  LoadModule fastcgi_module modules/mod_fastcgi.so

  <IfModule mod_fastcgi.c>  
    FastCGIExternalServer /usr/sbin/php-fpm -host 127.0.0.1:9000
    AddHandler php-fastcgi .php  

    #<LocationMatch "/status">
    #  SetHandler php-fastcgi-virt
    #  Action php-fastcgi-virt /usr/sbin/php-fpm.fcgi virtual
    #</LocationMatch>

    Action php-fastcgi /usr/sbin/php-fpm.fcgi  
    ScriptAlias /usr/sbin/php-fpm.fcgi /usr/sbin/php-fpm  

    <Directory /usr/sbin>  
      Options ExecCGI FollowSymLinks  
      SetHandler fastcgi-script  
      Order allow,deny  
      Allow from all  
    </Directory>  
  </IfModule> 

Start php-fpm and Apache:

  service php-fpm start
  service httpd start

Test it out with your favorite script or create a file pi.php:

  <?php
    phpinfo();
  ?>

Save it in /var/www/html/pi.php or in $HOME/public_html/pi.php, if you have configured UserDir in httpd.conf.

In a browser load http://localhost/pi.php or http://localhost/~yourname/pi.php.

This will show the PHP configuration values.

To test php-fpm's built-in statistics, edit httpd.conf and uncomment the four lines of the LocationMatch section:

  <LocationMatch "/status">
    SetHandler php-fastcgi-virt
    Action php-fastcgi-virt /usr/sbin/php-fpm.fcgi virtual
  </LocationMatch>

Edit /etc/php-fpm.conf and uncomment the line:

  pm.status_path = /status

Restart php-fpm and Apache:

  service php-fpm restart
  service httpd restart

Run some load on the system:

  ab -c 10 -t 60 http://localhost/pi.php  

Now http://localhost/status gives you the status of the server:

  pool:                 www
  process manager:      dynamic
  start time:           13/Mar/2012:14:25:53 -0700
  start since:          26
  accepted conn:        50001
  listen queue:         0
  max listen queue:     6
  listen queue len:     128
  idle processes:       2
  active processes:     1
  total processes:      3
  max active processes: 5
  max children reached: 1

The php-fpm.conf file documents other output formats for the statistics. It also shows the extensive functionality available with php-fpm.

Documentation on php-fpm is not ideal, but you can see some at http://php.net/manual/en/install.fpm.php. The php-fpm.org site has more, including a forum and wiki.

Php-fpm is commonly used in conjunction with the nginx webserver. For this, you will need to compile nginx yourself. Because php-fpm is installed and managed separately from the webserver, the php-fpm RPM will still be usable. Now that Apache 2.4 has been released with claims of improved performance, it will be interesting to see if web server popularity swings back towards Apache.

Finally, remember that the PHP RPMs on oss.oracle.com are for testing purposes only. They are not supported.

Update: In the Underground PHP & Oracle Manual, I simplified the httpd.conf options to this (note PHP was installed in a different location):

<IfModule mod_fastcgi.c>
    FastCGIExternalServer /opt/php544/sbin/php-fpm -host 127.0.0.1:9000
    AddHandler php-fastcgi .php
    Action php-fastcgi /mycgi
    ScriptAlias /mycgi /opt/php544/sbin/php-fpm
</IfModule>

PHP 5.4.0 RPMs for 64bit Oracle Linux 5.x are available

I've published some vanilla PHP 5.4.0 RPMs to make new feature testing easier for Oracle Linux 5.x 64 bit users. The standard set of RPMs is at oss.oracle.com/projects/php. The OCI8 extension is also available (this requires the free Oracle Instant Client 11.2 from ULN or OTN.)

Some of the features of PHP 5.4 are:

  • Improved memory usage and performance. Some impressive preliminary reports of the benefits include: http://news.php.net/php.internals/57760 and http://news.php.net/php.internals/57747.

  • File Upload progress support is natively implemented.

  • Support for Traits now allows code reuse:

        trait t1 {
    	function m1() { echo "hi"; }
    	function m2() { echo "bye"; }
        }
    
        class my_c1 {
    	use t1;
    	/*...*/
        }
    
        class my_c2 extends c2 {
    	use t1;
    	/*...*/
        }
    
  • A built-in HTTP server for development is included:

      php -S 127.0.0.1:8888
    
  • Improvements were made to the interactive PHP shell (when PHP is compiled with readline).

  • A shortened array syntax was introduced: $a = [1,2,3];

  • The default character set for several internal functions was changed to UTF-8.

  • Support for multibyte languages is now configurable at run-time instead of compile-time.

  • The value echo tag "<?=" is now always on.

  • Binary number support was added.

  • DTrace support was added.

  • A new typehint indicates a function argument must be callable.

  • Session entropy uses /dev/urandom or /dev/arandom by default for extra security if either is present at compile time.

  • Function call results can now be immediately dereferenced as arrays: foo()[0]

  • Class members can be accessed on instantiation: (new foo)->method()

For more changes see the migration documentation.

Thursday Jan 19, 2012

PHP VirtualBox VM has been refreshed

While I was recently on summer vacation (yes, it is summer in half the world), the VirtualBox VM for PHP was refreshed to a more recent Zend Server/Oracle Linux/Oracle Database XE stack.

I just pulled the VM down, imported it and had it booting and serving PHP pages in a very few minutes. VirtualBox really is marvelous. There is a link to download the VM near the foot of the Oracle Technology Network Developer VM page.

Thursday Jan 12, 2012

PHP 5.3.9 RPMs Available for Testing

I've updated my relatively "vanilla" PHP 5 RPMs to 5.3.9 on oss.oracle.com/projects/php. They are built for Oracle Linux 5.7 (and RHEL 5.7). I've included the OCI8 extension for Oracle DB, of course. The various MySQL extension are there and use the mysqlnd driver, so installation doesn't require any client-side MySQL libraries. Note the PHP 5.3.9 RPMs are for testing only. For production users I recommend Zend Server.

Wednesday Jan 11, 2012

PHP OCI8 1.4.7 is available on PECL

Following on the heels of the PHP 5.3.9 release, I've bundled OCI8 1.4.7 for PECL. The PECL OCI8 1.4.7 code is the same as included in PHP 5.3.9 and can be used to update older PHP installations. The release notes are here. There are several bug fixes; upgrading is recommended.

Monday Oct 24, 2011

PHP oci_password_change() ORA-1017 gotcha with Oracle Database 11.2.0.3

The Oracle Instant Client 11.2.0.3 for Linux is now available on OTN. I know many users will upgrade sooner rather than later to get the "latest and greatest" release.

However, if your PHP application uses oci_password_change (or its old alias ocipasswordchange), you should upgrade the client libraries (Instant or normal "full" client) and the Oracle Database version to 11.2.0.3 at the same time. Otherwise oci_password_change will fail with the error "ORA-1017: invalid username/password". If you're not using oci_password_change then this won't impact you.

Since the protocol change is in the Oracle libraries, the change is not just limited PHP. Any C program using Oracle's Call Interface "OCIPasswordChange" API to change passwords will also have the same restrictions.

Monday Sep 26, 2011

Scripting Language Related Sessions at Oracle OpenWorld and JavaOne, October 2011

Oracle OpenWorld and JavaOne conferences are happening in San Francisco next week. It will be a busy and exciting time.

First, here's a shout out: For me the conference kicks off on Sunday morning. Marcelle Kratochvil from Piction (heavy users of PHP and Oracle DB) is hosting the inaugural Unstructured Data with Multimedia SIG for Oracle Database and MySQL database (32440) Sunday 9:00 am in Moscone West room 2011.

Below are some of the scripting and related sessions happening during the week.

Exhibition Hall

During the Exhibition Hall hours, come and talk to us at the Database Access Services and APIs booth. This year we're in Moscone South, Left SL-067. The Tuxedo application server booth is Moscone South, Right - SR-202. At JavaOne look out for the NetBeans booth, Hilton San Francisco - HHJ-023.

Scripting Sessions, Birds-of-a-Feather Meetings, and Hands-on-Labs at OOW

  • The Oracle Tuxedo team has scripting language support in their powerful application server environment:
    High-Performance Web Applications with C/C++/PHP/Python (15705)
    Monday, 05:00 PM, Moscone South - 300
  • This year we are running introductory Hands-on Lab sessions for three languages concurrently. Come along and choose which language you'd like to dip your toes into:
    Develop and Deploy High-Performance Web 2.0 PHP, Ruby, or Python Applications (30082)
    Monday, 05:00 PM, Marriott Marquis - Salon 10/11
  • Come and ask questions at the round table Birds-of-a-Feather session:
    Meet the Oracle Database Clients Developers: C, C++, PHP, Python, Ruby, and Perl (26240)
    Monday, 07:30 PM, Marriott Marquis - Salon 8
  • My overview and state-of-the-nation session is:
    PHP, Ruby, Python, and Perl: Develop and Deploy Mission-Critical Apps with Oracle Database 11g (14704)
    Wednesday, 11:45 AM, Marriott Marquis - Salon 8
  • The Tuxedo team Hands-on-Lab lets you code in C/C++/PHP/Python/Ruby:
    Develop High-Performance, Service-Oriented C/C++ Applications for Oracle Exalogic (31120)
    Thursday, 12:00 PM, Marriott Marquis - Salon 3/4
  • Raimonds Simanovskis, maintainer of the Rails adapter for Oracle is giving a session:
    Extending Oracle E-Business Suite with Ruby on Rails (8604)
    Thursday, 03:00 PM, Moscone West - 2002/2004

Several other sessions discuss topics that scripting language devotees will find invaluable:

  • Build, Deploy, and Troubleshoot Highly Performant, Highly Available Apps with Oracle Database (14703)
    Wednesday, 05:00 PM, Moscone South - 303
  • Net Services: Best Practices for Performance, Scalability, and High Availability (14345)
    Wednesday, 01:15 PM, Moscone South - 303

Also check out the full Oracle Tuxedo application server schedule here.

Scripting at JavaOne

Over in the concurrent JavaOne conference there are several scripting sessions driven by San Francisco's EngineYard. This year they have JRuby sessions but with their recent aquisition of PHP technnology, I hope they'll have more on PHP in one of the OOW streams next year:

  • Accelerate Your Business and Aim for the Cloud with Java and JRuby (25284)
    Wednesday, 03:00 PM, Parc 55 - Embarcadero
  • From Java to Rails: Techniques for Adding Ruby Agility to Your Java Web Stack (24582)
    Monday, 05:30 PM, Parc 55 - Market Street
  • Real-World JRuby (23600)
    Wednesday, 04:30 PM, Parc 55 - Market Street
  • Script Bowl 2011: A Scripting Languages Shootout (22060)
    Wednesday, 08:30 AM, Hilton San Francisco - Grand Ballroom B

Also keep an eye out for the various NetBeans IDE sessions and demo booth.

Linux

Check out the four pages of Focus on Linux sessions and events.

MySQL

There is a veritable plethora of MySQL content - four pages of sessions and activites are listed in the Oracle Focus on MySQL. Don't forget the MySQL Community Reception Tuesday 7:00pm - 9:00pm in the Marriott Marquis - Foothill G.

Having started this post with a shout out, let me end with one to Bill Karwin, who was instrumental in getting PHP's Zend Framework off the ground. He is talking about MaatKit and SQL Injection.

Update: Ligaya Turmelle, well known in the PHP community, confirmed overnight that despite a recent job move she is still on track to present her MySQL Performance Tuning talk (16040).

You can search the OOW session catalog here and the JavaOne session catalog here.

Saturday Sep 03, 2011

Oracle 11.2 XE with newly updated PHP Developer's Guide is available

Oracle Database 11g Release 2 Express Edition aka "Oracle XE" is now available on Linux 64 bit and Windows 32 bit. This is a free version of the Oracle Database. Windows 64 bit is "in the works" but Linux 32 bit is not planned.

Check out the newly updated Oracle Database Express Edition 2 Day + PHP Developer's Guide. As well as HTML and PDF variants, the manual is available in Mobi and EPUB formats.

The 2 Day + PHP manual has steps for installing PHP and walks through creating an introductory application. It shows different ways to interact with Oracle XE and introduces PHP 5.3 features.

The example shows the mechanics of DB interaction. It builds the application from the ground up so you can understand how to construct your own high performance applications. If you want to continue the learning path and use a PHP framework, Oracle 11g XE works with the Developing a PHP Web Application with Oracle Database 11g training.

The Oracle 11.2 XE Linux install is RPM based. It takes just a few minutes to install, prompting only for a few necessary details such as desired passwords and ports. You need root access to install it.

Oracle 11.2 XE supports DRCP connection pooling so you'll really be able to maximize the small footprint database for PHP applications.

Readers who want to dive deeper into detail about PHP and advanced features available in other editions of Oracle Database might be interested in the later sections of the Underground PHP and Oracle Manual.

Wednesday Jun 15, 2011

Oracle Resources for PHP

Here are some key resources for working with PHP and Oracle technologies.

  • Overall Links

    Oracle : Oracle Technology Network (OTN)

    Homepage : The PHP Developer Center -- downloads, how-tos, sample code and discussion forums brought to you by OTN.

    Read : PHP Articles -- a collection of informative articles on OTN.

    Technology : Download the newest versions of other software in Oracle's Technology Stack. Read about other Oracle application development tools.

  • PHP Oracle Database Extension : OCI8 is the most available and scalable PHP adapter for the Oracle database. It is included with PHP, and is also separately downloadable for upgrading older PHP releases. OCI8 works with PHP 4 and PHP 5, and will compile with Oracle 9iR2, 10g and 11g client libraries. Oracle's standard cross-version compatibility and connectivity is applicable, so OCI8 can connect to older or newer databases, locally or remotely. The latest OCI8 release includes support for Oracle Database 11g Database Resident Connection Pooling (DRCP), and for Fast Application Notification (FAN) [whitepaper here], and also has support for Oracle's authentication and end-to-end tracing meta data attributes [article here]. These features improve scalability and availabilty of the OCI8 extension.

    The OCI8 extension can also be used with the Oracle In-Memory Database Cache.

    Book : The Underground PHP & Oracle Manual -- the definitive, free guide to using PHP with Oracle Database.

    Tutorial : Oracle & PHP tutorials -- step by step Oracle Learning Library tutorials for using PHP with Oracle. (Free OTN login required)

    Read : PHP & High Scalability -- covers enabling DRCP and FAN for PHP applications.

    Read : PHP Web Auditing, Authorization and Monitoring with Oracle Database -- learn how to audit individual web users, automatically apply rules to individual web users to restrict data access, and monitor and trace database usage per application user.

    Forum : The Oracle & PHP forum -- technical discussion forum for using PHP with Oracle.

    Blog : PHP and Oracle: Christopher Jones -- The latest news on PHP and Oracle.

  • Database : Oracle Express Edition Database -- an entry-level, small-footprint database based on the standard Oracle Database code base that's free to develop, deploy, and distribute; fast to download; and simple to administer.

    PHP OCI8 works with Oracle Database XE the same way it works with the other editions of Oracle Database.

    Oracle Database XE is a great starter database for:

    • Developers working on PHP, Python, Ruby, Java, .Net and other open source applications.
    • DBAs who need a free, starter database for training and deployment
    • Independent Software Vendors (ISVs) and hardware vendors who want a starter database to distribute free of charge
    • Educational institutions and students who need a free database for their curriculum

    Install Now!

  • Database : MySQL -- The world's most popular open source database. MySQL Community Edition is the freely downloadable version. Commercial customers have the flexibility of choosing from multiple editions to meet specific business and technical requirements. Also available for free is the MySQL Workbench for SQL Authoring, System Administration, and schema modeling.

    Recent additions to MySQL include semi synchronous replication, direct memcached-to-InnoDB access, multi-threaded replication, and enhanced partitioning. See What's new in MySQL 5.6

    PHP's mysqli or PDO_mysql extensions can be used to access MySQL.

    Check out the new Query Cache plugin documented here and the Replication and Load-Balancing plugin for master-slave-splitting, documented [here].

    Homepage : Using MySQL With PHP -- Articles and downloads

    Forum : PHP and MySQL forum -- The place to ask questions

    Blog : Planet MySQL -- The latest MySQL news from the community

    Blog : Johannes Schl├╝ter -- A member of the MySQL connector team and the current PHP Release Master.

    Blog : Me Talking Out Loud -- Ligaya Turmelle, a member of MySQL Support. (and contributer to The PHP Anthology: 101 Essential Tips, Tricks & Hacks)

    Blog : Internet Super Hero -- Ulf Wendel, a member of the MySQL connector team.

    Books : Many! Including PHP and MySQL Web Development and Web Database Applications with PHP & MySQL and Learning PHP & MySQL: Step-by-Step Guide to Creating Database-Driven Web Sites

    Install Now!

  • Database : Oracle Berkeley DB 11g -- provides the complex data management features found in enterprise class databases. These facilities include high throughput, low-latency reads, non-blocking writes, high concurrency, data scalability, in-memory caching, ACID transactions, automatic and catastrophic recovery when the application, system or hardware fails, high availability and replication in an application configurable package. Simply configure the library and use the particular features available to satisfy your particular application needs. Berkeley DB can be configured to address any application need from the hand-held device to the datacenter, from a local storage solution to a world-wide distributed one, from kilobytes to petabytes.

    Berkeley DB's SQL API allows PHP's sqlite3 and PDO_sqlite extensions to be used.

    Berkeley DB also includes source code for a php_db4 extension.

    Install Now!

  • IDE : NetBeans -- lets you develop desktop, mobile and web applications using Java, PHP, C/C++ and more. Runs on Windows, Linux, Mac OS X and Solaris. NetBeans IDE is open-source and free.

    Understand: NetBeans PHP features -- All the great features that NetBeans has for PHP developers

    Learn: NetBeans PHP Learning Trail -- the best way to learn how to use NetBeans

    Install Now!

  • Virtualize : Oracle VM VirtualBox -- powerful Cross-platform Virtualization Software for x86-based systems. "Cross-platform" means that it installs on Windows, Linux, Mac OS X and Solaris x86 computers. And "Virtualization Software" means that you can create and run multiple Virtual Machines, running different operating systems, on the same computer at the same time. For example, you can run Windows and Linux on your Mac, run Linux and Solaris on your Windows PC, or run Windows on your Linux systems.

    Oracle VM VirtualBox is available as Open Source or pre-built Binaries for Windows, Linux, Mac OS X and Solaris.

    OTN hosts some pre-built Developer VMs , including an Oracle Tuxedo Web Application Server VM showing PHP support. There is also a Zend Server VM for PHP created by Zend. Zend Server is a complete, Oracle-enabled, enterprise-ready Web Application Server for running and managing PHP applications that require a high level of reliability, performance and security. Zend Server is available in community and supported editions.

    Install Now!

  • Operating system : Oracle Linux -- free to download and distribute. As one of the most widely deployed operating systems today, Linux is increasingly being adopted for cloud-based solutions. Oracle Linux is the most complete and integrated solution available and delivers higher performance and better reliability at up to 7 times lower cost than Red Hat. Why Choose Oracle Linux over Red Hat Linux?

    Oracle Linux 5.6 and 6 introduced PHP 5.3 packages. Subscribers to the Unbreakable Linux Network have access to an additional pre-built PHP OCI8 RPM, and also direct access to an RPM for installing Zend Server .

    Users of older Linux 5 releases can get PHP 5.3 RPMs from oss.oracle.com .

    Install Now!

  • Application Server : Oracle Tuxedo -- provides a robust, grid enabled platform for developing enterprise applications. It provides mainframe-class scale and performance on open, distributed systems for software written in C, C++, COBOL, PHP, Python and Ruby. Oracle Tuxedo provides cost-effective reliability, extreme scalability and throughput of hundreds of thousands of transactions per second. Functionality like Web services, SCA programming model, metadata driven application development make it simple to develop and integrate applications written in many programming languages.

    PHP applications can be hosted on the Tuxedo platform without requiring any code changes and can leverage high availability, scalability and integration capabilities.

    The whitepaper Oracle Tuxedo - An Enterprise Platform for Dynamic Languages explains the architecture and features. A Tuxedo Demo VM for VirtualBox is available for immediate testing.

    Install Now!

  • Client Libraries : Oracle Instant Client -- a small footprint set of libraries that allows applications and tools to connect to an existing Oracle Database. Oracle OCI, OCCI, Pro*C, ODBC, and JDBC applications work without modification.

    The PHP OCI8 extension can be built with Oracle Instant Client to connect to a remote database.

    Instant Client is provided under a separate OTN Development and Distribution License for Instant Client that allows most licensees to download, redistribute, and deploy in production environments, without charge.

    Install Now!

About

Tourists looking out over an Opal mine
I'm a Product Manager in Server Technologies, working on scripting languages and developer-access.
Email: christopher.jones@oracle.com
Twitter: http://twitter.com/ghrd
Book: Free PHP Oracle book
Download: PHP Linux RPMs with the OCI8 extension
Links: OTN PHP Developer Center

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today