Friday Mar 14, 2014

Performance improvement for OCI_RETURN_LOBS in PHP OCI8 2.0.8

Reducing "round trips" to the database server is important for performance and ultimately for system scalability. A server round-trip is defined as the trip from PHP to the database server and back to PHP.

Reducing round trips makes everything more efficient: PHP is faster, there is less network overhead, the database doesn't have to do a context switch or do any processing for you. Applications have some control over round trips, for example by effectively using prefetching or by using an appropriate execute mode to minimize unneccessary rollbacks at the end of a script.

The bug filer of Bug 66875 noticed that PHP OCI8's implementation itself could reduce round trips if a particular LOB column meta data value was cached for the duration of a query instead of being re-requested for each row of the query result set.

So, now with OCI8 2.0.8, you should see a performance increase if you are doing multi-row queries involving LOB columns returned as OCI_RETURN_LOBS:

$s = oci_parse($c, "select mylobcol from mylobtab");
oci_execute($s);
while (($row = oci_fetch_array($s, OCI_ASSOC+OCI_RETURN_LOBS)) !== false) {
    echo $row['MYLOBCOL'], "\n";
}

The bug filer tested the patch and added his performance improvement benchmark results to the bug report. The benefit in your environment will vary greatly with the network setup, schema, and how many LOB columns are queried. Test it out and let me know how the new version helps you.

There is no immediate change for LOBs fetched with OCI-Lob::read() and OCI-Lob::load(). This would require a more complex patch than I want to apply at this time. Queries that don't use LOBs are not affected in any way by the patch.

OCI8 2.0 is included in the forthcoming PHP 5.6 code base. For PHP 5.2 to PHP 5.5 you can install it from PECL. PHP 5.5 RPMs with PHP OCI8 2.0.8 are available from oss.oracle.com.

Finally, if your LOB queries return multiple rows, you might also like this tip to reduce PHP memory usage.

Thursday Jul 25, 2013

PHP OCI8 2.0.0-devel for Oracle Database is on PECL

PHP OCI8 2.0.0-devel is now available on PECL (and in the PHP 'master' branch). This is a development release, so changes may occur. I'm looking for feedback, particularly on:

  • Feature design, including naming choices
  • Install feedback from different platforms with different build tool versions
  • Whether to drop support for installing on PHP 4. I'm likely to do this. (The OCI8 1.4.10 release is usable for installing on PHP 4)

OCI8 2.0 can be installed as a shared extension from PECL with:

pecl install oci8-devel

Alternatively, if you have OCI8 1.x currently installed as a static extension, then you will need to build PHP again. This is easy using a trunk snapshot from snaps.php.net. Instructions for building are in various places on the web, including in The Underground PHP and Oracle Manual.

The OCI8 2.0.0-devel release contains a bunch of clean ups, and some new and changed features. The full list is found in the package.xml file:

    - NEW FUNCTIONALITY:
 
      - Added Implicit Result Set support for Oracle Database 12c.
	Streaming of all IRS's returned from a PL/SQL block is available
	via oci_fetch_array, oci_fetch_assoc, oci_fetch_object and
	oci_fetch_row (but not oci_fetch or oci_fetch_all).
	Alternatively individual IRS statement resources can be obtained
	with the new function 'oci_get_implicit_resultset' and passed to
	any oci_fetch_* function.

      - Added DTrace probes enabled with PHP's generic --enable-dtrace

    - IMPROVED FUNCTIONALITY:
 
      - Using 'oci_execute($s, OCI_NO_AUTO_COMMIT)' for a SELECT no
	longer unnecessarily initiates an internal ROLLBACK during
	connection close.  This can improve overall scalability by
	reducing "round trips" between PHP and the database.
 
    - CHANGED FUNCTIONALITY:
 
      - PHPINFO() CHANGES:
 
        - The oci8.event and oci8.connection_class values are now shown
          only when the Oracle client libraries support the respective
          functionality.
 
        - Connection statistics are now in a separate phpinfo() table.
 
        - Temporary LOB and Collection support status lines in
          phpinfo() were removed.  These features have always been
          enabled since 2007.
 
      - OCI_INTERNAL_DEBUG() CHANGES:
 
        - The oci_internal_debug() function is now a no-op.  Use PHP's
          --enable-dtrace functionality with DTrace or SystemTap instead.
 
    - INTERNAL CHANGES:
 
      - Fixed a potential NULL pointer dereference flagged by Parfait
        static code analysis.
 
      - Extended testing of existing OCI8 functionality.
 
      - Improved test output portability when using the PHP development
        web server to run tests.
 
      - Removed no-longer necessary Unicode patterns from tests
        (vestiges of PHP's previous PHP 6 project)
        
      - Improved build portability by removing compilation type cast
        warnings with some compilers.
 
      - Fixed compilation warnings when building with Oracle 9.2
        client libraries.
 
      - Updated code to use internal macro PHP_OCI_REGISTER_RESOURCE.
 
      - Regularized code prototypes and fixed some in-line documentation
        prototypes.
 
      - Fixed code folding. 

Subsequent blog posts will talk about the features in OCI8 2.0.0.

Tuesday Feb 12, 2013

The Mysterious PHP RFC Process and How You Can Change the Web

The PHP RFC process has been in place for a while, and users new to core PHP development are starting to use RFCs to propose desirable features.

Here are some personal observations and suggestions that show how I have seen feature acceptance and the RFC process work in practice. These notes augment the steps in How To Create an RFC. I hope they help set expectations about the PHP RFC process and feature acceptance in the PHP language.

  1. Before starting an RFC, review existing RFCs and search the mail list archives for similar ideas. Use this information to explain what your RFC will bring to the language.

  2. If you're new to PHP core development, mail the "internals" mail list to judge interest before you start your RFC. If you get no reaction (this is likely) or positive reaction then create the RFC. Core PHP developers with karma will know when they can skip this step.

  3. Avoid presenting an RFC idea to the "internals" mail list with email that begins "I don't know much about ... but ...". Do some research first.

  4. There are many really good ideas for improving PHP, however some of them are really tedious or technically risky or hard. If you are about to email the "internals" mail list saying "someone should do ...", then don't hit "Send". Work out how you could do it, and then send a patch.

  5. If the overall goals of PHP are not clear to you, you may not be alone. However, as an RFC writer you need to learn the general trajectory of the language and where web development is heading. You then need to enthuse everyone with your feature and show what it brings to the language.

  6. Don't start an RFC (or mail list discussion) about standardizing PHP function names and function argument orders. There are several historical reasons why the functions are what they are (including compatibility with underlying libraries), and many good reasons why the change would be counter-productive causing more end-user confusion, would lead to unmaintainable PHP engine code, and generally be a waste of everyone's time when they could be doing more interesting projects. This has been discussed ad infinitum. Review previous discussions and feel free to fork PHP on github.

  7. Your RFC should talk about all PHP areas that will be affected: php.ini, different SAPIs, engine, extensions, etc. List similar features. List similar features in other languages. Link to references. Give an estimate of the actual positive impact to user code. Put the proposed voting options in the RFC so they can be included in its discussion. Include the proposed acceptance criteria (2/3 or 50%+1 majority - see the original Voting RFC) to avoid later argument.

  8. Your RFC will be used to create the PHP documentation, so make its sections re-usable. Explain concepts and details. Keep the RFC technical and have plenty of examples.

  9. Many current PHP RFCs don't contain enough detail, nor do they capture the pros & cons that were discussed on the mail list. You can, and should, do better than these RFCs.

  10. An implementation is not mandatory while proposing an RFC but it can help persuade people and help fine-tune the design. Note that if you are unable to implement the RFC, and no one else volunteers during the discussion period, then your RFC is unlikely ever to be implemented.

  11. Don't start an implementation too early. Get support for the feature concept from the "internals" mail list before spending time on coding - unless you want a learning experience.

  12. If you do have an implementation, make it clear whether the implementation is a simple prototype or is expected to be the final code. This is specially important during the vote.

  13. It's easy to get sidetracked by mail list trolling or well intentioned but over-eager discussion. Take advice on board but make sure your feature doesn't end up being designed by a committee. Don't blame the "PHP core community" for derailing discussions when the issues of non- code-contributors on the mail list are at fault.

  14. For every user who wants a backwardly incompatible change to PHP, there is a user who will scream that this will be disastrous to them. As the RFC writer you need to walk the line between these groups. At voting time, other people may see the line in a different place than you do.

  15. There is no need to respond to every discussion email individually. You should batch up your responses and manage the discussion intelligently.

  16. Don't get side tracked by how long it will take a feature to reach end users. It will reach them eventually. You can always supply an unofficial patch or maybe create a PECL extension for users of current versions of PHP.

  17. Don't let mail list discussion drag on too long. High levels of discussion can be a symptom that an RFC is contentious. Having an early vote about only the feature's concept can prevent over-investment in an RFC and implementation.

  18. With long, fragmented discussions, not everyone will read every email. Update the RFC at regular intervals, and let people know what has changed. Include the RFC URL in all your emails so anyone dipping into a mail thread has a better chance of finding the RFC.

  19. PHP is popular so you can often find support from "many" people for cool language features. This doesn't mean those features are a "good thing" for the language or its implementation.

  20. There are multiple PHP communities. Don't interpret support in one community as universal support. Listen carefully to the key PHP developers on the "internals" mail list (they may not be the ones doing the most discussion). They are the people who will undoubtedly end up doing final integration of the feature (e.g. with the op-code cache), fixing edge cases, and maintaining it in the long term. They are the ones that have seen the PHP code base complexity grow and become more difficult to maintain and extend. Even if they like your idea, they may not be able to contribute time and effort to make it stable.

  21. The PHP core development community needs more end-users to become core language developers but the barrier to entry is high because of the complexity of the API, and because new features need very careful integration with a complex language. See the point above about existing developers not having enough time to support every good idea. This just means you need perseverance to become a core developer yourself. You can gain karma by being a regular code contributor before starting your magnum opus RFC.

  22. Some areas of PHP are complex or niche. Sometimes feature suggestions will be greeted by an apparent lack of interest. Don't be discouraged. This just means you need to take a stronger leadership role, and also prove your credentials by first working on the existing code base.

  23. Keep your RFC updated with a summary of all the positive and negative mail list discussion points and examples.

    • This prevents arguments being rehashed on the mail list each year.
    • Documentation will have clear use cases and show what does and doesn't work.
    • Future RFC writers can see trends and build on previous ideas.
  24. Before initiating a vote on your RFC, make sure the RFC contains enough detail so that the vote is solely on the RFC contents, not on any half-remembered or misinterpretable mail list discussions. Un- (or badly) expressed intentions will only cause you frustration when your RFC is rejected.

  25. Don't leave any ambiguity in the RFC. As well as stating what will be changed, also state what won't be changed. Ambiguity will hurt the chances of a successful outcome because voters will be unsure that the feature has been fully thought through. Make sure there are no "Open Issues" in the RFC when voting. Make a decision about the choices before opening the vote, or mark the issues clearly as "Future topics for exploration".

  26. Make sure any vote is clear about what is being voted on: is it the idea, the specification, or the implementation? Is a particular version of PHP being targeted? Is the implementation going to be merged to the core language or made as a PECL extension?

  27. Warn the "internals" mail list before starting the vote. This notification gives you a chance to fine tune the wording of your poll; this wording has caused contention in the past. The approaching deadline may also cause last-minute RFC responses. These may be significant enough to require further discussion.

  28. Set a deadline for the vote (sadly the current voting RFC doesn't require this). Having a deadline will forestall suggestions that a vote was deliberately left open until it succeeded, and then closed before the "negative" side could rally support.

  29. During the voting period, it is common for people to continue mail list discussion. You may need to halt the vote and address any issues.

  30. Prior to the end of the voting period, remind the mail list that the vote is about to close.

  31. Positive voting poll outcomes for RFs that are just "concepts" can be interpreted as support for "good ideas" rather than being a definitive guarantee that a feature will appear in a future version of PHP. As a feature is later investigated and further discussed, the early vote may become irrelevant. Similarly, where there is no chance of an implementation being written, a positive poll outcome is just an indicator of desire.

  32. If your RFC is rejected, it is not the end of the world. Sometimes ideas are submitted "before their time", and it takes an acclimatization period for other developers to see their value. As the core PHP developer base goes through natural turn-over, revisiting an idea after a (long) period may result in different appreciation. (This does not apply to some ideas, for example standardizing function names). See the previous points about becoming a core contributor - having karma goes a long way towards getting an RFC accepted, not only because experienced contributors know which RFCs are worth suggesting, and know how to make proposals understandable. When your RFC is rejected, continue working with the PHP core development community and revisit the RFC later.

  33. You can always fork PHP from github.

In summary, the PHP development process is an organic process and subject to flexibility. It is based heavily around users contributing features that they require. This therefore requires high investment from users who want change. Very rarely do PHP core developers have bandwidth to take on external ideas, no matter how good they are. Feature acceptance has to be pragmatic. The core PHP contributors would love to see more people get commit karma and contribute to PHP.

This post has gone on long enough. My notes are current for the time being. I'm sure there are observations or clarifications you can make. If you want to add anything, please post a comment.

(Note: this page has been augmented and re-ordered since first being published)

Monday Sep 26, 2011

Scripting Language Related Sessions at Oracle OpenWorld and JavaOne, October 2011

Oracle OpenWorld and JavaOne conferences are happening in San Francisco next week. It will be a busy and exciting time.

First, here's a shout out: For me the conference kicks off on Sunday morning. Marcelle Kratochvil from Piction (heavy users of PHP and Oracle DB) is hosting the inaugural Unstructured Data with Multimedia SIG for Oracle Database and MySQL database (32440) Sunday 9:00 am in Moscone West room 2011.

Below are some of the scripting and related sessions happening during the week.

Exhibition Hall

During the Exhibition Hall hours, come and talk to us at the Database Access Services and APIs booth. This year we're in Moscone South, Left SL-067. The Tuxedo application server booth is Moscone South, Right - SR-202. At JavaOne look out for the NetBeans booth, Hilton San Francisco - HHJ-023.

Scripting Sessions, Birds-of-a-Feather Meetings, and Hands-on-Labs at OOW

  • The Oracle Tuxedo team has scripting language support in their powerful application server environment:
    High-Performance Web Applications with C/C++/PHP/Python (15705)
    Monday, 05:00 PM, Moscone South - 300
  • This year we are running introductory Hands-on Lab sessions for three languages concurrently. Come along and choose which language you'd like to dip your toes into:
    Develop and Deploy High-Performance Web 2.0 PHP, Ruby, or Python Applications (30082)
    Monday, 05:00 PM, Marriott Marquis - Salon 10/11
  • Come and ask questions at the round table Birds-of-a-Feather session:
    Meet the Oracle Database Clients Developers: C, C++, PHP, Python, Ruby, and Perl (26240)
    Monday, 07:30 PM, Marriott Marquis - Salon 8
  • My overview and state-of-the-nation session is:
    PHP, Ruby, Python, and Perl: Develop and Deploy Mission-Critical Apps with Oracle Database 11g (14704)
    Wednesday, 11:45 AM, Marriott Marquis - Salon 8
  • The Tuxedo team Hands-on-Lab lets you code in C/C++/PHP/Python/Ruby:
    Develop High-Performance, Service-Oriented C/C++ Applications for Oracle Exalogic (31120)
    Thursday, 12:00 PM, Marriott Marquis - Salon 3/4
  • Raimonds Simanovskis, maintainer of the Rails adapter for Oracle is giving a session:
    Extending Oracle E-Business Suite with Ruby on Rails (8604)
    Thursday, 03:00 PM, Moscone West - 2002/2004

Several other sessions discuss topics that scripting language devotees will find invaluable:

  • Build, Deploy, and Troubleshoot Highly Performant, Highly Available Apps with Oracle Database (14703)
    Wednesday, 05:00 PM, Moscone South - 303
  • Net Services: Best Practices for Performance, Scalability, and High Availability (14345)
    Wednesday, 01:15 PM, Moscone South - 303

Also check out the full Oracle Tuxedo application server schedule here.

Scripting at JavaOne

Over in the concurrent JavaOne conference there are several scripting sessions driven by San Francisco's EngineYard. This year they have JRuby sessions but with their recent aquisition of PHP technnology, I hope they'll have more on PHP in one of the OOW streams next year:

  • Accelerate Your Business and Aim for the Cloud with Java and JRuby (25284)
    Wednesday, 03:00 PM, Parc 55 - Embarcadero
  • From Java to Rails: Techniques for Adding Ruby Agility to Your Java Web Stack (24582)
    Monday, 05:30 PM, Parc 55 - Market Street
  • Real-World JRuby (23600)
    Wednesday, 04:30 PM, Parc 55 - Market Street
  • Script Bowl 2011: A Scripting Languages Shootout (22060)
    Wednesday, 08:30 AM, Hilton San Francisco - Grand Ballroom B

Also keep an eye out for the various NetBeans IDE sessions and demo booth.

Linux

Check out the four pages of Focus on Linux sessions and events.

MySQL

There is a veritable plethora of MySQL content - four pages of sessions and activites are listed in the Oracle Focus on MySQL. Don't forget the MySQL Community Reception Tuesday 7:00pm - 9:00pm in the Marriott Marquis - Foothill G.

Having started this post with a shout out, let me end with one to Bill Karwin, who was instrumental in getting PHP's Zend Framework off the ground. He is talking about MaatKit and SQL Injection.

Update: Ligaya Turmelle, well known in the PHP community, confirmed overnight that despite a recent job move she is still on track to present her MySQL Performance Tuning talk (16040).

You can search the OOW session catalog here and the JavaOne session catalog here.

About

Tourists looking out over an Opal mine
I'm a Product Manager in Server Technologies, working on scripting languages and developer-access.
Email: christopher.jones@oracle.com
Twitter: http://twitter.com/ghrd
Book: Free PHP Oracle book
Download: PHP Linux RPMs with the OCI8 extension
Links: OTN PHP Developer Center

Search

Archives
« July 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
  
       
Today