X

Recent Posts

Miscellaneous

Cache, cache, cache! (Part 3: What to cache?)

Happy New Year everyone! Hope you could enjoy your holidays!!Let's start this year with the third part of my little series of thoughts about caching. After my small memcached intro and thoughts about caching architectures, I now focus on the data you should consider to cache in your web application.[1] Cache HTMLObviously the biggest performance win you can achieve is by caching the whole output of your web application: a simple reverse proxy scenario. This works very well for mostly static pages, but for highly dynamical and user-specific content this is not an option: there is no advantage in caching a web page, which gets obsolete within the next moment.Probably the best way to solve this dilemma is to implement a so-called partial-page cache: Let your application cache just portions of the page and leave the rest, where it makes no sense to cache, dynamic.It's very important that you implement this in a very top layer of your application. Probably exactly that layer, which software architects will call presentation layer. Sure, this is likely to break you framework architecture, but to quote chapter 55 of the Tao Te Ching: The movement of the TaoBy contraries proceeds;And weakness marks the courseOf Tao's mighty deeds.But seriously: If you have to stay in the boundaries of a framework, Ajax is a good way to bypass this restrictions and helps to implement such a cache in a restricted architecture. But be aware that this will raise the number of HTTP requests on your frontend web servers.An effective caching strategy will always mess your beautifully designed software architecture up. Having just one (central) caching layer looks great in system diagrams and it's better than no cache at all, but it's definitely not the end of the rope.[2] Cache complex data structuresIf you don't want to break your framework architecture or you don't like the idea of caching HTML at all, and I totally understand your point, you should consider about caching other (lower level, but still complex) structures of data.Some examples for suitable data structures:user profilesfriends listscurrent user listslist of locations, branches, countries, languages, ...top 10 (whatever) listspublic statistical data...The main challenge lies in identifying the most proper data structures. This is no easy task and strongly depends on the kind of web application you run or plan to run. Avoid caching simple data sets, like row-level data from the database. Don't think row-level.That's the best advice you should keep in mind. (Note to myself: I need to put this on a t-shirt. I found this phrase in Memcached Internals, a wonderful article inspired by a talk by Brian Aker and Alan Kasindorf.)At a first glance Ajax may be an obvious technology to combine with such a cache. But please be aware that moving application logic away from the server-side application to the client side is always a very dangerous task, which easily may compromise the security of your application.Which allows me to end this post with another quote from Laozi (Tao Te Ching, chapter 63):All difficult things in the worldare sure to arise from a previous statein which they were easy.

Happy New Year everyone! Hope you could enjoy your holidays!! Let's start this year with the third part of my little series of thoughts about caching. After my small memcached intro and thoughts about...

Miscellaneous

Cache, cache, cache! (Part 2: Architectures)

On Tuesday I focused mainly on memcached and PHP, but today I'll take a wider look at cacheing architectures in general. The main question about defining a cache architecture is to decide where to locate the caching component:[1] Status quo, the three-tier architectureIn theory, the commonly accepted standard architecture of a software product is divided into three tiers: the presentation tier, the application tier and finally the data tier. In the context of web applications we rediscover this tiers in the trinity of web server, application server and database server.In the above diagram we find these three tiers with the user (or in technical terms: the browser) on top of this stack.[2] Cache on topOne very obvious idea is to place the cache in front of the web server, between user and web server. Usually we find this architecture in a so called reverse proxy configuration. A reverse proxy is quite easy to set up and has a positive impact for web sites with more static content. But for highly dynamic web applications - like most of today's Web 2.0 applications - the caching benefit of a reverse proxy may be not that big.In general: having a reverse proxy is better than no caching at all. A reverse proxy will give you always a performance benefit.[3] Cache in between of web and application serverLet's move the cache one level down the stack in between web server and application server. On the fist sight this may look like a very good idea, because the cache now protects the application server. But on the second sight you'll realize that this configuration is mostly the same as that one from architecture 2, just without the benefit also caching your web server's data.For exotic scenarios there may be a good reason for this configuration (esp. in combination with load balancing functionality) but in general you should favor architecture 2 over this one.[4] Cache in between of application and databaseAnd another level down in the stack. The cache now sits between application server and database. Again this looks good, and seems to be a good idea - on the first sight. But on the second or third sight you may realize that nearly every database system has its own internal query cache and our cache is only a cache for a cache. And caching a cache is basically never a good idea and can lead to unpredictable, bad consequences.Another difficulty with this approach is that it's hard to decide when the cache gets dirty (cache jargon for obsolete) and when it's time to clear the cache.[5] Cache inside of applicationAnd now half a level up again: right into the application tier. This is the most challenging but also the most powerful place to implement caching strategies. Identify time-critical and frequently accessed data during the development process and implement dedicated and customized caching mechanisms. But don't try do build an abstract, unified, common cache for everything.It's very important to find a specific and suitable solutions for each kind of data you want to cache in your application. Otherwise you'll will probably just end with another row-based cache for your database (like architecture 4) or some kind of reverse proxy (like architecture 2).ConclusionArchitectures 2, 3 and 4 can be easily setup by system administration without having to involve development in any way. It's mostly a matter of clever configuration which also may add some load balancing features. In general you'll definitely achieve a better performance of your application, but there is always a given limit by the architecture and scaling quality of your core application.Architecture 5 is probably the best choice, but - to get best results - needs to be started in an early stage and during the whole development and designing process of your web application you should always have caching in mind. What data is most frequently accessed? What data is expensive (hard to retrieve)? What data depends on user sessions? How up to date does the data need to be?If you are curious about these questions, please stay tuned for part 3.

On Tuesday I focused mainly on memcached and PHP, but today I'll take a wider look at cacheing architectures in general. The main question about defining a cache architecture is to decide where to...

Miscellaneous

Cache, cache, cache! (Part 1: memcached)

Caching is probably the most important technique you should use in nowadays web sites or web application. Sure, scaling your hardware is still the final answer to all your load problems, but with some kind of caching your application will scale far better rather than without.Currently my favorite caching tool is memcached. It's a slim and ultra fast distributed caching system. Memcached is basically a key-value store, which stores all data non-persistently in memory and if your server goes down all the data is also gone because it's not stored somewhere on a hard disk.Memcached is not meant to be a database, and you'll still need a database to store your data persistently.Setting up memcachedI'm a very lazy guy and try to avoid boring duties like installing memcached. That's why I love using Sun's Web Stack, which already includes memcached and is so easy to use. If you're not a Web Stack user please take a look at the memcached FAQ to learn how to install memcached on your system.To add memcached to my IPS-based Web Stack installation I simply call these two commands:[oswald@localhost ~/demo]$ bin/pkg install sun-memcachedDOWNLOAD PKGS FILES XFER (MB)Completed 1/1 9/9 0.17/0.17 PHASE ACTIONSInstall Phase 30/30 PHASE ITEMSReading Existing Index 7/7Indexing Packages 1/1[oswald@localhost ~/demo]$ bin/setup-webstackNow all I need to do is to start the daemon:[oswald@localhost ~/demo]$ bin/sun-memcached startStarting memcachedMemcached has no support for any access control at all and you should use memcached only on private networks or secure you installation with a firewall (port 11211, by the way).Using memcached with PHPAs I already mentioned memcached is a simple key-value store which is very easy to use for programmers. To show the basic idea I put this small PHP script together:<?php $memcache = new Memcache(); if(!$memcache->connect('localhost', 11211)) die("Couldn't connect to memcached! Cruel world!"); $key="zaphod"; $result = $memcache->get($key); if($result) { echo "$key is $result"; } else { $value="cool"; echo "Set $key to $value";$memcache->set($key,$value); }?>There are three main functions you will need to understand in order to work with memcached:connect(host,port)to connect to your memcached server. If you have multiple memcached servers running you can use addServer() to add one or more servers to the connection pool.get(key) Retrieves the value for the given key.set(key,value) Stores the given value for the given key. set() also allows you to define an expiration time for the key-value pair.On the first execution of this script the cache is empty and you'll get this output:Set zaphod to coolOn the second execution, the value for zaphod is already set and you'll see: zaphod is coolThat's all. That's the basic way to use memcached.What's next...The next step is to decide what information you want to cache and where do you want to cache. Both are very crucial decisions which determine success or failure of your cache. So, stay tuned for part 2. ;)

Caching is probably the most important technique you should use in nowadays web sites or web application. Sure, scaling your hardware is still the final answer to all your load problems, but with some...

MySQL

PHP's MySQLi extension: Storing and retrieving blobs

There are a lot of tutorial out there describing how to use PHP's classic MySQL extension to store and retrieve blobs. There are also many tutorials how to use PHP's MySQLi extension to use prepared statements to fight SQL injections in your web application. But there are no tutorials about using MySQLi with any blob data at all.Until today... ;)Preparing the databaseOkay, first I need a table to store my blobs. In this example I'll store images in my database because images usually look better in a tutorial than some random raw data.mysql> CREATE TABLE images (id INT(10) UNSIGNED NOT NULL AUTO_INCREMENT,image MEDIUMBLOB NOT NULL,PRIMARY KEY (id));Query OK, 0 rows affected (0.02 sec)In general you don't want to store images in a relational database. But that's another discussion for another day.Storing the blobTo make a long story short, here's the code to store a blob using MySQLi:<?php$mysqli=mysqli_connect('localhost','user','password','db');if (!$mysqli)die("Can't connect to MySQL: ".mysqli_connect_error());$stmt = $mysqli->prepare("INSERT INTO images (image) VALUES(?)");$null = NULL;$stmt->bind_param("b", $null);$stmt->send_long_data(0, file_get_contents("osaka.jpg"));$stmt->execute();?>If you already used MySQLi, most of the above should look familiar to you. I highlighted two pieces of code, which I think are worth looking at:The $null variable is needed, because bind_param() always wants a variable reference for a given parameters. In this case the "b" (as in blob) parameter. So $null is just a dummy, to make the syntax work.In the next step I need to "fill" my blob parameter with the actual data. This is done by send_long_data(). The first parameter of this method indicates which parameter to associate the data with. Parameters are numbered beginning with 0. The second parameter of send_long_data() contains the actual data to be stored.While using send_long_data(), please make sure that the blob isn't bigger than MySQL's max_allowed_packet:mysql> SHOW VARIABLES LIKE 'max_allowed_packet';+--------------------+----------+| Variable_name | Value |+--------------------+----------+| max_allowed_packet | 16776192 | +--------------------+----------+1 row in set (0.00 sec)If your data exceeds max_allowed_packet, you probably don't get any errors returned from send_long_data() or execute(). The saved blob is just corrupt!Simply raise the value max_allowed_packet to whatever you'll need. If you're not able to change MySQL's configuration, you'll need to send the data in smaller chunks:$fp = fopen("osaka.jpg", "r");while (!feof($fp)) { $stmt->send_long_data(0, fread($fp, 16776192));}Usually the default value of 16M should be a good start.Retrieving the blobGetting the blob data out of the database is quite simple and follows the usual way of MySQLi:<?php$mysqli=mysqli_connect('localhost','user','password','db');if (!$mysqli)die("Can't connect to MySQL: ".mysqli_connect_error());$id=1; $stmt = $mysqli->prepare("SELECT image FROM images WHERE id=?"); $stmt->bind_param("i", $id);$stmt->execute();$stmt->store_result();$stmt->bind_result($image);$stmt->fetch();header("Content-Type: image/jpeg");echo $image; ?>Connect to the database, prepare the SQL statement, bind the parameter(s), execute the statement, bind the result to a variable, and fetch the actual data from the database. In this case there is no need to worry about max_allowed_packet. MySQLi will do all the work:By the way...If you want to insert a blob from the command line using MySQL monitor, you can use LOAD_FILE() to fetch the data from a file:mysql> INSERT INTO images (image) VALUES( LOAD_FILE("/home/oswald/osaka.jpg") );Be aware that also in this case max_allowed_packet limits the amount of data you're able to send to the database:mysql> SHOW VARIABLES LIKE 'max_allowed_packet';+--------------------+-------+| Variable_name | Value |+--------------------+-------+| max_allowed_packet | 7168 | +--------------------+-------+1 row in set (0.00 sec)mysql> INSERT INTO images (image) VALUES( LOAD_FILE("/home/oswald/osaka.jpg") );ERROR 1048 (23000): Column 'image' cannot be nullmysql> SET @@max_allowed_packet=16777216;Query OK, 0 rows affected (0.00 sec)mysql> SHOW VARIABLES LIKE 'max_allowed_packet';+--------------------+----------+| Variable_name | Value |+--------------------+----------+| max_allowed_packet | 16777216 | +--------------------+----------+1 row in set (0.00 sec)mysql> INSERT INTO images (image) VALUES( LOAD_FILE("/home/oswald/osaka.jpg") );Query OK, 1 row affected (0.03 sec)

There are a lot of tutorial out there describing how to use PHP's classic MySQL extension to store and retrieve blobs. There are also many tutorials how to use PHP's MySQLi extension to use prepared...

Miscellaneous

Importing a VDI in VirtualBox

If you're used to be a VMware user and try to switch to the Open-Source side of the Force by using VirtualBox, you may run into difficulties if you try to import an existing VDI file into VirtualBox. Actually it's quite easy, if you know how.The main difference between VMware and VirtualBox is that VMware captures a whole virtual machine in an image, whereas VirtualBox only supports images of a hard disk. So in VirtualBox's world, you first need to create a new virtual machine, before using an existing VirtualBox image.First copy your VDI file into VirtualBox's virtual hard disks repository. On Mac OS X it's $HOME/Library/VirtualBox/HardDisks/.Start VirtualBox and create a new virtual machine (according to the OS you expect to live on the VirtualBox image):When you're asked for a hard disk image, select Use existing hard disk and click on the small icon on the right:Which will brings you to the Virtual Media Manager. Click on Add and select the VDI file from step 1. After leaving the Virtual Media Manager, you'll be back in your virtual machine wizard. Now you can select your new VDI as existing hard disk and finalize the creation process.Back in the main window, you're now able to start your new virtual machine:It's quite easy, if you know how.

If you're used to be a VMware user and try to switch to the Open-Source side of the Force by using VirtualBox, you may run into difficulties if you try to import an existing VDI file into VirtualBox....

PHP

Store PHP sessions in memcached

My last week blog topic was very much marked by Apache load balancing. Well, I promised to leave this topic alone for a while, but there is one related topic that is worth spending a minute on.The TheoryIf your web application is distributed across multiple servers you'll quickly run in sessions problems because each backend server (aka worker) usually stores its session informations locally.Now, if subsequent HTTP requests are handled by different workers, every time a new sessions is created or, even worse, sessions getting mixed up.To overcome this problem there are two solutions: Use a session-aware load balancer that binds a user session to the same worker.Or keep all session data in a central storage.Both solutions have the similar drawback: if a worker goes down, all session data of this worker are lost. If the central storage goes down, all sessions are lost. But consider the following: you'll probably have tons of workers, and since every computer is supposed to fail after a specific period of time, the probability of a worker failure is much higher than for a single storage server. It depends on what do you want: A system that runs all the time with small failures or a system that fails completely from time to time?And finally, losing session data sounds worse than it actually is: usually the users only have to login again to restore their session data. That's sad, but it's not the end of the world. Okay, your system may get into trouble if thousands of users try to re-login at the same time, but that's another problem. The SolutionMy favorite solution is the second one: keep all session data in a central place. And in this scenario I'll use Apache/PHP as my "application server" and memcached as central storage for my session data. If you read and still remember the title of this post, you're probably not surprised.On the left: my load balancer, in the middle my worker farm, and on the right: my single and central memcached server. By the way: You can also have multiple memcached servers, but for this blog post I'll keep it simple.The RequirementsFirst, let's check if PHP was build with memcached support:serverA ~% php -m | egrep memcachememcache...on each worker node: serverA to serverD.Second, I check if memcached is running on serverM:serverM ~% ps -efa | egrep memcachedoswald 1543 1 0 15:21:17 ? 0:00 /home/oswald/webstack1.5/lib/memcached -d ...Perfecto.The ConfigurationNow I need to change the PHP configuration on each worker node: Open php.ini on serverA to serverD and search for these lines:[Session]; Handler used to store/retrieve data.session.save_handler = filesAnd change the configuration like this:[Session]; Handler used to store/retrieve data.session.save_handler = filessession.save_handler = memcachesession.save_path = "tcp://serverM:11211"Make sure that the settings are the same on all your workers.That's all. Yes, that's the basic configuration. PHP's sessions will now get stored on the memcached node serverM. No more magic needed.The ProofBut as we say in Germany: "Prudence is the mother of the china cabinet." Before we can grab the beer, we should make sure everything works as we expect it to.I put this code in a file named session.php in the document root directory of all my worker nodes:<?php session_start();if(isset($_SESSION['zaphod'])){ echo "Zaphod is ".$_SESSION['zaphod']."!\\n";} else { echo "Session ID: ".session_id()."\\n"; echo "Session Name: ".session_name()."\\n";echo "Setting 'zaphod' to 'cool'\\n";$_SESSION['zaphod']='cool';} ?>From the outside I use lynx to access this file:% lynx -source 'http://serverA/session.php'Session ID: df58bc9465f27aa20218c11caba6750fSession Name: PHPSESSIDSetting 'zaphod' to 'cool'A new session with the ID df58bc9465f27aa20218c11caba6750f was created and PHP uses the session name PHPSESSID to identify the session parameter. And the session variable zaphod was set to the value cool.Now I add the session information PHPSESSID=df58bc9465f27aa20218c11caba6750f to my URL and rerun the new lynx command:% lynx -source 'http://serverA/session.php?PHPSESSID=df58bc9465f27aa20218c11caba6750f'Zaphod is cool!Yes, I got the expected output: Zaphod is cool! Proving the session data is available on serverA. But that's not a big surprise, what's about the other nodes? I replace serverA with serverB in my URL:% lynx -source 'http://serverB/session.php?PHPSESSID=df58bc9465f27aa20218c11caba6750f'Zaphod is cool!Bingo, serverB also has the same session data as serverA.And for serverC? It's also the same:% lynx -source 'http://serverC/session.php?PHPSESSID=df58bc9465f27aa20218c11caba6750f'Zaphod is cool!And so on... for each worker node the session data will be the same.A dream came true.

My last week blog topic was very much marked by Apache load balancing. Well, I promised to leave this topic alone for a while, but there is one related topic that is worth spending a minute on. The...

Apache

Easy HTTP load balancing with Apache

Usually a single AMP system is enough to serve - let's say - around 500 concurrent users. Sometimes more, sometimes less, strongly depending on the particular web application, the overall architecture of your system, of course the hardware itself, and how you define "concurrent users".Nevertheless, if your server gets too slow, you'll need to take actions. You may upgrade your server up to the maximum (aka vertical scaling), optimize your software (aka refactoring), and finally add more servers (aka horizontal scaling). The whole process of horizontal scaling is quite complex and far too much for a single blog post, but here's a first shot. Others will follow.Today I'll focus on one single aspect of horizontal scaling: an HTTP load balancer.On the left: a whole crowd of people ready to visit our web site. On the right: our server farm (called workers). And in the middle: our current hero, the load balancer. The purpose of the load balancer (in this case an HTTP load balancer) is to distribute all incoming requests to our backend web servers. The load balancer hides all our backend servers to the public, and from the outside it looks like a single server doing all of the work.The RecipeOkay, let's start. Step by step.Since version 2.2 the Apache web server ships a load balancer module called mod_proxy_balancer. All you need to do is to enable this module and the modules mod_proxy and mod_proxy_http:LoadModule proxy_module mod_proxy.soLoadModule proxy_http_module mod_proxy_http.soLoadModule proxy_balancer_module mod_proxy_balancer.soPlease don't forget to load mod_proxy_http, because you wouldn't get any error messages if it's not loaded. The balancer just won't work.Because mod_proxy makes Apache become an (open) proxy server, and open proxy servers are dangerous both to your network and to the Internet at large, I completely disable this feature:ProxyRequests Off<Proxy \*>Order deny,allowDeny from all</Proxy>The load balancer doesn't need this feature at all.Now I need to make sure all my backend web servers have the same content:serverA htdocs% cat index.htmlThis is A.serverB htdocs% cat index.htmlThis is B.serverC htdocs% cat index.htmlThis is C.serverD htdocs% cat index.htmlThis is D.Okay, in this case the content differs, but I need this to show how the load balancer works.And here's the actual load balancer configuration:<Proxy balancer://clusterABCD>BalancerMember http://serverABalancerMember http://serverBBalancerMember http://serverCBalancerMember http://serverDOrder allow,denyAllow from all</Proxy>ProxyPass / balancer://clusterABCD/The <Proxy>...</Proxy> container defines which backend servers belong to my balancer. I chose the name clusterABCD for this server group, but you are free to choose any name you want.And the ProxyPass directive instructs the Apache to forward all incoming requests to this group of backend servers.That's all? Yes, that's all. Here's the prove:# repeat 12 lynx -source http://loadbalancerThis is A.This is B.This is C.This is D.This is A.This is B.This is C.This is D.This is A.This is B.This is C.This is D.Each request to the load balancer is forwarded to one of the backend servers. By default Apache simply counts the number of requests and makes sure every backend server gets the same amount of requests forwarded.If you want to know more about available balancing algorithms please refer to Apache's mod_proxy_balancer manual.Did you ever imagine setting up a load balancer would be this easy? Of course, there is more to say about (HTTP) load balancing and much more about vertical scaling too, but this is only a blog posting and not a place for such an expansive reference. If time and space allows I'll go into further details on this in the near future.

Usually a single AMP system is enough to serve - let's say - around 500 concurrent users. Sometimes more, sometimes less, strongly depending on the particular web application, the overall architecture...

Sun

Roller: Category list with entry counter

Sun Microsystem's blogs.sun.com employee blogging site (affectionately named BSC) uses Apache Roller to manage the site and house all the blogs. Roller is an open source Java blog software that for example also drives the US Government's blog.usa.gov andthe IBM Developer Works blogs. But there is one feature I really missed: An entry counter in my sidebar's category list. It's a standard feature in the Wordpress world, but in Roller I didn't found any option enabling such a counter.But as they say, "If the mountain won't come to Mohammed, then Mohammed will go to the mountain": If the feature will not come to me, then I will need to take care by myself.And here is my mountain:#set($rootCategory = $model.weblog.getWeblogCategory("nil"))#set($cats = $rootCategory.getWeblogCategories())#foreach($cat in $cats) #set($entriesList = $model.weblog.getRecentWeblogEntries($cat.name, 500)) #set($count = $entriesList.size()) #if($model.weblogCategory && $model.weblogCategory.path == $cat.path) <li class="selected"><a href="$url.category($cat.path)">$cat.name ($count)</a></li> #else <li><a href="$url.category($cat.path)">$cat.name ($count)</a></li> #end#endIt's quite easy and straight forward: Get a list of all categories, for each category get all the blog entries (in this case limited to 500, because I don't know how this scales), count the entries, and finally generate some HTML.If everything would be that easy! :)

Sun Microsystem's blogs.sun.com employee blogging site (affectionately named BSC) uses Apache Roller to manage the site and house all the blogs. Roller is an open source Java blog software that for...

PHP

Typographic headlines with PHP (Part 2)

Yesterday, I started a small tutorial on how to implement typographic headlines with PHP. There were some aspects to be aware of, but in general it was an easy and straight forward process. The final result looked like this:But there was one big issue I had with my script: It was far to slow (33 requests per second) for use in a production environment. But today, I'll extend my previous script with a simple caching mechanism to make it ready for the real world.Welcome to the PleasuredomeBasically that's where I left yesterday:$font_file="./FFF Tusj.ttf";$font_size=64;$text = "An Example";$bb = imagettfbbox($font_size,0,$font_file,$text);$bb_width = $bb[4]-$bb[6];$bb_height = $bb[3]-$bb[5];$image = imagecreate($bb_width, $bb_height); $fillcolor = imagecolorallocate($image, 255, 255, 255); $fontcolor = imagecolorallocate($image, 69, 138, 186); imagefill($image, 0, 0, $fillcolor); imagettftext($image, $font_size, 0, abs($bb[6]), $bb_height-$bb[3], $fontcolor, $font_file, $text);header("Content-Type: image/png");imagepng($image);I'll put this in a function - let's say - fancyheadline($text) and will add two lines of code at the beginning of this function and change the last lines a little bit.The final script will look like this (changes highlighted):function fancyheadline($text){$cache="cache/".md5($text).".png";if(!file_exists($cache)){ $font_file="./FFF Tusj.ttf";$font_size=64;$bb = imagettfbbox($font_size,0,$font_file,$text);$bb_width = $bb[4]-$bb[6];$bb_height = $bb[3]-$bb[5];$image = imagecreate($bb_width, $bb_height); $fillcolor = imagecolorallocate($image, 255, 255, 255); $fontcolor = imagecolorallocate($image, 69, 138, 186); imagefill($image, 0, 0, $fillcolor); imagettftext($image, $font_size, 0, abs($bb[6]), $bb_height-$bb[3],$fontcolor, $font_file, $text); header("Content-Type: image/png");imagepng($image,$cache);} return '<img src="'.$cache.'" alt="'.htmlspecialchars($text).'">';}echo fancyheadline("An Example");What's happening here is that I first generate a MD5 hash of my headline string $text and check if a cache file containing this hash exists or not. If it doesn't exists, create the file containing the rendered headline image. If it exists, do nothing.At the end of the script I return a string consisting of an IMG HTML tag which would display the cached image file. (With htmlspecialchars () I convert special characters like ", &, < and > to their corresponding HTML entities, because these chars may break the validity of HTML.)Before I can run my script, I need to create the cache directory and give it appropriate permissions:% mkdir cache% chmod a+rwx cacheOn a production server you probably don't want to give write permissions to everyone, but will change the ownership of the directory to www-data or whatever user my web server runs as.Now it's time to access the script in my trusted browser:This looks exactly like my original yesterday's example above, but this time the script generates the headline image "in the background" and only outputs some HTML code referring to this image:<img src="cache/e6cf1c67e6acfa204bb784cd6b25839f.png" alt="An Example">In other words: I reduced the use and need of PHP as much as possible.Final words by ApacheBenchFirst let's check the PHP script itself:% ab -n 1000 http://demo/headline.phpThis is ApacheBench, Version 2.3...Document Path: /headline.phpDocument Length: 71 bytes...Requests per second: 1801.51 [#/sec] (mean)...1800 requests per second. Yes, that's what I wanted to hear. But it's easy to explain: The headline image is generated only once, upon the first request. And for all following requests the script only refers to the already generated image.And if I benchmark the image itself:% ab -n 1000 http://demo/cache/e6cf1c67e6acfa204bb784cd6b25839f.pngThis is ApacheBench, Version 2.3...Document Path: /cache/e6cf1c67e6acfa204bb784cd6b25839f.pngDocument Length: 7888 bytes...Requests per second: 3308.09 [#/sec] (mean)...Of course, because it's now just static data, and I'll get the most performance possible out of my server. I started with 33 requests per second and ended somewhere in between of 1800 and 3300 requests per second.Moving at one million miles an hour... Welcome to the Pleasuredome!

Yesterday, I started a small tutorial on how to implement typographic headlines with PHP. There were some aspects to be aware of, but in general it was an easy and straight forward process. The final...

PHP

Typographic headlines with PHP (Part 1)

My recent blog post about scaling images with PHP gave me the idea to write something about creating typographic headlines with PHP. At Apache Friends we're using this technique since many years to get rid of the usual boring and everywhere available "web fonts" like Helvetica, Times and Verdana.For this example I chose the font Tusj by Norwegian graphic designer Magnus Cederholm. Okay, this font will only work for very large headlines, but it's looks cool and it's perfect for this demo's purposes because the TTF file is very huge (1.5 MB) and that makes the processing in PHP quite slow. (Yes, in this case, I want to slow down my PHP script.)The BasicsFirst, I define some basic parameter: TTF font file, the font size, and an example text.$font_file = "./FFF Tusj.ttf";$font_size = 64;$text = "An Example";In the next step I've to find out, how big my image needs to be to take the rendered text. That's not so trivial because it strongly depends on the used characters, the choosen size and of course the font itself. To solve this problem PHP offers the imagettfbbox() function:$bb = imagettfbbox($font_size, 0, $font_file, $text);$bb_width = $bb[4]-$bb[6];$bb_height = $bb[3]-$bb[5];With imagecreate() I now can create the image using $bb_width and $bb_width for the size:$image = imagecreate($bb_width, $bb_height); Define two colors: one for the background and one for the foreground.$fillcolor = imagecolorallocate($image, 255, 255, 255); $fontcolor = imagecolorallocate($image, 69, 138, 186); Fill the background using $fillcolor:imagefill($image, 0, 0, $fillcolor); Render the text to the image:imagettftext($image, $font_size, 0, abs($bb[6]),$bb_height-$bb[3], $fontcolor,$font_file, $text);I don't want to go into the details of this function, for a detailed explanation of all parameters please refer to the PHP manual.And finally send it to the browser:header("Content-Type: image/png"); imagepng($image);Basically that's all you need to do. Here's the output of the above PHP script:Because I used imagecreate() and imagepng() the file I got is an indexed-colored 8-bit PNG with a file size of 8 KB. Some VariationsIf I use imagecreatetruecolor() and imagepng() I will get a truecolored 24-bit PNG with a file size of 41 KB: And if I use imagejpeg() instead of imagepng() I will get a truecolored 24-bit JPEG with a quality setting of 100 and a file size of 41 KB:All three variations look exactly the same, but the indexd-colored 8-bit PNG is the smallest one. So for this purpose an 8-bit PNG seems to be the best choice.Turning off antialiasing?By default imagettftext() uses an antialiasing algorithm to smooth the output. Using the negative of a color index turns this feature off:imagettftext ($image, $font_size, 0, abs($bb[6]),$bb_height-$bb[3], -$font_color, $font_file, $text);Sometimes (usually in case of small font sizes) this will give you a sharper and better looking result, but in this special case it definitely looks worse:Welcome to the Real WorldAs I mentioned in the beginning, I intentionally chose a font based on a very large TTF file, which makes it very expensive for PHP to render a headline. Let's take a look at some quick benchmark results:% ab -n 1000 http://demo/headline.phpThis is ApacheBench, Version 2.3...Benchmarking demo (be patient)...Document Path: /headline.phpDocument Length: 7888 bytes...Requests per second: 33.52 [#/sec] (mean)...Autsch... 33 requests per second is far to slow for a real world scenario. Yes, if I had chosen a smaller font the results would be much better, but probably the script will still be not suitable for use in a production environment. However, a simple caching mechanism should easily solve this issue.But not today, stay tuned for part 2 of this tutorial. Live long and prosper.

My recent blog post about scaling images with PHP gave me the idea to write something about creating typographic headlines with PHP. At Apache Friendswe're using this technique since many years to get...

PHP

Scaling images in PHP (done right)

Scaling images in PHP is quite easy, but there are some things to consider. (If you're short of time, right at the end you'll find the final script.)Read the original image with imagecreatefromjpeg()First of all you'll need to read the original image. If it's a JPEG file the imagecreatefromjpeg() function is the right choice:$source_image = imagecreatefromjpeg("osaka.jpg");If it's a GIF file you'll take imagecreatefromgif(), and if it's a PNG you will prefer imagecreatefrompng().For this small tutorial I'll use this image from the Osaka Aquarium Kaiyukan.Get the size of the original image: getimagesize() vs. imagesx() and imagesy()Reading the image is quite easy, but the first pitfall you'll encounter if you prepare to scale an image, and need to find out the dimensions of the original image.Most tutorials will propose this way:$source_image = imagecreatefromjpeg("osaka.jpg");$source_image_size = getimagesize("osaka.jpg");The problem with the getimagesize() function is that it needs to reopen the file to get the actual image size. This is usually not a big issue if you're reading a local file, but it will get critical if you're reading a file from the network like in:$source_image = imagecreatefromjpeg("http://someurl/osaka.jpg");$source_image_size = getimagesize("http://someurl/osaka.jpg");Every time you call getimagesize() the whole image file get transferred over the network, and that quickly became mission-critical. Since you already have the image loaded with imagecreatefromjpeg() there is no need to load it again:$source_image = imagecreatefromjpeg("osaka.jpg");$source_imagex = imagesx($source_image);$source_imagey = imagesy($source_image);Create the destination image: imagecreate() vs. imagecreatetruecolor()Now we need to prepare the (scaled) destination image. There are two PHP functions which can be used to create an image: imagecreate() and imagecreatetruecolor(). The first creates a palette based image (with a maximum of 256 different colors), and the second one creates a true color image (with a maximum - as far as I know - of 256\*256\*256 = 16 million colors).Let's compare the results of both function: On the left imagecreate() and on the right imagecreatetruecolor():It's obvious: As long as you work with photographic images you'll need more than 256 colors.So let's decide to use imagecreatetruecolor() and define a target size of 300x200 pixels:$dest_imagex = 300;$dest_imagey = 200;$dest_image = imagecreatetruecolor($dest_imagex, $dest_imagey);Scale the image: imagecopyresized() vs. imagecopyresampled()Now it's time to do the actual scaling of the image. Again PHP offers to function for this purpose: imagecopyresized() and imagecopyresampled(). The first one uses a very simple algorithm to scale the image, it's fast but the quality is really poor. The second one uses a better, but slower algorithm, resulting in a very high quality image.Poor quality, but fast:imagecopyresized($dest_image, $source_image, 0, 0, 0, 0, $dest_imagex, $dest_imagey, $source_imagex, $source_imagey);Best quality, but slow:imagecopyresampled($dest_image, $source_image, 0, 0, 0, 0, $dest_imagex, $dest_imagey, $source_imagex, $source_imagey);I don't want to go into the details of this functions, for a detailed explanation of all parameters please refer to the PHP manual.Let's compare the results: On the left imagecopyresized() and on the right imagecopyresampled():Again it's obvious: The quality of imagecopyresampled() is much better. In my opinion there is never a reason to use the faster imagecopyresized(). Why would I ever want a low quality image? Even if it's faster to get?And finally push the image to the browser: imagejpeg() vs. imagepng()After scaling the image, we now need to push the image to the user's browser. Probably the most popular image formats in the Internet are currently PNG and JPEG. Both will work great with photographic images but true-colored and loss-less PNG images usually results in larger file sizes than JPEG images.To send a PNG image (with best compression rate 9) to the browser:header("Content-Type: image/png");imagepng($dest_image,NULL,9);Or a JPEG image (with best quality 100):header("Content-Type: image/jpeg");imagejpeg($dest_image,NULL,100);And in comparison, imagepng() on the left vs. imagejpeg() on the right:Both look absolutely the same, but the JPEG image has a size of 57 KB (using the best quality of 100) and the PNG image is 102 KB big (using the highest available compression rate).What's the best JPEG quality to choose?JPEG images are not only smaller but also give you the flexibility to choose the quality and by this indirectly the file size. In PHP you can choose the quality in a range from 0 (worst quality, smaller file) to 100 (best quality, biggest file). Let' take a look. Quality 100 (57 KB) and quality 80 (16 KB):If you look very carefully at the quality 80 version on the right, you'll see very small artifacts by the JPEG compression.Quality 60 (12 KB) and quality 40 (8 KB):The loss of quality gets worse, and in my opinion the quality 40 image on the right looks terrible.And now the whole script...Many words end in a small script:<?php $source_image = imagecreatefromjpeg("osaka.jpg");$source_imagex = imagesx($source_image);$source_imagey = imagesy($source_image);$dest_imagex = 300;$dest_imagey = 200;$dest_image = imagecreatetruecolor($dest_imagex, $dest_imagey);imagecopyresampled($dest_image, $source_image, 0, 0, 0, 0, $dest_imagex, $dest_imagey, $source_imagex, $source_imagey);header("Content-Type: image/jpeg");imagejpeg($dest_image,NULL,80);?>In this script I used a quality of 80, that's just my personal preference. You may choose whatever you like. But please, not less than 40.PostscriptIn many tutorials the PHP script ends with several imagedestroy() function calls. imagedestroy() frees any memory associated with an image. This is a good idea if you sequentially work with different image resources within a single PHP script. But if the imagedestroy() is right at the end of a script, you may omit this function. When the script ends PHP will automatically free any resources.

Scaling images in PHP is quite easy, but there are some things to consider. (If you're short of time, right at the end you'll find the final script.) Read the original image with imagecreatefromjpeg() Fi...

Sun

Easy deploy your web apps with Sun GlassFish Web Stack

Today I want to show you a quick installation walkthrough of Sun GlassFish Web Stack. I'm using Solaris 10 in this walkthrough, but installation on RHEL is absolutely the same. As a small deployment example for a web application I'll do an installation of WordPress.Web Stack InstallationOkay, first step: Get the Sun GlassFish Web Stack.Simply enter http://sun.com/webstack in your browser.After clicking on the Get It button, you're asked to pick your platform: Red Hat Enterprise Linux or Solaris 10. (You may wonder why there is no OpenSolias download, that's quite simple: because the Web Stack is already sipped with OpenSolaris (2009.06), all you need to do to install the main components is to do a pkg install amp. And you'll get the Apache, MySQL and PHP components of Web Stack installed right on your system.)After picking the platform, you need to decide which kind of distribution you want to download: native packages or the IPS-based distribution:The native packaging distribution (aka RPM/SVR4) differs two version: one including Java-based components like Tomcat, GlassFish and Hudson, and one without.For this demo I picked my personal favorite, the IPS-based distribution, because it allows me a non-root install and gives me the ability to place my Web Stack anywhere I want in the directory tree of my system. So I don't need to have root access and can install Web Stack as a regular user.After the download is finished I open a terminal and take a look at my system environment:[oswald@localhost ~]$ df -hFilesystem Size Used Avail Use% Mounted on/dev/mapper/VolGroup00-LogVol00 7.2G 2.3G 4.5G 34% //dev/hda1 99M 12M 83M 13% /boottmpfs 125M 0 125M 0% /dev/shm[oswald@localhost ~]$ pwd/home/oswald[oswald@localhost ~]$ iduid=500(oswald) gid=500(oswald) groups=500(oswald)[oswald@localhost ~]$ ls -ltotal 14452-rw-r--r-- 1 oswald 14765264 Oct 12 14:43 webstack-image-1.5-b09-redhat-i586.zipMy current working directory is my home directory. There is enough space left on my file system. I'm a non-root, regular user, and I see the downloaded IPS installer image.Now I'm unzipping the image to my home directory:[oswald@localhost ~]$ unzip -q webstack-image-1.5-b09-redhat-i586.zip This creates a directory named webstack1.5 by default. You can rename it to anything you want or simply keep the default name. In this case I choose to name it demo: [oswald@localhost ~]$ mv webstack1.5 demoNow I change into that directory and start the Update Tool:[oswald@localhost ~]$ cd demo[oswald@localhost demo]$ bin/updatetool So, what we're seeing here is the update tool. Our current Sun GlassFish Web Stack installation is highlighted on the left hand side. If you have other products installed or multiple Web Stack installs, all these images will show up too. In this case I've only one image and once I click on Available Add-ons.Now I pick all the components I want to use in this demo, so I select Apache HTTP server, MySQL server, PHP Server and the PHP MySQL connector. By the way, if I realize I need more packages later on, I can come back here anytime and install whatever I need. For now, I've selected all I want at this point, and all I have to do now is to press the beautiful green install button above the package list.After downloading, I get back to the shell and use the command line tool pkg to review the list of installed packages:oswald@localhost demo]$ bin/pkg listNAME (PUBLISHER) VERSION STATE UFIXpkg 1.111.3-30.2210 installed ----pkg-toolkit-incorporation 2.2.1-30.2210 installed ----python2.4-minimal 2.4.4.0-30.2210 installed ----sun-apache22 2.2.11-1.5 installed ----sun-mysql51 5.1.30-1.5 installed ----sun-mysql51lib 5.1.30-1.5 installed ----sun-php52 5.2.9-1.5 installed ----sun-php52-mysql 5.2.9-1.5 installed ----sun-wsbase 1.5-1.5 installed ----updatetool 2.2.1-30.2210 installed ----wxpython2.8-minimal 2.8.8-30.2210 installed ----You can also use pkg to install components, like in the Update Tool. So for the terminal addicted: if you don't like a GUI, there is also a command-line alternative.Okay, let's go on. I now start the Apache web server:[oswald@localhost demo]$ bin/sun-apache22 startStarting apache22And to check if the Apache is really running and working I'll take my browser to http://localhost. But wait a second...As we probably all know, by default, the Apache web server runs on port 80, which is the default port on which most Web servers run. But in the Unix world all port numbers below 1024 require root permissions to bind. And since I started Apache as a normal non-root user, the Apache will be unable to bind to port 80.If you use Web Stack as non-root user, Web Stack will add 10000 to every port number below 1024. So in this case Apache will use 10080 as it's favorite port.At http://localhost:10080/ we find the welcome page of Sun GlassFish Web Stack:But what's life with just an Apache, let's also start the MySQL and let the database server join our team:[oswald@localhost demo]$ bin/sun-mysql51 startStarting mysql51Because everyone on my system and on my network can now access my MySQL server, I strongly need to secure my installation by setting a password for MySQL's root user:[oswald@localhost demo]$ mysql/5.1/bin/mysqladmin -u root password "demo"Okay, demo is probably not a secure password for a production environment, but for this demo purposes it's a very appropriate one.That's all. Your AMP stack is now ready.Example DeploymentAs an example I will now install the famous blogging software WordPress within my new AMP stack. Not because it's so difficult to do, but it's a good and pragmatic way to show the AMP stack is working and out-of-the-box capable to run popular web applications.First I need to create a database and a user for WordPress to access the database and to store its data.Okay, that should be an everyday task for a MySQL DBA: [oswald@localhost demo]$ bin/mysql -uroot -pdemoWelcome to the MySQL monitor. Commands end with ; or \\g.Your MySQL connection id is 2Server version: 5.1.30-log Source distributionType 'help;' or '\\h' for help. Type '\\c' to clear the buffer.mysql> CREATE DATABASE wordpress;Query OK, 1 row affected (0.01 sec)mysql> GRANT ALL PRIVILEGES ON wordpress.\* TO 'wordpress'@'localhost' \\ IDENTIFIED BY 'demo';Query OK, 0 rows affected (0.98 sec)mysql> QUITGet into the MySQL monitor, aka the MySQL command line tool, to connect to the database server. I use the CREATE DATABASE statement to create a database named wordpress.And with the GRANT statement I create a user named wordpress identified by the password demo.Again this is probably not a good password for production use, but for a demo this is perfecto.After changing into Apache's document root directory I wget the latest wordpress version:[oswald@localhost demo]$ cd var/apache2/2.2/htdocs[oswald@localhost htdocs] wget -q http://wordpress.org/latest.tar.gzWith GNU tar I extract the archive:[oswald@localhost htdocs]$ /usr/sfw/bin/tar xfz wordpress-2.8.4.tar.gzThis creates a directory called wordpress.Now I can access my WordPress installation by pointing my browser to http://localhost:10080/wordpress.From now on it's very easy. Simply enter the details for the database server connection:The database name: wordpress.The user name: wordpress.The password: demo.The database host: localhost.And the table prefix wp_.And on the next page:I choose a name for my blog, and enter my email address. Press Install WordPress. And now it only takes a few moments and the installation is done. (Write down your random admin password, you will probably need it later.)Now I want to check if the Wordpress installation was successful and again point my browser to http://localhost:10080/wordpress.And, voilà, there it is: my shiny new "Sun GlassFish Web Stack" blog. Hooray.That's all. Welcome to the wonderful world of AMP.

Today I want to show you a quick installation walkthrough of Sun GlassFish Web Stack. I'm using Solaris 10 in this walkthrough, but installation on RHEL is absolutely the same. As a small deployment...

Miscellaneous

What is Java?

Just learned that the question "what is java" is on place 3 of Google's Zeitgeist ranking 2008 for "what is" questions - right after "what is love" (first place) and "what is life" (second place).Indeed a good question. Let's take a look a Google's top hits to this question:About.com (Hit #9)Java is a computer programming language. It enables programmers to write computer instructions using English based commands, instead of having to write in numeric codes. It’s known as a “high-level” language because it can be read and written easily by humans. Like English, Java has a set of rules that determine how the instructions are written. These rules are known as its “syntax”. Once a program has been written, the high-level instructions are translated into numeric codes that computers can understand and execute. (Source: http://java.about.com/od/gettingstarted/a/whatisjava.htm)Webopedia.com (Hit #7)A high-level programming language developed by Sun Microsystems. Java was originally called OAK, and was designed for handheld devices and set-top boxes. Oak was unsuccessful so in 1995 Sun changed the name to Java and modified the language to take advantage of the burgeoning World Wide Web.(Source: http://www.webopedia.com/TERM/J/Java.html)Wikipedia.org (Hit #4)Java is a programming language originally developed by James Gosling at Sun Microsystems and released in 1995 as a core component of Sun Microsystems' Java platform. The language derives much of its syntax from C and C++ but has a simpler object model and fewer low-level facilities. Java applications are typically compiled to bytecode (class file) that can run on any Java Virtual Machine (JVM) regardless of computer architecture. (Source: http://en.wikipedia.org/wiki/Java_(programming_language))Boutell.com (Hit #3)Java is a technology that allows software designed and written just once for an idealized "virtual machine" to run on a variety of real computers, including Windows PCs, Macintoshes, and Unix computers. On the web, Java is quite popular on web servers, used "under the hood" by many of the largest interactive websites. Here it serves the same role that PHP, ASP or Perl might, although traditionally Java has been used for larger-scale projects.(Source: http://www.boutell.com/newfaq/definitions/java.html)Java.com (Hit #1)What is Java? Java allows you to play online games, chat with people around the world, calculate your mortgage interest, and view images in 3D, just to name a few. It's also integral to the intranet applications and other e-business solutions that are the foundation of corporate computing.(Source: http://www.java.com/en/download/whatis_java.jsp)As you see, it's not an easy question to answer!

Just learned that the question "what is java" is on place 3 of Google's Zeitgeist ranking 2008 for "what is" questions - right after "what is love" (first place) and "what is life" (second place). Indee...

Apache

Apache's graceful restart (reprise)

In my last week's blog entryUrban legends: Apache reload(ed) I tried to prove that an Apache reload is quite exactly the same as the restart of an Apache web server.One of my dear readers - yes, at least someone seems to read this blog - pointed out that this is not always true, and a reload sometimes work and sometimes not. In contrast to a restart, which always work like a charm.I strongly assume that's a classic observer effect:In physics, the term observer effect refers to changes that the act of observation will make on the phenomenon being observed. Let's imagine: You have a web site, you have an Apache web server. With your browser you're on your web site, you change your Apache's configuration, you reload your Apache, you reload your browser, and - surprise - you don't see the new configuration active. You reload your browser again and again. Still, the old configuration. What's wrong?Okay, let's do this step by step.My Apache is running:# apache2ctl status Apache Server Status for localhost Current Time: Friday, 02-Oct-2009 11:46:29 CEST Restart Time: Friday, 18-Sep-2009 09:55:56 CEST Parent Server Generation: 12 Server uptime: 14 days 1 hour 50 minutes 32 seconds 87 requests currently being processed, 89 idle workers..KK_._K_K.KK_._KK.KKK.K_K.K_K_._KKKK._KKK__._KK_C_KK_.KK.C_K__KK_K.K_K_KKK_KK_C_KKK__K_KKKK.KWK__KKKK.K_.__..K.._..___K___W__KKW_K_K_K_._K_K____K_K__K__K..K_KK__K______K_KK_K_K____K___K___.__.____K_K__._.K..................................................Three childs are closing the connection (C), 3 are sending a reply to any browser (W), 89 are waiting for a new connection (_) and 81 childs are kept alive by KeepAlive (K). The one red K represents my own bowser's connection.Now I'm reloading my Apache:# /etc/init.d/apache2 reloadWait a second, and ask again for the status:# apache2ctl status Apache Server Status for localhost Current Time: Friday, 02-Oct-2009 11:47:04 CEST Restart Time: Friday, 18-Sep-2009 09:55:56 CEST Parent Server Generation: 13 Server uptime: 14 days 1 hour 51 minutes 8 seconds 71 requests currently being processed, 80 idle workersG.GG_.__GG.GG_._GG.___.___.____._GG__.__GGG_.K_GGK__K_.G_.___K_G___.G___GGG_____KG__GG__.GG_.__GG__KG_.__._W.._.._..___GG__K_GG__K__G_G_.GK_K____GKG__G___..K..G.KK..G...K.KGGWGC....KG..G.......GG...._G.._.G..................................................Now, after one or two seconds, 51 Apache childs are waiting for their graceful end. Including the G representing my own bowser's connection.In parallel I'm reloading my browser (which accesses the web site my Apache's hosting) in a 3 seconds interval.After 2 minutes I look again at my Apache's status:# apache2ctl status Apache Server Status for localhost Current Time: Friday, 02-Oct-2009 11:48:40 CEST Restart Time: Friday, 18-Sep-2009 09:55:56 CEST Parent Server Generation: 13 Server uptime: 14 days 1 hour 52 minutes 44 seconds 77 requests currently being processed, 51 idle workersKKK__KKKWKK__K_K..W_K_KKK___K_K____K_K_K__.K_K.._K_K.K..KWK._KKK.K_._KKK_KK___..KG_K_KK___.K.KK__KK_K.KK_K_W..K.....__K..W__K..W_KK_._....K.KKK__.K.KK.K_K..K...._W......K.K.._.K....K.................K...K....................................................There is still one child process waiting for it's graceful end. That's the one I kept alive with of my own browser. And this child still has it's old configuration active and that's why I'll never notice the new config within my own bowser, but everyone else already got the new configuration.To catch up, I just have to wait at least KeepAlive seconds, and than doing a final reload in my browser.# apache2ctl status Apache Server Status for localhost Current Time: Friday, 02-Oct-2009 11:49:00 CEST Restart Time: Friday, 18-Sep-2009 09:55:56 CEST Parent Server Generation: 13 Server uptime: 14 days 1 hour 53 minutes 4 seconds 63 requests currently being processed, 76 idle workers.K__KKKK_K_K______..KKK______W_K_C_K___KK.___KKK.___K_K__K__KKK.W_KC_KK_CK._K_KK___K_KK____._KK___K_____C__K__CK.___KK_K___K_KK_C_K..K__K.C..K..__......K......_..K....._....K......C.K.K._.....................................................................No more gracefully dying childs anywhere. And finally I noticed the new configuration in my own browser.That's the reason why people think an Apache reload sometimes work and sometimes not.

In my last week's blog entryUrban legends: Apache reload(ed) I tried to prove that an Apache reload is quite exactly the same as the restart of an Apache web server. One of my dear readers - yes, at...

Miscellaneous

Qs about you and Linux

My last blog entry about Linus Torvals' thoughts on the goto statement brought and old email interview back to my mind, which I had the honor to have with him a long time ago in 1994.From cs.Helsinki.FI!Linus.Torvalds Wed Jun 22 20:45:59 1994From: torvalds@cs.Helsinki.FI (Linus Torvalds)Date: Wed, 22 Jun 1994 21:45:52 +0300In-Reply-To: Kai Seidler's message as of Jun 22, 12:13X-Mailer: Mail User's Shell (7.2.4 2/2/92)To: oswald@duplox.wz-berlin.de (Kai Seidler)Subject: Re: Qs about you and linuxStatus: ROKai Seidler: "Qs about you and linux" (Jun 22, 12:13):> > Did your role as linux programmer has changed over the time? From the> alone linux programmer (1991) to a linux god? How much time do you> spend today in programming in opposition to manage kernel-patches,> answer stupid questions (about bugs, and like this one :), visit> congresses?Oh, it has changed, all right. In 1991, I essentially coded 8 hours aday and didn't mind about other people or "secondary" stuff likeportability etc. As it stands now, I get to code occasionally when Ifind some time and have something interesting to do, but most of mylinux time is simply "management" these days. I'm not wearing anysuits, though :-)Just reading mail takes about 2 hours a day - I also read the newsgroupswhen I can, but that usually means just col.announce and selectedarticles from col.development. Applying patches isn't that bad: I havepeople I trust that do the large patches and then I just need to checkthem over against obvious problems. The "un-trusted" patches are muchrarer. Actually, the above sounds worse than it is. The fact is that the basickernel mostly works well enough and one reason for me not coding quiteas much as I use to do is simply that the basic functionality that I'vepersonally always concentrated on doesn't need that much care any more. The patches these days are mostly networking and device drivers with theoccasional smaller stuff elsewhere. Conferences haven't been a problem until recently, and on the whole theyhaven't really proved too distractive. I don't really like givingtalks, but I like meeting people and traveling and I also feel it'ssimething that needs to be done at this point. > Why do you, and all the other people, such an enormous work for free?> What do you mean? Is it fame? What did you get back from the Linux> community?Well, the fame certainly doesn't hurt, of course: I expect to be able toget a good job once I get my studies completed and decide to leave theuniversity. But mostly it's just a project I like doing, and one whichpeople appreciate. A hobby of the best kind, in short.. I think that'strue for most of the kernel developers.> How is your relation to the FreeBSD community? As far as I see, the > Linux and FreeBSD don't like each other very much, but I may be wrong.> Maybe it's "only" the old "war" between SysV and BSD?Actually, we are on a friendly standing with the FreeBSD people (Ihaven't been much in contact with NetBSD). The communities easily getinte flames over which is better, but I know both the linux and BSDkernel developers are much too involved with their (our) own projects toreally mind any of the flames. I've met with some of the FreeBSD people a few times (on conferences),and they are nice (jkh has something like 11 cats: I just have two).It's hard to co-operate too much, though: it takes a lot of time and itseems to actually be easier just to concentrate on your own project.LinusThat was 15 years ago. Nobody would have thought at that time that Linux would later become such a big competitor for commercial Unix-esque operating systems.

My last blog entry about Linus Torvals' thoughts on the goto statement brought and old email interview back to my mind, which I had the honor to have with him a long time ago in 1994. From...

Miscellaneous

Is goto the root of all evil?

As PHP 5.3 introduced the goto statement the old discussion about the evilness of goto came back to the surface of the Internet. By coincidence I stumbled over this 6 years old discussion with Linus Torvalds about this topic. He's arguing goto makes the source code more readable, but read his thoughts and decide for yourself:From: Linus TorvaldsSubject: Re: any chance of 2.6.0-test\*?Date: Sun, 12 Jan 2003 12:22:26 -0800 (PST)On Sun, 12 Jan 2003, Rob Wilkens wrote:> > However, I have always been taught, and have always believed that> "goto"s are inherently evil. They are the creators of spaghetti codeNo, you've been brainwashed by CS people who thought that Niklaus Wirthactually knew what he was talking about. He didn't. He doesn't have afrigging clue.> (you start reading through the code to understand it (months or years> after its written), and suddenly you jump to somewhere totally> unrelated, and then jump somewhere else backwards, and it all gets ugly> quickly). This makes later debugging of code total hell. Any if-statement is a goto. As are all structured loops.And sometimes structure is good. When it's good, you should use it.And sometimes structure is _bad_, and gets into the way, and using a "goto" is just much clearer.For example, it is quite common to have conditionals THAT DO NOT NEST.In which case you have two possibilities - use goto, and be happy, since it doesn't enforce nestingThis makes the code _more_ readable, since the code just does what the algorithm says it should do. - duplicate the code, and rewrite it in a nesting form so that you can use the structured jumps.This often makes the code much LESS readable, harder to maintain, and bigger.The Pascal language is a prime example of the latter problem. Because it doesn't have a "break" statement, loops in (traditional) Pascal end up often looking like total shit, because you have to add totally arbitrary logic to say "I'm done now".LinusRead the full discussion at: http://kerneltrap.org/node/553/2131

As PHP 5.3 introduced the goto statement the old discussion about the evilness of goto came back to the surface of the Internet. By coincidence I stumbled over this 6 years old discussion with Linus Tor...

PHP

How PHP handles $variable data?

If you're using PHP you're usually don't care how PHP stores variables internally. But if you start working with references you probably better know what's going on behind the scenes.(Without) ReferencesLet's assume the following code:$a="Zaphod";$b=$a;$c=$a;You probably would assume that PHP now keeps the string Zaphod three times in memory. Actually all $a, $b and $c internally(!) reference to the same string Zaphod in memory. See diagram #1.You, the user, will never know you are actually working with references, because PHP hides this very well. For example: If you change the value of any of this variable the reference just points to the new string:$c="Beeblebrox";Now $c internally references to the new value Beeblebrox. See diagram #2.And after deleting the variable $b:unset($b);the variable $a still references to Zaphod. See diagram #3.With ReferencesIf you start using references it's - of course - slightly different. Let's start over:$a="Zaphod";$b=&$a;$c=&$a;Now $b and $c are real references to the value of $a. If you output the values you'll see no difference. See diagram #4.But if you change the value of $b...$b="Beeblebrox";...you'll also change the value of $a and $c because both are pointing the the same value (btw, a value is stored in a so called zval structure). See diagram #5.Deleting a variable is likely the same as without references:unset($b);And the variable $b disappears but the value (as long as referenced by any other variable) stays in memory. Diagram #6.ConclusionEven if you don't know, you're probably using references all the time. You don't need to use references to save memory, because PHP already uses references internally, and hides this very well.

If you're using PHP you're usually don't care how PHP stores variables internally. But if you start working with references you probably better know what's going on behind the scenes. (Without)...

Apache

Urban legends: Apache reload(ed)

What's the difference between reloading and restarting an Apache web server? If you google for this you'll find a lot of (wrong) information which may sum up like this:A reload just let Apache re-read it's configuration file, without restarting the Apache. But if you need to do bigger changes to the config, like adding or removing modules or virtual hosts, you'll may need to do a real restart. Something like this.If Penn & Teller would care about Apache configuration, they would agree: This is bullshit!Apache never supported something like a reload mechanism. And therefore there is no such functionality. If you accept this fact, you're a step closer to the truth.OriginsOne origin of this legend is probably to be found in the fact that classic Unix daemons have a "reload" mechanism which is triggered by sending an HUP (hang up) signal to the process. A process getting such a HUP signal didn't hang up but reloaded its own configuration file. Without the need of restarting. Later this functionality was made available by the System V init scripts, which are still the most common and popular way of controlling Unix services. That's what we use if we call some script within /etc/init.d, /etc/rc.d, etc.Most of this scripts are enabling the user (aka root) to start, stop, reload a specific service (aka daemon). And for example if you look into /etc/init.d/crond on a RHEL 5.2 you can track down the reload to a single HUP signal:echo -n $"Reloading cron daemon configuration: "killproc crond -HUPAnd as for every other system daemon also Apache's init scripts offer the user to reload the Apache web server. An example from Debian 5.0:# /etc/init.d/apache2 Usage: /etc/init.d/apache2 {start|stop|restart|reload|...}.BTW: On OpenSolaris it's called refresh instead of reload. But that's just another wording.The TruthIf you track down this reload functionality you'll find something like this:On Debian's /etc/init.d/apache2:log_daemon_msg "Reloading web server config" "apache2"$APACHE2CTL graceful $2Or on OpenSolaris /lib/svc/method/http-apache22:cmd="graceful"${APACHE_BIN}/apachectl ${STARTUP_OPTIONS} ${cmd}So if you track down the "reload" you end up with a "graceful". And now we're at the beginning of this blog entry: Apache never supported something like a reload mechanism. And therefore there is no such functionality.And "graceful" means according to the Apache HTTP Server 2.2 Documentation:Graceful Restart: The USR1 or graceful signal causes the parent process to advise the children to exit after their current request (or to exit immediately if they're not serving anything). The parent re-reads its configuration files and re-opens its log files. As each child dies off the parent replaces it with a child from the new generation of the configuration, which begins serving new requests immediately. So the "reload" ends up in a restart of all the Apache children and from a internal configuration-releated view, a graceful restart is exactly the same as a regular restart. It's just better for the stability of your web site, because the children are ended after finishing their current HTTP request and not terminated while serving a client.The ConclusionIf you change you Apache's configuration do a "reload" or whatever your system calls it. There is no need for a regular restart. But you may need to wait some seconds until all the tiny Apache children processes catched up with the new configuration.PostscriptOn my RHEL 5.2 an Apache "reload" actually ends in an HUP signal:echo -n $"Reloading $prog: "killproc $httpd -HUPWhich would be right for a usual Unix daemon like crond but in case of Apache this means a restart and not a graceful restart. A graceful restart is triggered by an USR1 signal. Looks like a copy and paste error in RHEL 5.2. Probably this is fixed in newer releases.

What's the difference between reloading and restarting an Apache web server? If you google for this you'll find a lot of (wrong) information which may sum up like this: A reload just let Apache re-read...

Miscellaneous

How to upgrade VirtualBox Guest Additions on Solaris/OpenSolaris?

Today I needed to play around work out something on RHEL and OpenSolaris. I have both systems running in a VirtualBox on my Mac and because of the latest update to VirtualBox I was supposed to update also the so called Guest Additions on RHEL and OpenSolaris. The update went smoothly on RHEL, but on OpenSolaris I got this frightened message:Current administration requires that a unique instance of the<SUNWvboxguest> package be created. However, the maximum number ofinstances of the package which may be supported at one time on thesame system has already been met.No changes were made to the system. As you already may have noticed I'm not a native English speaker, but is requires a unique instance and maximum number of instances has already been met a proper way to express: The package is already installed, and that's why the package can't get installed another time?Indeed, as the VirtualBox User Manual (3.0.6) states:The Guest Additions should be updated by first uninstalling the existing Guest Additions and then installing the new ones. Attempting to install new Guest Additions without removing the existing ones is not possible.But how? That's not mentioned in the users manual. Why? I'm new to VirtualBox and it's the first time I (want to) upgrade the guest additions.»Look Dave, I can see you're really upset about this. I honestly think you ought to sit down calmly, take a stress pill, and think things over.«Okay, you're right, calm down. To cut a long story short:To uninstall the current package:# pkgrm SUNWvboxguestThe following package is currently installed: SUNWvboxguest Sun VirtualBox Guest Additions (i386) 3.0.4,REV=r50677.2009.08.04.19.24Do you want to remove this package? [y,n,?,q] y## Removing installed package instance <SUNWvboxguest>This package contains scripts which will be executed with super-userpermission during the process of removing this package.Do you want to continue with the removal of this package [y,n,?,q] y## Verifying package <SUNWvboxguest> dependencies in global zone## Processing package information.## Executing preremove script.......Removal of <SUNWvboxguest> was successful.And install the new package:/media/VBOXADDITIONS_3.0.6_52128# pkgadd -d VBoxSolarisAdditions.pkg The following packages are available: 1 SUNWvboxguest Sun VirtualBox Guest Additions (i386) 3.0.6,REV=r52128.2009.09.09.19.32Select package(s) you wish to process (or 'all' to processall packages). (default: all) [?,??,q]: aProcessing package instance <SUNWvboxguest> from </media/VBOXADDITIONS_3.0.6_52128/VBoxSolarisAdditions.pkg>Sun VirtualBox Guest Additions(i386) 3.0.6,REV=r52128.2009.09.09.19.32VirtualBox Personal Use and Evaluation License (PUEL)......This package contains scripts which will be executed with super-userpermission during the process of installing this package.Do you want to continue with the installation of <SUNWvboxguest> [y,n,?] yInstalling Sun VirtualBox Guest Additions as <SUNWvboxguest>......Please re-login to activate the X11 guest additions.If you have just un-installed the previous guest additions a REBOOT is required.Installation of <SUNWvboxguest> was successful.And because I just un-installed the previous guest additions I now initiate reboot:# /sbin/init 6εὕρηκα! ;)

Today I needed to play around work out something on RHEL and OpenSolaris. I have both systems running in a VirtualBox on my Mac and because of the latest update to VirtualBox I was supposed to update...

PHP

AWKish PHP

I love AWK. It is a wonderful tool for data processing on Unix systems. I truly love it. There is certainly no better tool to process and aggregate log files. I remember back when I introduced AWK to my students, there was always an immediately appreciative murmur in the round when the first AWK scripts showed their power.And it's so stable. Perhaps even more stable than all the new-fashioned stuff nowadays. I have one AWK monster script running since 1992 unchanged in a productive environment, without ever encountering any problems. No need to change anything after countless system upgrades. A dream within a dream. In the same time PHP would have released in ten new major releases and certainly wouldn't be no longer compatible with itself.But...In time of HTML entities, URL encoding and XML you quickly run unto the limits of AWK and it's becoming a very hard job to use AWK in this new context. Ever tried to URL decode or encode some string within AWK? Or process an XML file?However...Since PHP 5 the command line interface of PHP offers the wonderful options -R with its siblings -B and -E. With this three fellows, you can run PHP in a quasi AWK mode: -R for a line by line processing of the input stream, the option -B as an equivalent for AWK's BEGIN and -E for AWK's END. A small exampleAn example as old as the world: counting lines with AWK.# time awk 'BEGIN {s=0} {s++} END {print s}' access_log3279700.148u 0.040s 0:00.17The same in awkish PHP:# time php -B '$s=0;' -R '$s++;' -E 'print "$s\\n";' < access_log3279701.064u 0.044s 0:01.20Yes, of course one would use wc in the real world:# time wc -l access_log327970 access_log0.036u 0.048s 0:00.07The time command in this examples shows the big drawback of PHP: In comparison to AWK it's really slow. But this is understandable, because PHP is far more complex than AWK and PHP was made for web programming and not as a sysadmin's power tool. Always use a tool for the job it was designed to do. On the other hand PHP has all the functions you absolutely need in today's work environments. (Please don't mention XMLgawk or WebAWK now.)But to be honest: PHP is only a small blink in the history of the Internet. AWK is the A and O, the beginning and the end, the first and the last. Aho, Weinberger and Kernighan. You're my trinity.

I love AWK. It is a wonderful tool for data processing on Unix systems. I truly love it. There is certainly no better tool to process and aggregate log files. I remember back when I introduced AWK to...

MySQL

What's going on inside your MySQL server?

Your MySQL server is under heavy load or refuses any new connections because MySQL runs out of available threads. Ever wondered why?For me the easiest way to get a first and very helpful real-time insight in what is happening in your database server is to use the show processlist statement:mysql> show processlist;+--------+---------+-----------+-------------+----------------+------+--------------------+------------------------------------------------------------------+| Id | User | Host | db | Command | Time | State | Info |+--------+---------+-----------+-------------+----------------+------+--------------------+------------------------------------------------------------------+| 426144 | DELAYED | localhost | statistics | Delayed insert | 170 | Waiting for INSERT | || 431669 | root | localhost | NULL | Query | 0 | NULL | show processlist || 431677 | mantis | localhost | mantis | Query | 0 | Sending data | SELECT \* FROM mantis_custom_field_table WHERE id='3' |+--------+---------+-----------+-------------+----------------+------+--------------------+------------------------------------------------------------------+3 rows in set (0.00 sec)Only the first 100 characters of a statement are shown in the Info column. If you really need more information simply call show full processlist to get the whole complete statement information.

Your MySQL server is under heavy load or refuses any new connections because MySQL runs out of available threads. Ever wondered why? For me the easiest way to get a first and very helpful...

Apache

Save energy! Stop using CGI!

The day CGI was invented was a great day for the Internet, but a dark day for the history of how-to-do-thinks-right. CGI was great, because it gave us (standard computer nerds) the ability to easily implement dynamically generated HTML pages - the predecessor of todays web applications. The interface was so ingeniously simple and powerful that everyone could use his favorite programming language for implementing web services. You could use C, Perl, AWK, PostScript or even the Bourne Shell to write your web application. And people did.However, there is another side, a darker side, of CGI. It is not just ingeniously simple and powerful, in fact, it's also terribly slow. For every incoming HTTP request your system needs to fork a new process, and forking a process is an extremely expensive operation.It was a great blessing, but nowadays there are so many programming languages especially suited for web development - designed to run in a web server environment - armed with the features and functions best fitting in the needs of a web application. I say PHP, you say Java. Whatever.But if you stuck to a programming language of the past, please don't use CGI any more, use FastCGI or something comparable to connect your application with your web server. Or if you're lucky and your programming language is already aware of the web, there may be an even easier and better way.For example, if your're a real Perl programmer, then use mod_perl. It's so simple to run unaltered(!) Perl-CGI scripts with mod_perl:PerlModule ModPerl::PerlRun<Files ~ "\\.pl$"> SetHandler perl-script PerlResponseHandler ModPerl::PerlRun PerlOptions +ParseHeaders Options +ExecCGI</Files>Compared to a simple Hello World in Perl with CGI...% ab -n 1000 http://demo/helloworld.pl...Requests per second: 304.84 [#/sec] (mean)...... the same script with mod_perl...% ab -n 1000 http://demo/helloworld.pl...Requests per second: 956.16 [#/sec] (mean)......is three times faster. Without touching your Perl script.And if I actually write a Hello World which takes advantage of mod_perl, I get:# ab -n 1000 http://demo/helloworld...Requests per second: 1777.50 [#/sec] (mean)...Nearly six times faster, and that's only a simple example script!Stop using CGI! Green your IT! Save energy! Save the world! Save the Cheerleader!

The day CGI was invented was a great day for the Internet, but a dark day for the history of how-to-do-thinks-right. CGI was great, because it gave us (standard computer nerds) the ability to...

MySQL

MySQL and UTF-8 recipe

If you're leaving the secure world of ASCII characters you're usually enter a dangerous land of confusion. It's actually a very complex topic and from a technical point of view there is no one to blame.But if you don't care about the basics, don't care why or how, and just want a reliable and working system, simply follow these three advices to attain happiness and inner peace in your developer's life.Advice 1Define the character set for every column where you plan to store international character data:CREATE TABLE t1 ( col1 VARCHAR(42) CHARACTER SET utf8 );Or at least for every table:CREATE TABLE t1 ( col1 VARCHAR(42) ) CHARACTER SET utf8;The later example saves you some typing, but usually you don't need UTF-8 in every string column.If you omit this parameter, MySQL assumes a default value which may be different from system to system.Advice 2After connecting to MySQL always do a SET NAMES 'utf8' before doing anything else.For example in PHP:$mysqli=mysqli_connect('localhost','demo','demo','demo');$mysqli->query("SET NAMES 'utf8'");If you omit this parameter, MySQL assumes a default value which may be different from system to system.Advice 3Before sending any text to the browser set the charset in the Content-Type header of your HTTP response.For example in PHP:header("Content-Type: text/html; charset=utf-8");If you omit this parameter, your browser assumes a default value which may be different from system to system.A final word...This is a very simple recipe made for people who don't want to be bothered with technical details, but it's very important to follow all these advices, and not just one or two.

If you're leaving the secure world of ASCII characters you're usually enter a dangerous land of confusion. It's actually a very complex topic and from a technical point of view there is no one...

MySQL

SELECT LAST_INSERT_ID() FROM LEGEND

I don't know why or who was the first who introduced this legend, but if you review source code from time to time, you see one weird MYSQL SQL statement showing up regularly: SELECT LAST_INSERT_ID() FROM some table.If you google for »SELECT LAST_INSERT_ID() FROM« you'll get 98,100 hits. And if you use Google Code you actually find all the public available sources using this phrase: 216 times in some PHP code, 120 times in Perl code, 102 times in Java code, 21 times in Python code, and finally 1 time in Ruby code.But what's so bad about this statement? Let's go into the lab:mysql> SELECT \* FROM pages;+----+---------+-----------------------------------------+| id | name | content |+----+---------+-----------------------------------------+| 1 | welcome | Dear Traveler, welcome to my AMP world! | | 2 | about | This is about AMP! | | 3 | team | Apache, MySQL, and PHP. | +----+---------+-----------------------------------------+3 rows in set (0.00 sec)A simple table I used in an earlier blog entry. Now let's add some data:mysql> INSERT INTO pages (name,content) VALUES('faq','Typical mistakes');Query OK, 1 row affected (0.00 sec)And now we want to know which id our new data set got:mysql> SELECT LAST_INSERT_ID() FROM pages;+------------------+| LAST_INSERT_ID() |+------------------+| 4 | | 4 | | 4 | | 4 | +------------------+4 rows in set (0.00 sec)4 times a »4«? Sure, because LAST_INSERT_ID() got calculated for every row in that table. And since we have 4 rows in that table, we got the LAST_INSERT_ID() for 4 times. In this case it's not a big problem, but in case of a really large table this can create some serious issues. Especially if you know that this whole list of last IDs is also transferred from the server to the client.The statement is syntactically correct, but it's definitely not what you want.To get what you want, simply omit the FROM and the table name:mysql> SELECT LAST_INSERT_ID();+------------------+| LAST_INSERT_ID() |+------------------+| 4 | +------------------+1 row in set (0.00 sec)Good.

I don't know why or who was the first who introduced this legend, but if you review source code from time to time, you see one weird MYSQL SQL statement showing up regularly: SELECT LAST_INSERT_ID()...

Miscellaneous

Remember the time, security awareness was a topic?

Setting up this blog remembered me on an old security issue I stumbled upon some years ago. At that time I was simply lazy to investigate in more detail, but this time I took a closer look. Let's do the time warp...Before getting into this security thing, let's do a little time travel: Do you remember telnet and the so called rtools like rlogin/rsh/rcp/...? As ethernet hardware became cheaper and available for private use, these commands quickly disappeared from the TCP/IP dance floor and were quickly replaced by SSH counterparts. Do you also remember why did this happen? Yes, because these tools allowed an IP-based authentication and/or sent the user's password plain text over the network. They have done so since many years, but now because of the cheap network hardware it became easy for "everyone" to spy on a network and fetch all the passwords walking by.SSH stopped this. For a while...The presenceToday, with Web 2.0, security awareness seems to be a lost ideal from the past. No one cares about the privacy of private data or the security of their communication. So many of those fancy and stylish social network services use plain text password authentication over HTTP. Or if you take a look at the incredible popular and narcissistic blogosphere, where XML-RPC based APIs were used by blogging clients to talk to their online publishing system, you'll find this:An additional downside of XML-RPC based APIs is their lack of security. Although security can be introduced by other means such HTTPS, in their basic operation these transports deliver user credentials as plain text. This leaves an open door to password interception.("Beginning RSS and Atom programming" by Danny Ayers and Andrew Watt, 2005, page 525)Or before that in 2003 Mark Pilgrim stated in »The Atom API« about the MetaWeblog API:Passwords are sent in the clear, as plain text.That was 6 years ago. At least 6 years of plain text passwords. 6 years of free and unencrypted WLAN access at Starbucks or whatever other public place. You don't need to be a genius to smell a rat there.Who to blame?And you can't actually blame MetaWeblog or other XML-RPC APIs. Why should the API concern about encryption, if it's so easy to implement in a lower layer? Like in HTTPS as mentioned above. It's so easy to set up HTTPS and use it for the XML-RPC API. Sure, big and popular blog hosting services already do it this way, but myriads of self-maintained, self-hosted blogs don't. Doesn't matter if it's a private or a corporate blog.Can you blame the user? No. The user should be able to turn the computer on and off, and USE it. He's there to use the computer and not to think about it. I don't want to think about how an ATM secures my financial transactions.Can you blame the system administrators? Yes, a big yes. Whoever is responsible for the IT infrastructure should be aware of this issue and should take appropriate actions.Who else? Of course the software developers! Another big yes. Why do they make it so easy for the people to set up insecure systems? They can easily implement something to force the use HTTPS or at least, inform the users and ask them to actively do something to enable unencrypted XML-RPC over HTTP.

Setting up this blog remembered me on an old security issue I stumbled upon some years ago. At that time I was simply lazy to investigate in more detail, but this time I took a closer look. Let's do...

Apache

Easily boost your AMP website combo with mod_rewrite

The ScenarioImagine: You run a popular website using AMP technologies, but your hardware is awfully old and over the years, as your website became more and more popular, it's got slower and slower and slower. Today it runs with a load of 42 and smoke pours out of the TCP ports. And because you're a silly idealistic fool you've no money for a new hardware or to pay a reasonable hosting service. What now?The answer is in the book, the book of caches.The ExampleLet me set up a simple example:You've a PHP (or whatever) based system for your website and your requests usually look like:http://domain/index.php?page=welcomeTo get human readable and more SEOish addresses you're already using Apache's mod_rewrite in your .htaccess:RewriteEngine onRewriteRule \^([\^/]\*).html$ /index.php?page=$1 [L]Now all your website's URLs look like static HTML pages:http://domain/welcome.htmlPerfecto.Now let's focus on the backend. For this example I'll use the following very(!) simple(!) PHP code:<?php $mysqli=mysqli_connect('localhost','user','password','db'); if (!$mysqli) die("Can't connect to MySQL: ".mysqli_connect_error()); include("header.php"); include("navigation.php"); $stmt = $mysqli->prepare("SELECT content FROM pages WHERE name=?"); $stmt->bind_param('s', $_REQUEST['page']); $stmt->execute(); $stmt->bind_result($content); $stmt->fetch(); echo $content; $stmt->close(); $mysqli->close(); include("footer.php");?>The MySQL table pages looks like this:+----+---------+-----------------------------------------+| id | name | content |+----+---------+-----------------------------------------+| 1 | welcome | Dear Traveler, welcome to my AMP world! | | 2 | about | This is about AMP! | | 3 | team | Apache, MySQL, and PHP. | +----+---------+-----------------------------------------+The files header.php, navigation.php and footer.php contain some mix of PHP and HTML to build the navigation and some basic page layout. Everything put together may look like this in a browser:The BenchmarkNow, I'm using ApacheBench (included in every Apache installation) and fire 1000 sequential request at my website:% ab -n 1000 http://demo/welcome.html...Requests per second: 397.25 [#/sec] (mean)...In this case my AMP system was able to serve 397 request per second. Not bad, but it's also a very(!) simple(!) PHP script.Setting up the cacheFirst, I add some lines of code to my previous PHP script.One line just before the include("header.php") statement: ob_start();And this four lines at the end just after the include("footer.php") statement: $output=ob_get_contents(); file_put_contents("cache/".basename($_REQUEST['page']).".html", $output); ob_end_clean(); echo $output;ob_start() instructs PHP keep the generated output into an internal buffer. And the last 4 lines tell PHP to save this buffer into a file, for example: cache/welcome.html.Now I create a directory called cache next to my index.php file and make sure my Apache is able to write and access that directory:% mkdir cache% chmod a+rwx cacheIf I now reload my welcome page in my browser, a file named welcome.html gets created in this cache directory:% ls -l cachetotal 4-rw-r--r-- 1 www-data www-data 732 2009-09-02 13:02 welcome.htmlNow I add this lines to my mod_rewrite configuration (new lines highlighted):RewriteEngine onRewriteCond %{REQUEST_URI} \\.html$RewriteCond %{DOCUMENT_ROOT}/cache/%{REQUEST_URI} -sRewriteRule . /cache/%{REQUEST_URI} [L]RewriteRule \^([\^/]\*).html$ /index.php?page=$1 [L]These three lines reads like this: (first line) For all requests ending with ".html": (second line) If there is a file in the cache directory, named exactly like the resource my web server's got asked for, than (third line) send this file to the browser. If there is no such file, continue with calling the PHP script.The Rerun of the BenchmarkThat's all, now I rerun my benchmark from earlier:% ab -n 1000 http://demo/welcome.html...Requests per second: 1287.57 [#/sec] (mean)...Wow, that's about three times faster as the regular PHP version. And in this example I'm using a very(!) simple(!) PHP script. On a more complex system, the boost will be much higher. For example on www.apachefriends.org we're using a cache based on this recipe and we got a performance win of 300 times. (That's because we have a very complex - some may say crappy - CMS running.)Pros & ConsPros:quite easy to set upno additional software is needed, just an Apache with mod_rewritevery high performance win on slow systemsworks with every web programming language, not only PHPCons:the cache will never refreshthe system doesn't work with user sessionsBut all these drawbacks can be relatively easily solved by adding some more lines of program code or mod_rewrite configurations.

The Scenario Imagine: You run a popular website using AMP technologies, but your hardware is awfully old and over the years, as your website became more and more popular, it's got slower and slower...

Apache

Apache: No listening sockets available, shutting down

One of the advantages of maintaing a popular project like XAMPP is the immense amount of community feedback you get. In other words: your inbox is a daily challenge. PHP 5.3 incompatibilities are hot topics these days. Here's an Apache related one which hits my inbox on a very regular basis.Variant 1Apache complains about:(98)Address already in use: make_sock: could not bind to address [::]:80no listening sockets available, shutting downUnable to open logsThe "Unable to open logs" in this error message is very confusing and often points people in the wrong direction. There is nothing wrong with directory or file permissions, in this case there is simply another process already occupying port 80 and that prevents Apache from binding to this port. Find out which process is using this port and stop it. Alternatively let Apache use another port.Variant 2Actually if it's a permission issue you will see: (13)Permission denied: make_sock: could not bind to address [::]:80no listening sockets available, shutting downUnable to open logsHere Apache tried to bind port 80, but got rejected by the system because the Apache wasn't started as root. On Unix systems only processes by the user root are allowed to bind ports below 1024. To fix this: Let Apache bind to a port above 1024 or start Apache as root.Variant 3If you just get this error, without "no listening sockets available...":(98)Address already in use: make_sock: could not bind to address [::]:80Looks very similar to the first variant, but in this case you probably just have multiple "Listen 80" directives in you Apache's configuration. Become aware of all Include directives in your httpd.conf and remove all Listen duplicates.

One of the advantages of maintaing a popular project like XAMPP is the immense amount of community feedback you get. In other words: your inbox is a daily challenge. PHP 5.3 incompatibilities are hot...

Oracle

Integrated Cloud Applications & Platform Services