Wednesday Jan 20, 2010

Good Idea: Python with FastCGI (mod_fcgid)

A couple of days ago, I stumbled over an installation in which CGI was used to run a Python-based web application. Of course the applications ran terribly slow, and as I mentioned earlier in »Save energy! Stop using CGI!«, it's (nowadays) always a bad idea to use CGI. Not only it's tediously slow and bad software design, it's also soooo 90's.

What's the difference between CGI and FastCGI?

Let me use a metaphor to start. Imagine a well...

python_cgi.jpg python_fastcgi.jpg
Here you see the old-fasioned way of CGI: For every request you have to let the bucket all the way down into the well (fork a new process), allowing water to enter the bucket (initialize and execute your application), pull the bucket up to the surface and empty it (send the data to the web server and free all allocated memory). And here is the modern FastCGI way: Install the faucet (start the FastCGI process) and every time you need water, turn it on (connect and send a request), get water (calculate and get the answer), and turn it off (close the connection). No need to fork, initialize your application, and free the allocated memory on every single request.

Okay, seriously, let me show you how this works in practice.

Installing Python

For this demo I use Sun's Web Stack. It's probably the easiest way to demonstrate the performance differences between CGI and FastCGI. XAMPP doesn't support FastCGI, because with mod_perl for Perl and mod_php for PHP there is no real need for a FastCGI interface.

First, let me add Python to my basic web stack installation:

[oswald@sol10u7 ~/webstack1.5]% bin/pkg install sun-python26
DOWNLOAD                                    PKGS       FILES     XFER (MB)
Completed                                    1/1   2784/2784   12.65/12.65 

PHASE                                        ACTIONS
Install Phase                              2861/2861 
PHASE                                          ITEMS
Reading Existing Index                           7/7
Indexing Packages                                1/1
[oswald@sol10u7 ~/webstack1.5]% bin/setup-webstack

If you're familiar with Sun's Web Stack, you'll have noticed that I'm using the IPS installation of Web Stack. That's my favorite installation way, because it allows me to place the Web Stack in any directory I want and also allows me to run the stack without the need of root privileges.

Python with CGI

Setting up CGI is very, very easy and probably that's exactly the reason why so many people still use it.

Let me start with a simple "Hello World!" Python CGI script:

#!/home/oswald/webstack1.5/bin/python
print ""
print "Hello World!"

I named this file hello.py and put it into the cgi-bin directory of my Apache installation. In the case of Web Stack it's var/apache2/2.2/cgi-bin. Add execute permissions:

[oswald@sol10u7 ~]% chmod a+x var/apache2/2.2/cgi-bin/hello.py

Now I log into another box on the same network and use my favorite command-line web browser Lynx to test the newly created Hello World CGI:

[oswald@debian50 ~]% lynx -source http://sol10u7/cgi-bin/hello.py
Hello World!

Looks good. Now let's benchmark this script:

[oswald@debian50 ~]% ab -n 1000 http://sol10u7/cgi-bin/hello.py
...
Time taken for tests:   31.083 seconds
...
Total transferred:      256000 bytes
HTML transferred:       13000 bytes
Requests per second:    32.17 [#/sec] (mean)
...

32 requests/second. That's nothing to be proud of!

Python with FastCGI

And now let's try FastCGI by adding Apache's mod_fcgid to the Web Stack installation:

[oswald@sol10u7 ~/webstack1.5]% bin/pkg install sun-apache22-fcgid 
DOWNLOAD                                    PKGS       FILES     XFER (MB)
Completed                                    1/1         6/6     0.09/0.09 

PHASE                                        ACTIONS
Install Phase                                  24/24 
PHASE                                          ITEMS
Reading Existing Index                           7/7
Indexing Packages                                1/1
[oswald@sol10u7 ~/webstack1.5]% bin/setup-webstack

Activate the default configuration:

[oswald@sol10u7 ~/webstack1.5]% cp etc/apache2/2.2/samples-conf.d/fcgid.conf etc/apache2/
2.2/conf.d/

For those, who are not able or don't want to use Sun's Web Stack, the above fcgid.conf file basically contains the following directives:

LoadModule fcgid_module libexec/mod_fcgid.so
SharememPath /home/oswald/webstack1.5/var/run/apache2/2.2/fcgid_shm
SocketPath /home/oswald/webstack1.5/var/run/apache2/2.2/fcgid.sock
AddHandler fcgid-script .fcgi
<Location /fcgid>
    SetHandler fcgid-script
    Options ExecCGI
    allow from all
</Location>

As usual after changing Apache's configuration, we need to reload (aka graceful restart) the Apache to let the new configuration take effect:

[oswald@sol10u7 ~/webstack1.5]% apache2/2.2/bin/apachectl graceful

Now I create a new directory named fcgid directly inside of Apache's document root folder and change into that folder:

[oswald@sol10u7 ~/webstack1.5]% mkdir var/apache2/2.2/htdocs/fcgid
[oswald@sol10u7 ~/webstack1.5]% cd var/apache2/2.2/htdocs/fcgid

To let Python to talk with my Apache's mod_fcgid I need to install a so-called Python FastCGI/WSGI gateway. There are several solutions available for Python, but I personally prefer Allan Saddi's fcgi.py:

[oswald@sol10u7 htdocs/fcgid]% wget -q http://svn.saddi.com/py-lib/trunk/fcgi.py

The "Hello World!" Python FastCGI script looks a little different this time:

#!/home/oswald/webstack1.5/bin/python
from fcgi import WSGIServer
def app(environ, start_response):
	start_response('200 OK', [('Content-Type', 'text/html')])
	return('''Hello world!\\n''')
WSGIServer(app).run()

This time it's not the output of a script which is sent back to the browser, it's the return value of a function add() defining the data which goes to the user's browser. In this case it's the simple character string "Hello World!\\n".

Like in the CGI example above, the Python script needs to be executable:

[oswald@sol10u7 htdocs/fcgid]% chmod a+x hello.py

The content of my fcgid directory now looks like this:

[oswald@sol10u7 htdocs/fcgid]% ls -l
total 90
-rw-r--r--   1 oswald   other      44113 Jul 26  2006 fcgi.py
-rwxr-xr-x   1 oswald   other        223 Jan 19 12:48 hello.py

And - like in my CGI example above - I now test the script with Lynx:

[oswald@debian50 ~]% lynx -source http://sol10u7/fcgid/hello.py
Hello world!

And after everything looks fine, I start a little benchmark:

[oswald@debian50 ~]% ab -q -n 1000 http://sol10u7/fcgid/hello.py
...
Time taken for tests:   1.747 seconds
...
Total transferred:      235000 bytes
HTML transferred:       13000 bytes
Requests per second:    572.44 [#/sec] (mean)
...

Yes, gotcha. 572 requests per seconds: that sounds reasonable. Remember the 32 requests/second from CGI? Do you want the well or do you take the faucet? Sure, implementing a FastCGI program is far more challenging then coding a simple CGI solution, but 572 against 32 requests per second? Do I need to say more?

Fotos: On the right "Faucet" by Joe Shlabotnik, and on the left "Well" by echiner1. Both licensed under Creative Commons.

Thursday Jan 07, 2010

Cache, cache, cache! (Part 3: What to cache?)

Happy New Year everyone! Hope you could enjoy your holidays!!

Let's start this year with the third part of my little series of thoughts about caching. After my small memcached intro and thoughts about caching architectures, I now focus on the data you should consider to cache in your web application.

cache-what.png

[1] Cache HTML

Obviously the biggest performance win you can achieve is by caching the whole output of your web application: a simple reverse proxy scenario. This works very well for mostly static pages, but for highly dynamical and user-specific content this is not an option: there is no advantage in caching a web page, which gets obsolete within the next moment.

Probably the best way to solve this dilemma is to implement a so-called partial-page cache: Let your application cache just portions of the page and leave the rest, where it makes no sense to cache, dynamic.

It's very important that you implement this in a very top layer of your application. Probably exactly that layer, which software architects will call presentation layer. Sure, this is likely to break you framework architecture, but to quote chapter 55 of the Tao Te Ching:

The movement of the Tao
By contraries proceeds;
And weakness marks the course
Of Tao's mighty deeds.

But seriously: If you have to stay in the boundaries of a framework, Ajax is a good way to bypass this restrictions and helps to implement such a cache in a restricted architecture. But be aware that this will raise the number of HTTP requests on your frontend web servers.

An effective caching strategy will always mess your beautifully designed software architecture up. Having just one (central) caching layer looks great in system diagrams and it's better than no cache at all, but it's definitely not the end of the rope.

[2] Cache complex data structures

If you don't want to break your framework architecture or you don't like the idea of caching HTML at all, and I totally understand your point, you should consider about caching other (lower level, but still complex) structures of data.

Some examples for suitable data structures:

  • user profiles
  • friends lists
  • current user lists
  • list of locations, branches, countries, languages, ...
  • top 10 (whatever) lists
  • public statistical data
  • ...

The main challenge lies in identifying the most proper data structures. This is no easy task and strongly depends on the kind of web application you run or plan to run. Avoid caching simple data sets, like row-level data from the database. Don't think row-level. That's the best advice you should keep in mind. (Note to myself: I need to put this on a t-shirt. I found this phrase in Memcached Internals, a wonderful article inspired by a talk by Brian Aker and Alan Kasindorf.)

At a first glance Ajax may be an obvious technology to combine with such a cache. But please be aware that moving application logic away from the server-side application to the client side is always a very dangerous task, which easily may compromise the security of your application.

Which allows me to end this post with another quote from Laozi (Tao Te Ching, chapter 63):

All difficult things in the world
are sure to arise from a previous state
in which they were easy.

Thursday Dec 17, 2009

Cache, cache, cache! (Part 2: Architectures)

On Tuesday I focused mainly on memcached and PHP, but today I'll take a wider look at cacheing architectures in general. The main question about defining a cache architecture is to decide where to locate the caching component:

cache-architectures.png

[1] Status quo, the three-tier architecture

In theory, the commonly accepted standard architecture of a software product is divided into three tiers: the presentation tier, the application tier and finally the data tier. In the context of web applications we rediscover this tiers in the trinity of web server, application server and database server.

In the above diagram we find these three tiers with the user (or in technical terms: the browser) on top of this stack.

[2] Cache on top

One very obvious idea is to place the cache in front of the web server, between user and web server. Usually we find this architecture in a so called reverse proxy configuration. A reverse proxy is quite easy to set up and has a positive impact for web sites with more static content. But for highly dynamic web applications - like most of today's Web 2.0 applications - the caching benefit of a reverse proxy may be not that big.

In general: having a reverse proxy is better than no caching at all. A reverse proxy will give you always a performance benefit.

[3] Cache in between of web and application server

Let's move the cache one level down the stack in between web server and application server. On the fist sight this may look like a very good idea, because the cache now protects the application server. But on the second sight you'll realize that this configuration is mostly the same as that one from architecture 2, just without the benefit also caching your web server's data.

For exotic scenarios there may be a good reason for this configuration (esp. in combination with load balancing functionality) but in general you should favor architecture 2 over this one.

[4] Cache in between of application and database

And another level down in the stack. The cache now sits between application server and database. Again this looks good, and seems to be a good idea - on the first sight. But on the second or third sight you may realize that nearly every database system has its own internal query cache and our cache is only a cache for a cache. And caching a cache is basically never a good idea and can lead to unpredictable, bad consequences.

Another difficulty with this approach is that it's hard to decide when the cache gets dirty (cache jargon for obsolete) and when it's time to clear the cache.

[5] Cache inside of application

And now half a level up again: right into the application tier. This is the most challenging but also the most powerful place to implement caching strategies. Identify time-critical and frequently accessed data during the development process and implement dedicated and customized caching mechanisms. But don't try do build an abstract, unified, common cache for everything.

It's very important to find a specific and suitable solutions for each kind of data you want to cache in your application. Otherwise you'll will probably just end with another row-based cache for your database (like architecture 4) or some kind of reverse proxy (like architecture 2).

Conclusion

Architectures 2, 3 and 4 can be easily setup by system administration without having to involve development in any way. It's mostly a matter of clever configuration which also may add some load balancing features. In general you'll definitely achieve a better performance of your application, but there is always a given limit by the architecture and scaling quality of your core application.

Architecture 5 is probably the best choice, but - to get best results - needs to be started in an early stage and during the whole development and designing process of your web application you should always have caching in mind. What data is most frequently accessed? What data is expensive (hard to retrieve)? What data depends on user sessions? How up to date does the data need to be?

If you are curious about these questions, please stay tuned for part 3.

Tuesday Dec 15, 2009

Cache, cache, cache! (Part 1: memcached)

Caching is probably the most important technique you should use in nowadays web sites or web application. Sure, scaling your hardware is still the final answer to all your load problems, but with some kind of caching your application will scale far better rather than without.

cachecachecache.png

Currently my favorite caching tool is memcached. It's a slim and ultra fast distributed caching system. Memcached is basically a key-value store, which stores all data non-persistently in memory and if your server goes down all the data is also gone because it's not stored somewhere on a hard disk.

Memcached is not meant to be a database, and you'll still need a database to store your data persistently.

Setting up memcached

I'm a very lazy guy and try to avoid boring duties like installing memcached. That's why I love using Sun's Web Stack, which already includes memcached and is so easy to use. If you're not a Web Stack user please take a look at the memcached FAQ to learn how to install memcached on your system.

To add memcached to my IPS-based Web Stack installation I simply call these two commands:

[oswald@localhost ~/demo]$ bin/pkg install sun-memcached
DOWNLOAD                                    PKGS       FILES     XFER (MB)
Completed                                    1/1         9/9     0.17/0.17 

PHASE                                        ACTIONS
Install Phase                                  30/30 
PHASE                                          ITEMS
Reading Existing Index                           7/7
Indexing Packages                                1/1
[oswald@localhost ~/demo]$ bin/setup-webstack

Now all I need to do is to start the daemon:

[oswald@localhost ~/demo]$ bin/sun-memcached start
Starting memcached

Memcached has no support for any access control at all and you should use memcached only on private networks or secure you installation with a firewall (port 11211, by the way).

Using memcached with PHP

As I already mentioned memcached is a simple key-value store which is very easy to use for programmers. To show the basic idea I put this small PHP script together:

<?php
        $memcache = new Memcache();
        if(!$memcache->connect('localhost', 11211))
                die("Couldn't connect to memcached! Cruel world!");

        $key="zaphod";

        $result = $memcache->get($key);

        if($result)
        {
                echo "$key is $result";
        }
        else
        {
                $value="cool";
                echo "Set $key to $value";
                $memcache->set($key,$value);
        }
?>

There are three main functions you will need to understand in order to work with memcached:

connect(host,port)
to connect to your memcached server. If you have multiple memcached servers running you can use addServer() to add one or more servers to the connection pool.
get(key)
Retrieves the value for the given key.
set(key,value)
Stores the given value for the given key. set() also allows you to define an expiration time for the key-value pair.

On the first execution of this script the cache is empty and you'll get this output:

Set zaphod to cool

On the second execution, the value for zaphod is already set and you'll see:

zaphod is cool

That's all. That's the basic way to use memcached.

What's next...

The next step is to decide what information you want to cache and where do you want to cache. Both are very crucial decisions which determine success or failure of your cache. So, stay tuned for part 2. ;)

Thursday Nov 05, 2009

Importing a VDI in VirtualBox

If you're used to be a VMware user and try to switch to the Open-Source side of the Force by using VirtualBox, you may run into difficulties if you try to import an existing VDI file into VirtualBox. Actually it's quite easy, if you know how.

The main difference between VMware and VirtualBox is that VMware captures a whole virtual machine in an image, whereas VirtualBox only supports images of a hard disk. So in VirtualBox's world, you first need to create a new virtual machine, before using an existing VirtualBox image.

  1. First copy your VDI file into VirtualBox's virtual hard disks repository. On Mac OS X it's $HOME/Library/VirtualBox/HardDisks/.

  2. Start VirtualBox and create a new virtual machine (according to the OS you expect to live on the VirtualBox image):

    virtualbox1.jpg
  3. When you're asked for a hard disk image, select Use existing hard disk and click on the small icon on the right:

    virtualbox2.jpg
  4. Which will brings you to the Virtual Media Manager. Click on Add and select the VDI file from step 1.

    virtualbox3.jpg
  5. After leaving the Virtual Media Manager, you'll be back in your virtual machine wizard. Now you can select your new VDI as existing hard disk and finalize the creation process.

    virtualbox4.jpg
  6. Back in the main window, you're now able to start your new virtual machine:

    virtualbox5.jpg

It's quite easy, if you know how.

Tuesday Oct 13, 2009

What is Java?

Just learned that the question "what is java" is on place 3 of Google's Zeitgeist ranking 2008 for "what is" questions - right after "what is love" (first place) and "what is life" (second place).

Indeed a good question. Let's take a look a Google's top hits to this question:

About.com (Hit #9)

Java is a computer programming language. It enables programmers to write computer instructions using English based commands, instead of having to write in numeric codes. It’s known as a “high-level” language because it can be read and written easily by humans. Like English, Java has a set of rules that determine how the instructions are written. These rules are known as its “syntax”. Once a program has been written, the high-level instructions are translated into numeric codes that computers can understand and execute. (Source: http://java.about.com/od/gettingstarted/a/whatisjava.htm)

Webopedia.com (Hit #7)

A high-level programming language developed by Sun Microsystems. Java was originally called OAK, and was designed for handheld devices and set-top boxes. Oak was unsuccessful so in 1995 Sun changed the name to Java and modified the language to take advantage of the burgeoning World Wide Web.
(Source: http://www.webopedia.com/TERM/J/Java.html)

Wikipedia.org (Hit #4)

Java is a programming language originally developed by James Gosling at Sun Microsystems and released in 1995 as a core component of Sun Microsystems' Java platform. The language derives much of its syntax from C and C++ but has a simpler object model and fewer low-level facilities. Java applications are typically compiled to bytecode (class file) that can run on any Java Virtual Machine (JVM) regardless of computer architecture. (Source: http://en.wikipedia.org/wiki/Java_(programming_language))

Boutell.com (Hit #3)

Java is a technology that allows software designed and written just once for an idealized "virtual machine" to run on a variety of real computers, including Windows PCs, Macintoshes, and Unix computers. On the web, Java is quite popular on web servers, used "under the hood" by many of the largest interactive websites. Here it serves the same role that PHP, ASP or Perl might, although traditionally Java has been used for larger-scale projects.
(Source: http://www.boutell.com/newfaq/definitions/java.html)

Java.com (Hit #1)

What is Java? Java allows you to play online games, chat with people around the world, calculate your mortgage interest, and view images in 3D, just to name a few. It's also integral to the intranet applications and other e-business solutions that are the foundation of corporate computing.
(Source: http://www.java.com/en/download/whatis_java.jsp)

As you see, it's not an easy question to answer!

Wednesday Oct 07, 2009

Qs about you and Linux

My last blog entry about Linus Torvals' thoughts on the goto statement brought and old email interview back to my mind, which I had the honor to have with him a long time ago in 1994.

From cs.Helsinki.FI!Linus.Torvalds Wed Jun 22 20:45:59 1994
From: torvalds@cs.Helsinki.FI (Linus Torvalds)
Date: Wed, 22 Jun 1994 21:45:52 +0300
In-Reply-To: Kai Seidler's message as of Jun 22, 12:13
X-Mailer: Mail User's Shell (7.2.4 2/2/92)
To: oswald@duplox.wz-berlin.de (Kai Seidler)
Subject: Re: Qs about you and linux
Status: RO

Kai Seidler: "Qs about you and linux" (Jun 22, 12:13):
>  
> Did your role as linux programmer has changed over the time? From the
> alone linux programmer (1991) to a linux god? How much time do you
> spend today in programming in opposition to manage kernel-patches,
> answer stupid questions (about bugs, and like this one :), visit
> congresses?

Oh, it has changed, all right.  In 1991, I essentially coded 8 hours a
day and didn't mind about other people or "secondary" stuff like
portability etc.  As it stands now, I get to code occasionally when I
find some time and have something interesting to do, but most of my
linux time is simply "management" these days.  I'm not wearing any
suits, though :-)

Just reading mail takes about 2 hours a day - I also read the newsgroups
when I can, but that usually means just col.announce and selected
articles from col.development.  Applying patches isn't that bad: I have
people I trust that do the large patches and then I just need to check
them over against obvious problems.  The "un-trusted" patches are much
rarer. 

Actually, the above sounds worse than it is.  The fact is that the basic
kernel mostly works well enough and one reason for me not coding quite
as much as I use to do is simply that the basic functionality that I've
personally always concentrated on doesn't need that much care any more. 
The patches these days are mostly networking and device drivers with the
occasional smaller stuff elsewhere. 

Conferences haven't been a problem until recently, and on the whole they
haven't really proved too distractive.  I don't really like giving
talks, but I like meeting people and traveling and I also feel it's
simething that needs to be done at this point. 

> Why do you, and all the other people, such an enormous work for free?
> What do you mean? Is it fame? What did you get back from the Linux
> community?

Well, the fame certainly doesn't hurt, of course: I expect to be able to
get a good job once I get my studies completed and decide to leave the
university.  But mostly it's just a project I like doing, and one which
people appreciate.  A hobby of the best kind, in short..  I think that's
true for most of the kernel developers.

> How is your relation to the FreeBSD community? As far as I see, the 
> Linux and FreeBSD don't like each other very much, but I may be wrong.
> Maybe it's "only" the old "war" between SysV and BSD?

Actually, we are on a friendly standing with the FreeBSD people (I
haven't been much in contact with NetBSD).  The communities easily get
inte flames over which is better, but I know both the linux and BSD
kernel developers are much too involved with their (our) own projects to
really mind any of the flames. 

I've met with some of the FreeBSD people a few times (on conferences),
and they are nice (jkh has something like 11 cats: I just have two).
It's hard to co-operate too much, though: it takes a lot of time and it
seems to actually be easier just to concentrate on your own project.

		Linus

That was 15 years ago. Nobody would have thought at that time that Linux would later become such a big competitor for commercial Unix-esque operating systems.

Tuesday Oct 06, 2009

Is goto the root of all evil?

As PHP 5.3 introduced the goto statement the old discussion about the evilness of goto came back to the surface of the Internet.

By coincidence I stumbled over this 6 years old discussion with Linus Torvalds about this topic. He's arguing goto makes the source code more readable, but read his thoughts and decide for yourself:

From: Linus Torvalds
Subject: Re: any chance of 2.6.0-test\*?
Date: 	Sun, 12 Jan 2003 12:22:26 -0800 (PST)

On Sun, 12 Jan 2003, Rob Wilkens wrote:
> 
> However, I have always been taught, and have always believed that
> "goto"s are inherently evil.  They are the creators of spaghetti code

No, you've been brainwashed by CS people who thought that Niklaus Wirth
actually knew what he was talking about. He didn't. He doesn't have a
frigging clue.

> (you start reading through the code to understand it (months or years
> after its written), and suddenly you jump to somewhere totally
> unrelated, and then jump somewhere else backwards, and it all gets ugly
> quickly).  This makes later debugging of code total hell.  

Any if-statement is a goto. As are all structured loops.

And sometimes structure is good. When it's good, you should use it.

And sometimes structure is _bad_, and gets into the way, and using a 
"goto" is just much clearer.

For example, it is quite common to have conditionals THAT DO NOT NEST.

In which case you have two possibilities

 - use goto, and be happy, since it doesn't enforce nesting

	This makes the code _more_ readable, since the code just does what 
	the algorithm says it should do.

 - duplicate the code, and rewrite it in a nesting form so that you can 
   use the structured jumps.

	This often makes the code much LESS readable, harder to maintain, 
	and bigger.

The Pascal language is a prime example of the latter problem. Because it 
doesn't have a "break" statement, loops in (traditional) Pascal end up 
often looking like total shit, because you have to add totally arbitrary 
logic to say "I'm done now".

		Linus

Read the full discussion at: http://kerneltrap.org/node/553/2131

Friday Sep 25, 2009

How to upgrade VirtualBox Guest Additions on Solaris/OpenSolaris?

Today I needed to play around work out something on RHEL and OpenSolaris. I have both systems running in a VirtualBox on my Mac and because of the latest update to VirtualBox I was supposed to update also the so called Guest Additions on RHEL and OpenSolaris.

The update went smoothly on RHEL, but on OpenSolaris I got this frightened message:

Current administration requires that a unique instance of the
<SUNWvboxguest> package be created.  However, the maximum number of
instances of the package which may be supported at one time on the
same system has already been met.

No changes were made to the system.

As you already may have noticed I'm not a native English speaker, but is requires a unique instance and maximum number of instances has already been met a proper way to express: The package is already installed, and that's why the package can't get installed another time?

Indeed, as the VirtualBox User Manual (3.0.6) states:

The Guest Additions should be updated by first uninstalling the existing Guest Additions and then installing the new ones. Attempting to install new Guest Additions without removing the existing ones is not possible.

But how? That's not mentioned in the users manual. Why? I'm new to VirtualBox and it's the first time I (want to) upgrade the guest additions.

»Look Dave, I can see you're really upset about this. I honestly think you ought to sit down calmly, take a stress pill, and think things over.«

Okay, you're right, calm down. To cut a long story short:

  1. To uninstall the current package:
    # pkgrm SUNWvboxguest
    The following package is currently installed:
       SUNWvboxguest  Sun VirtualBox Guest Additions
                      (i386) 3.0.4,REV=r50677.2009.08.04.19.24
    Do you want to remove this package? [y,n,?,q] y
    ## Removing installed package instance <SUNWvboxguest>
    This package contains scripts which will be executed with super-user
    permission during the process of removing this package.
    Do you want to continue with the removal of this package [y,n,?,q] y
    ## Verifying package <SUNWvboxguest> dependencies in global zone
    ## Processing package information.
    ## Executing preremove script.
    ...
    ...
    Removal of <SUNWvboxguest> was successful.
  2. And install the new package:
    /media/VBOXADDITIONS_3.0.6_52128# pkgadd -d VBoxSolarisAdditions.pkg 
    The following packages are available:
      1  SUNWvboxguest     Sun VirtualBox Guest Additions
                           (i386) 3.0.6,REV=r52128.2009.09.09.19.32
    Select package(s) you wish to process (or 'all' to process
    all packages). (default: all) [?,??,q]: a
    Processing package instance <SUNWvboxguest> from 
    </media/VBOXADDITIONS_3.0.6_52128/VBoxSolarisAdditions.pkg>
    Sun VirtualBox Guest Additions(i386) 3.0.6,REV=r52128.2009.09.09.19.32
    VirtualBox Personal Use and Evaluation License (PUEL)
    ...
    ...
    This package contains scripts which will be executed with super-user
    permission during the process of installing this package.
    Do you want to continue with the installation of <SUNWvboxguest> [y,n,?] y
    Installing Sun VirtualBox Guest Additions as <SUNWvboxguest>
    ...
    ...
    Please re-login to activate the X11 guest additions.
    If you have just un-installed the previous guest additions a REBOOT is required.
    Installation of <SUNWvboxguest> was successful.
  3. And because I just un-installed the previous guest additions I now initiate reboot:
    # /sbin/init 6
εὕρηκα! ;)

Tuesday Sep 15, 2009

I (wireframing) Robot

Last week one my primary occupation was wireframing. It took me quite a long time to finish the first wireframes using a normal vector graphics editor and I started to look after a simpler and more efficient way to create my mockups. I quickly stumbled over Balsamiq Mockups.

balsamicmockups.jpg

It's by far not as featureful as products as OmniGraffle, or if you're living on the dark side of the moon: as Visio, but it's very easy to use and it's acutally fun using it.

So, if you need to do wireframes from time to time, Balsamiq Mockups is probably the right tool for you. If you do wireframes and nothing but wireframes all day long, you probably better stay with the tool you're already using.

With US$ 79 it's quite pricey, and I personally think that's too expensive. But if you need it, you'll pay.

Monday Sep 07, 2009

Perlify your command line

I'm not a Perl expert, not even someone who know how to actually program in Perl. So please don't expect anything miraculous about Perl coming through my keyboard. I happen to prefer PHP or AWK personally, but I respect Perl very much. And sometimes, even recently, it has saved my life.

The directory tree of a big server changed and I needed to update outdated path details in some hundreds configurations files.

Usually I do this using sed, but this time I was on a system without any GNU sed available:

[oswald@solaris10 ~] cat life
My life is so bad!
[oswald@solaris10 ~] sed -i 's/bad/good/g' life
sed: illegal option -- i

In this case Perl was my life saver:

[oswald@solaris10 ~] perl -pi -e 's/bad/good/g' life
[oswald@solaris10 ~] cat life
My life is so good!

Thank you, Perl.

Friday Sep 04, 2009

Remember the time, security awareness was a topic?

Setting up this blog remembered me on an old security issue I stumbled upon some years ago. At that time I was simply lazy to investigate in more detail, but this time I took a closer look.

Let's do the time warp...

Before getting into this security thing, let's do a little time travel: Do you remember telnet and the so called rtools like rlogin/rsh/rcp/...? As ethernet hardware became cheaper and available for private use, these commands quickly disappeared from the TCP/IP dance floor and were quickly replaced by SSH counterparts. Do you also remember why did this happen? Yes, because these tools allowed an IP-based authentication and/or sent the user's password plain text over the network. They have done so since many years, but now because of the cheap network hardware it became easy for "everyone" to spy on a network and fetch all the passwords walking by.

SSH stopped this. For a while...

The presence

Today, with Web 2.0, security awareness seems to be a lost ideal from the past. No one cares about the privacy of private data or the security of their communication. So many of those fancy and stylish social network services use plain text password authentication over HTTP. Or if you take a look at the incredible popular and narcissistic blogosphere, where XML-RPC based APIs were used by blogging clients to talk to their online publishing system, you'll find this:

An additional downside of XML-RPC based APIs is their lack of security. Although security can be introduced by other means such HTTPS, in their basic operation these transports deliver user credentials as plain text. This leaves an open door to password interception.
("Beginning RSS and Atom programming" by Danny Ayers and Andrew Watt, 2005, page 525)

Or before that in 2003 Mark Pilgrim stated in »The Atom API« about the MetaWeblog API:

Passwords are sent in the clear, as plain text.

That was 6 years ago. At least 6 years of plain text passwords. 6 years of free and unencrypted WLAN access at Starbucks or whatever other public place. You don't need to be a genius to smell a rat there.

Who to blame?

And you can't actually blame MetaWeblog or other XML-RPC APIs. Why should the API concern about encryption, if it's so easy to implement in a lower layer? Like in HTTPS as mentioned above. It's so easy to set up HTTPS and use it for the XML-RPC API. Sure, big and popular blog hosting services already do it this way, but myriads of self-maintained, self-hosted blogs don't. Doesn't matter if it's a private or a corporate blog.

Can you blame the user? No. The user should be able to turn the computer on and off, and USE it. He's there to use the computer and not to think about it. I don't want to think about how an ATM secures my financial transactions.

Can you blame the system administrators? Yes, a big yes. Whoever is responsible for the IT infrastructure should be aware of this issue and should take appropriate actions.

Who else? Of course the software developers! Another big yes. Why do they make it so easy for the people to set up insecure systems? They can easily implement something to force the use HTTPS or at least, inform the users and ask them to actively do something to enable unencrypted XML-RPC over HTTP.

Wednesday Aug 26, 2009

Perl now supports DTrace out of the box

Just scanned through the Perl 5.10.1 delta document and found this promising statement:

Some support for DTrace has been added. See "DTrace support" in INSTALL.

And according INSTALL all you need to do is to add -Dusedtrace to Configure and your Perl will be ready to DTrace. Very cool.

About

Kai 'Oswald' Seidler writes about his life as co-founder of Apache Friends, creator of XAMPP, and technology evangelist for web tier products at Sun Microsystems.

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today