Good Idea: Python with FastCGI (mod_fcgid)
By Kai Seidler on Jan 20, 2010
A couple of days ago, I stumbled over an installation in which CGI was used to run a Python-based web application. Of course the applications ran terribly slow, and as I mentioned earlier in »Save energy! Stop using CGI!«, it's (nowadays) always a bad idea to use CGI. Not only it's tediously slow and bad software design, it's also soooo 90's.
What's the difference between CGI and FastCGI?
Let me use a metaphor to start. Imagine a well...
|Here you see the old-fasioned way of CGI: For every request you have to let the bucket all the way down into the well (fork a new process), allowing water to enter the bucket (initialize and execute your application), pull the bucket up to the surface and empty it (send the data to the web server and free all allocated memory).||And here is the modern FastCGI way: Install the faucet (start the FastCGI process) and every time you need water, turn it on (connect and send a request), get water (calculate and get the answer), and turn it off (close the connection). No need to fork, initialize your application, and free the allocated memory on every single request.|
Okay, seriously, let me show you how this works in practice.
For this demo I use Sun's Web Stack. It's probably the easiest way to demonstrate the performance differences between CGI and FastCGI. XAMPP doesn't support FastCGI, because with mod_perl for Perl and mod_php for PHP there is no real need for a FastCGI interface.
First, let me add Python to my basic web stack installation:
[oswald@sol10u7 ~/webstack1.5]% bin/pkg install sun-python26 DOWNLOAD PKGS FILES XFER (MB) Completed 1/1 2784/2784 12.65/12.65 PHASE ACTIONS Install Phase 2861/2861 PHASE ITEMS Reading Existing Index 7/7 Indexing Packages 1/1 [oswald@sol10u7 ~/webstack1.5]% bin/setup-webstack
If you're familiar with Sun's Web Stack, you'll have noticed that I'm using the IPS installation of Web Stack. That's my favorite installation way, because it allows me to place the Web Stack in any directory I want and also allows me to run the stack without the need of root privileges.
Python with CGI
Setting up CGI is very, very easy and probably that's exactly the reason why so many people still use it.
Let me start with a simple "Hello World!" Python CGI script:
#!/home/oswald/webstack1.5/bin/python print "" print "Hello World!"
I named this file hello.py and put it into the cgi-bin directory of my Apache installation. In the case of Web Stack it's var/apache2/2.2/cgi-bin. Add execute permissions:
[oswald@sol10u7 ~]% chmod a+x var/apache2/2.2/cgi-bin/hello.py
Now I log into another box on the same network and use my favorite command-line web browser Lynx to test the newly created Hello World CGI:
[oswald@debian50 ~]% lynx -source http://sol10u7/cgi-bin/hello.py Hello World!
Looks good. Now let's benchmark this script:
[oswald@debian50 ~]% ab -n 1000 http://sol10u7/cgi-bin/hello.py ... Time taken for tests: 31.083 seconds ... Total transferred: 256000 bytes HTML transferred: 13000 bytes Requests per second: 32.17 [#/sec] (mean) ...
32 requests/second. That's nothing to be proud of!
Python with FastCGI
And now let's try FastCGI by adding Apache's mod_fcgid to the Web Stack installation:
[oswald@sol10u7 ~/webstack1.5]% bin/pkg install sun-apache22-fcgid DOWNLOAD PKGS FILES XFER (MB) Completed 1/1 6/6 0.09/0.09 PHASE ACTIONS Install Phase 24/24 PHASE ITEMS Reading Existing Index 7/7 Indexing Packages 1/1 [oswald@sol10u7 ~/webstack1.5]% bin/setup-webstack
Activate the default configuration:
[oswald@sol10u7 ~/webstack1.5]% cp etc/apache2/2.2/samples-conf.d/fcgid.conf etc/apache2/ 2.2/conf.d/
For those, who are not able or don't want to use Sun's Web Stack, the above fcgid.conf file basically contains the following directives:
LoadModule fcgid_module libexec/mod_fcgid.so SharememPath /home/oswald/webstack1.5/var/run/apache2/2.2/fcgid_shm SocketPath /home/oswald/webstack1.5/var/run/apache2/2.2/fcgid.sock AddHandler fcgid-script .fcgi <Location /fcgid> SetHandler fcgid-script Options ExecCGI allow from all </Location>
As usual after changing Apache's configuration, we need to reload (aka graceful restart) the Apache to let the new configuration take effect:
[oswald@sol10u7 ~/webstack1.5]% apache2/2.2/bin/apachectl graceful
Now I create a new directory named fcgid directly inside of Apache's document root folder and change into that folder:
[oswald@sol10u7 ~/webstack1.5]% mkdir var/apache2/2.2/htdocs/fcgid [oswald@sol10u7 ~/webstack1.5]% cd var/apache2/2.2/htdocs/fcgid
To let Python to talk with my Apache's mod_fcgid I need to install a so-called Python FastCGI/WSGI gateway. There are several solutions available for Python, but I personally prefer Allan Saddi's fcgi.py:
[oswald@sol10u7 htdocs/fcgid]% wget -q http://svn.saddi.com/py-lib/trunk/fcgi.py
The "Hello World!" Python FastCGI script looks a little different this time:
#!/home/oswald/webstack1.5/bin/python from fcgi import WSGIServer def app(environ, start_response): start_response('200 OK', [('Content-Type', 'text/html')]) return('''Hello world!\\n''') WSGIServer(app).run()
This time it's not the output of a script which is sent back to the browser, it's the return value of a function add() defining the data which goes to the user's browser. In this case it's the simple character string "Hello World!\\n".
Like in the CGI example above, the Python script needs to be executable:
[oswald@sol10u7 htdocs/fcgid]% chmod a+x hello.py
The content of my fcgid directory now looks like this:
[oswald@sol10u7 htdocs/fcgid]% ls -l total 90 -rw-r--r-- 1 oswald other 44113 Jul 26 2006 fcgi.py -rwxr-xr-x 1 oswald other 223 Jan 19 12:48 hello.py
And - like in my CGI example above - I now test the script with Lynx:
[oswald@debian50 ~]% lynx -source http://sol10u7/fcgid/hello.py Hello world!
And after everything looks fine, I start a little benchmark:
[oswald@debian50 ~]% ab -q -n 1000 http://sol10u7/fcgid/hello.py ... Time taken for tests: 1.747 seconds ... Total transferred: 235000 bytes HTML transferred: 13000 bytes Requests per second: 572.44 [#/sec] (mean) ...
Yes, gotcha. 572 requests per seconds: that sounds reasonable. Remember the 32 requests/second from CGI? Do you want the well or do you take the faucet? Sure, implementing a FastCGI program is far more challenging then coding a simple CGI solution, but 572 against 32 requests per second? Do I need to say more?Fotos: On the right "Faucet" by Joe Shlabotnik, and on the left "Well" by echiner1. Both licensed under Creative Commons.