Tarballs and CGI scripts


One of the things that a performance group tends to generate is a huge amount of data, and while we make all of this available online, sometimes people like to just grab the data and analyse it later. So in order to make it easier for them to do this (and the fact that it is definately needed to finish up the current project I'm working on) I wrote up a quick cgi script earlier on today that automagically generates tarballs from a cgi script. It may be of use to some folks.

First a quick explanation about using stdout for output, in tar you use the - option, while in compress you just don't give it any options. So to create a compressed tarball on standard out for the directory your in you would do the following.

tar -cf \* - | compress
Obviously this isn't really much use unless you like your terminal beeping at you a lot (personally I don't, even when listening to techno[1] ;) ). So lets make a little bit more use of these features.

Our perl script does the following, first up it creates our cgi header, we need to set the mime type correctly (or semi correctly in this case, CGI.pm has, I belive, a limitation about what it will allow you to use if you wish to give a binary file a name and force its download, but I could be wrong on this). So in this case the mime type is set to being "application/octet-stream" while we use the -attachment attribute to give a file name.

Next its just a matter of outputting the actual tarfile, so we switch into binary mode for the output, and then read in the output we are getting from standard out, buffering it to make sure we don't run into any strangeness, and then print out each buffer. And thats it. The relevant snippets of code are below for your perusal.

#!/usr/bin/perl

use CGI qw(:standard);
use CGI::Carp qw(fatalsToBrowser);
use constant BUFFER_SIZE => 4_096;
use strict;

my $buffer = "";
my $tar = "/usr/bin/tar";
my $compress = "/usr/bin/compress";

my $cgi = new CGI;
my $dir = $cgi->param("dir");

print $cgi->header(-type => "application/octet-stream", 
		   -expires => "-1d", 
		   -attachment => "$dir.tar.Z" );

binmode STDOUT;
local \*TARBALL;

chdir($dir);
open(TARBALL, "$tar -cf - \* | $compress |");
while ( read( TARBALL, $buffer, BUFFER_SIZE ) ) {
    print $buffer;
}
close TARBALL;

[1] Miss Kittin's Live at I Love Techno is the current album of choice... more electro than techno though, which is a good thing
Comments:

Post a Comment:
  • HTML Syntax: NOT allowed
About

fintanr

Search

Archives
« April 2014
MonTueWedThuFriSatSun
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
    
       
Today