Tuesday Feb 05, 2008

Using HTTP compression to speed up content delivery in Sun Java System Web Server 7.0 update 2

Using HTTP compression to speed up content delivery in Sun Java System Web Server 7.0 update 2


If you are looking for faster web page downloads, you can use HTTP compression feature which compresses your content to speed it up. It is especially beneficial for low bandwidth connections. This also reduces the number of bytes transferred and improves the overall server performance.

How it Works

Browsers that support compressed content send an Accept-encoding header with value gzip, deflate. Our Web Server sees the header and chooses to provide compressed content, and sends Content-encoding: gzip response header. The browser on seeing this header tries to decompress the content and renders it.

Now the question is how to enable HTTP compression feature can be configured in obj.conf. Let me explain that in detail in this blog.

HTTP Compression of Static Files

    For dynamic compression of static files in  Sun Java System Web Server 7.0 Update 2, we need to use compress-file SAF with find-compressed SAF.

    The compress-file function creates the compressed file in the subdirectory specified if the file size is in between min-size and max-size the first time a request is sent on the URI. If Web Server needs to compress that static file for every static request then it may not really improve performance. It will reduce the network traffic but will cause more CPU usage. So content compression will only happen once or until the uncompressed original file is modified. If check-age is set to true, it recreates the compressed file if the compressed version is not as recent as the non-compressed version. The find-compressed function checks if a compressed version of the requested file is available. If the following conditions are met, find-compressed changes the path to point to the compressed file:
  • A compressed version is available.
  • The compressed version is as recent as the non-compressed version.
  • The client supports compression.
  • If the HTTP method is GET or HEAD.

Example

    This is an example of how server compresses html files and places them in .compressed directory (relative to the original file location). Modify the default object of obj.conf as shown below

<Object name="default">
NameTrans fn="assign-name" from="\*.html" name="find-compressed"
...
Service method=(GET|HEAD|POST) type=\*~magnus-internal/\* fn=compress-file subdir=".compressed-files"
Service method=(GET|HEAD|POST) type=\*~magnus-internal/\* fn=send-file
...
</Object>

<Object name="find-compressed">
PathCheck fn="find-compressed"
</Object>

    However, if you plan to put precompressed content in the location where the original file is present yourself, you can just use find-compressed SAF . A compressed version of a file must have the same file name as the non-compressed version but with a .gz suffix. For example, the compressed version of a file named /httpd/docs/index.html would be named /httpd/docs/index.html.gz. To compress files, you can use the freely available gzip program.
<Object name="default">
NameTrans fn="assign-name" from="\*.html" name="find-compressed"
...
</Object>
<Object name="find-compressed">
PathCheck fn="find-compressed"

</Object>

HTTP Compression of Dynamic Content

    The http-compression filter compresses outgoing  dynamic content such as the output from SHTML pages, CGI programs, or pages created with JavaServer PagesTM (JSPTM) technology that need to be interpreted by the server.

Example

Output fn="insert-filter" type="text/\*" filter="http-compression" vary="on" compression-level="9"

In this example, type="text/\*" restricts compression to documents that have a MIME type of text/\* (for example, text/ascii, text/css, text/html, and so on).

Suppose you want to compress only JSPs. Modify the default object of obj.conf as shown below
<Object name="default">
NameTrans fn="assign-name" from="\*.jsp" name="compress-on-demand"
...
</Object>

<Object name="compress-on-demand">
Output fn="insert-filter" filter="http-compression"
</Object>

Parameters

Here is a list of parameters you can use if you want to tune these functions

Parameters for compress-file

  • subdir
    • (Optional) This should be a directory name only relative to the directory in which the original non-compressed file is located.
    •  To overwrite a pre-compressed compressed file lying in docroot  set the subdir to ".".
    • The default value  is "." will overwrite any precompressed gz files if any.
  • check-age            (Optional) The values can be true or false.        The default value is true.
  • vary                   (Optional) The values can be true or false.        The default value is true.
  • compression-level (Optional) The values can be 1 to 9.                  The default value is 6.
  • min-size             (Optional) The values can be 0 to INT_MAX.           The default value is 256.
  • max-size         (Optional) The values can be min-size to INT_MAX. The default value is 1048576.

Parameters for find-compressed

  • check-age
    • (Optional)  Specifies whether to check if the compressed version is older than the non-compressed version. 
    • The values can be yes or no. By default, the value is set to yes.
    • If set to yes, the compressed version will not be selected if it is older than the non-compressed version.
    •  If set to no, the compressed version is always selected, even if it is older than the non-compressed version.
  • vary 
    • (Optional) Specifies whether to insert a Vary: Accept-Encoding header.
    • The values can be yes or no. By default, the value is set to yes.
    • If set to yes, a Vary: Accept-Encoding header is always inserted when a compressed version of a file is selected.
    •  If set to no, a Vary: Accept-Encoding header is never inserted. 

Parameters for http-compression

  • vary 
    • Controls whether the filter inserts a Vary: Accept-encoding header.
    • If vary is absent, the default value is yes. yes tells the filter to insert a Vary: Accept-encoding header when it compresses content. 
    • no tells the filter to never insert a Vary: Accept-encoding header.
  • fragment-size
    • Size in bytes of the memory fragment used by the compression library to control how much to compress at a time.
    • The default value is 8096.
  • compression-level
    • Controls the compression level used by the compression library.
    • Valid values are from 1 to 9. A value of 1 results in the best speed.
    • A value of 9 results in the best compression.
    • The default value is 6.
  • window-size 
    • Controls an internal parameter of the compression library.
    • Valid values are from 9 to 15.
    • Higher values result in better compression at the expense of memory usage.
    • The default value is 15.
  • memory-level 
    • Controls how much memory is used by the compression library.
    • Valid values are from 1 to 9.
    • A value of 1 uses the minimum amount of memory but is slow.
    • A value of 9 uses the maximum amount of memory for optimal speed.
    • The default value is 8.

Using wadm (Administration CLI) to configure these HTTP compression features

You can use the following wadm commands to configure these.
For compression of static content i.e. find-compressed and compress-file functions use
wadm> enable-precompressed-content
Usage: enable-precompressed-content --help|-?
  or   enable-precompressed-content [--echo] [--no-prompt] [--verbose] [--uri-pattern=pattern] [--no-vary-header] [--no-age-check] [--compress-file] [--sub-dir=sub-directory] [--compression-level=1-9] [--min-size=size-in-bytes] [--max-size=size-in-bytes] --config=config-name --vs=vs-name
CLI014 vs is a required option.
wadm>
For compression of dynamic content i.e. http-compression function use
wadm> enable-on-demand-compression
Usage: enable-on-demand-compression --help|-?
  or   enable-on-demand-compression [--echo] [--no-prompt] [--verbose] [--uri-pattern=pattern] [--no-vary-header] [--fragment-size=size] [--compression-level=1-9] --config=config-name --vs=vs-name
CLI014 vs is a required option.

You can also try experimenting with other related CLIs
  • get-precompressed-content-prop
  • get-on-demand-compression-prop
  • disable-precompressed-content
  • disable-on-demand-compression

How do I test if my configuration is working or not

    You can use telnet to submit HTTP requests, impersonating a web browser. The stuff in bold is the stuff I typed.  Here's an example on a Solaris machine testserver, where I have a Web Server instance running on port 2600:
$ telnet testserver 2600
Trying 1.2.3.4...
Connected to testserver.
Escape character is '\^]'.
GET / HTTP/1.0
[press enter twice]

HTTP/1.1 200 OK
Server: Sun-Java-System-Web-Server/7.0
Date: Wed, 06 Feb 2008 12:09:45 GMT
Content-type: text/html
Last-modified: Fri, 25 Jan 2008 00:12:38 GMT
Content-length: 355
Connection: close

<html>...</html>
Connection to testserver closed by foreign host.
$
I'll try that same HTTP request again, but this time indicate that I want a compressed response:
$telnet testserver 2600
Trying 1.2.3.4...
Connected to testserver.
Escape character is '\^]'.
GET / HTTP/1.0
[press enter]
Accept-encoding: gzip [press enter twice]


HTTP/1.1 200 OK
Server: Sun-Java-System-Web-Server/7.0
Date: Wed, 06 Feb 2008 12:09:45 GMT
Content-type: text/html

Last-modified: Fri, 25 Jan 2008 00:12:38 GMT
Content-encoding: gzip
Vary: accept-encoding
Connection: close

MPËNÄ0...ó²õã\\å
Connection to testserver closed by foreign host.
$

When I ask for a compressed response, the server responds with a Content-encoding: gzip header and returns compressed response instead of the html.


\*\*Note that compress-file SAF is not available in Web Server 6.1, in that case you have to put pre compressed content into document root for find-compressed to work.

References

  1. Web Server software forum questions
  2. Compressing Web Content with mod_gzip and mod_deflate
  3. Best Practices for Speeding Up Your Web Site
  4. HTTP Compression
  5. Web Server 7.0 update 2 documentation about find-compressed function
  6. Web Server 7.0 update 2 documentation about http-compression function
  7. my previous blog Dynamic Compression Of Static Files

Friday Jun 09, 2006

Dynamic compression of static files in Sun Java System Web Server 7.0

Dynamic compression of static files in Sun Java System Web Server 7.0


    Check out the new cool features in Sun Java System Web Server 7.0 which can be downloaded for free from http://www.sun.com/download/index.jsp?cat=Web%20%26%20Proxy%20Servers&tab=3&subcat=Web%20Servers .
In this blog I will talk about dynamic compression of static files.

    We have also implemented caching of compressed data for static files. We have added a new service function "compress-file" which will compress static files (if the compressed file doesn't exist) and serves it from the cache if the compressed version already exists.
Lets say if I want to create .gz files on the fly for  static files, all I have to do is to modify obj.conf as shown below. Use it with find-compressed SAF. Add it before send-file function.
<Object name="default">
...
NameTrans fn="assign-name" from="\*.html" name="find-compressed"
...
Service method=(GET|HEAD|POST) type=\*~magnus-internal/\* fn=compress-file subdir=".compressed-files"
Service method=(GET|HEAD|POST) type=\*~magnus-internal/\* fn=send-file
...
</Object>

<Object name="find-compressed">
PathCheck fn="find-compressed"
</Object>
Note that subdir is a directory name only relative to the directory in which the original non-compressed file is located. "." means use the same directory as the original file to create compressed files.

Parameters this new "compress-file" SAF takes are subdir, check-age(true), vary(true), compression-level(6), min-size(256), max-size(1048576).

Let me take an example, if we have a static file in the docroot static1.txt.
# ls -al
-rwxr--r-- 1 root root 1386 Aug 1 21:16 static1.txt
Send a GET request with Accept-Encoding: gzip header, it returns the compressed content,
# telnet 0 2701
Trying 0.0.0.0...
Connected to 0.
Escape character is '\^]'.
GET /static1.txt HTTP/1.0
Accept-Encoding: gzip

HTTP/1.1 200 OK
Server: Sun-Java-System-Web-Server/7
Date: Mon, 01 Aug 2005 15:47:11 GMT
Content-length: 892
Content-type: text/plain
Vary: accept-encoding
Content-encoding: x-gzip
Last-modified: Mon, 01 Aug 2005 15:47:11 GMT
Accept-ranges: bytes
Connection: close

z`B®(¼?øKä0¢TeÑB¨÷Q$A
....
O9êÓÌ
iñÂ6ì¹­ ³Í§8z ªè)óÍ6

We can verify that the Web Server creates a .gz file in .compressed-files subdirectory.
# ls -la
drwxr-xr-x 2 webservd webservd 512 Aug 1 21:17 .compressed-files
-rwxr-xr-x 1 root root 1386 Aug 1 21:16 static1.txt
# ls -l .compressed-files/
total 2
-rw------- 1 webservd webservd 892 Aug 1 21:17 static1.txt.gz

Also note that the content length on access file is the same as the .gz file in the .compressed-files directory.
# tail ../logs/access
format=%Ses->client.ip% - %Req->vars.auth-user% [%SYSDATE%] "%Req->reqpb.clf-request%" %Req->srvhdrs.clf-status% %Req->srvhdrs.content-length%
abc.def.com - - [01/Aug/2005:21:17:11 +0530] "GET /static1.txt HTTP/1.0" 200 892
...



About

Meena Vyas

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today