Tweaking the Cache Lookup logic
By user13334066 on Qer 27, 2006
I mean, if the cachefile does not exist, check if the cache lockfile exists - which would mean that some other thread is downloading the same URL at this very moment, and so if we wait for a certian while, the download will be over, and we can pick up the cache file.
Here is a script which does exactly that:
cache-root = "/somewhere/proxy-install/proxy-instance/cache" cache-ndirs = 16 sleeptime = 100 # # Generate the cachefile's name # cf = cachefilename $rq.reqpb.uri # # And "convert" it to the lockfile's name # lcf = $cf lcf += ".lck"
if test -f $lcf then # # The lockfile exists. What this means is, someone else is downloading # this very same file at this moment. So let's just wait for the download # to finish, and then pick up the cachefile. We save time, as well as # network usage. # oldsize = 0 while true do millisleep $sleeptime if test ! -f $lcf then # # Ok, the lockfile has disappeared. So the download which # we were waiting to end, is over. But just make sure that # the new cachefile is in place. # if test -f $cf then # # Cool - our "gamble" has payed off. Set the connect-mode # to "never" so that we pick up the cachefile without # attempting a connect to the remote server. # rq.vars.connect-mode = never fi break fi done fi # # Over to the proxy-retrieve SAF now. It will find the cachefile in place, # and since we have specified connect-mode = never, will serve the client # from the cachefile. # req noaction
This script can be inserted as a Service SAF which gets executed just before the proxy-retrieve SAF.
< Object ppath="http://.\*"> ObjectType fn="cache-enable" query-maxlen="10" log-report="off" ObjectType fn="cache-setting" lm-factor="0.10" max-uncheck="7200" Service fn="nsh" script="/someplace/scripts/cachelookup.sh" Service fn="proxy-retrieve" method="\*" < / Object>
Now say, we don't want this happening to all requests - maybe only to certian requests, say those involving huge downloads, like mp3 files. In such cases, we can just add something like this to the very beginning of the script:
if $rq.reqpb.uri != ".\*.mp3" then req noaction fi