Development update: Archiver
By Ted Pogue on Mar 06, 2008
We are actively working on the archiver to improve SAM scalability and performance. One area we are focusing on is reducing the scanning and therefore metadata footprint of the archiver. Currently, the noscan method of the archiver gathers inode (file) events to determine whether and when an inode needs to be archived. After receiving the event, the archiver schedules a scan of the associated directory. The list of directories to be scanned become the archiver's worklist. We are working to improve archiver performance by changing the sam-arfind daemon's worklist from a list of directories requiring scanning (in order to find files requiring examination), to the actual list of file inodes requiring examination.
Keeping the actual list of inodes should significantly reduce the frequency
scans done by the archiver. In general, most inode events would require
examination within minutes or hours. However, for those longer time frame
examinations, only those files that need to be examined within a "rolling window" of time are kept in the worklist. The duration of the
"rolling window" is specified by the 'background_interval' in the
archiver.cmd file. The default might be 24 hours. Files that require
examination after this time will be found during the next background
scan. The time of day for the background scan is specified by the
'background_time' in the archiver.cmd file. The default might be
(midnight). The intent of these tunables is to allow the administrator to schedule the background scans at a frequency and time that minimizes archiver
scan impact to production.
'til next time, Ted