Development update: Archiver

We are actively working on the archiver to improve SAM scalability and performance. One area we are focusing on is reducing the scanning and therefore metadata footprint of the archiver. Currently, the noscan method of the archiver gathers inode (file) events to determine whether and when an inode needs to be archived. After receiving the event, the archiver schedules a scan of the associated directory. The list of directories to be scanned become the archiver's worklist. We are working to improve archiver performance by changing the sam-arfind daemon's worklist from a list of directories requiring scanning (in order to find files requiring examination), to the actual list of file inodes requiring examination.


Keeping the actual list of inodes should significantly reduce the frequency scans done by the archiver. In general, most inode events would require examination within minutes or hours. However, for those longer time frame examinations, only those files that need to be examined within a "rolling window" of time are kept in the worklist. The duration of the "rolling window" is specified by the 'background_interval' in the archiver.cmd file. The default might be 24 hours. Files that require examination after this time will be found during the next background scan. The time of day for the background scan is specified by the 'background_time' in the archiver.cmd file. The default might be (midnight). The intent of these tunables is to allow the administrator to schedule the background scans at a frequency and time that minimizes archiver scan impact to production.

'til next time, Ted

Comments:

Post a Comment:
  • HTML Syntax: NOT allowed
About

Ted Pogue

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today