Monday Aug 27, 2007

File Events Notification

Recently, I added the File Events Notification(FEN) - PSARC 2007/027, facility to Solaris which can be used to monitor files and directories for changes. FEN is implemented as an event source under the Event Ports framework. This facility is available starting in Nevada build 72.

Many applications need to monitor files & directories for status change caused by non communicating processes. For example a file manager monitors the directory(folder) for changes so that it can update the list of files it displays when files are created or deleted. Some applications may need to monitor config files and reload them when they change. The current method is to stat the file or directory periodically to find out if they have changed.

The FEN facility provides a mechanism to monitor files and directories for status changes and receive event notification when the change occurs. This will serve as an appropriate and an efficient alternative to the current method where applications have to periodically stat files & directories.

There are different file system  change notification mechanisms available on other operating systems. Some of them implement event queuing in the kernel and provide additional context, like file names, along with the change event. For example when a directory is monitored, one event for each file that gets created in the directory, is delivered. The events get queued in the kernel while the application processes each of them. This exposes scalability issues.

Obviously, the rate at which events can get generated and delivered will be much faster then the rate at which the application receives and processes them. The events will have to be queued up locking down kernel memory till the application processes them. A limit will have to be imposed on the number of events that can get queued, which means events can be missed due to overflow. The application will have to implement a fall back method to handle the overflow condition. The scalability issues have been discussed on perf-discuss forum.

In the approach we have taken with FEN, it provides file change event notification in accordance with what an application can find out by stat'ing and comparing time stamps. There is no queuing of events. Once an event is delivered, the file/directory watch gets automatically de-activated(de registered). The application has to re-register the file to start watching. This behavior is intentional. It aids multi threaded programming using the file events notification API. It also helps filter out redundant events. High resolution time stamps are used to ensure that no change events get missed between the time a file is processed and it is registered to be monitored.

For example - a multi threaded program could create a pool of worker threads to process  file events. Now if file events watch is not de-activated  after event deliver, and continues to deliver events, then more then one thread could potentially collect events from one file and attempt to process the file. Then the application will have to implement synchronization mechanism, complicating the usage of this facility.

Where as, if the file events watch is de-registered after an event is delivered, one thread would collect the event and process the file without the possibility of another thread receiving an event and processing the same file. Once processing is finished the file can be re-register to activate the file events watch on it. Time stamps passed in at the time of re-registering will be used, to determine if the file has changed while it was being processed.

The FEN facility is implemented as an event source under the Event Ports framework. Hence the FEN API will be an extension to the Event Ports API.

Event Ports :

Is an unified event notification framework, that was introduced in Solaris 10. It provides API using which an application can collect event notifications from different subsystems(event sources). The current set of subsystems that provide event notification under the Event Port framework are :

  • Async I/O
  • Posix Timers
  • User events
  • Posix Message Queues
  • polling file descriptors

And now :-

  • File Events Notification(FEN)

With the Event ports architecture, there is an emphasis on aiding simpler multi threaded programming using the API.  Implementation of the event sources under the event ports framework attempt to support this. The event notification is inherently asynchronous as different function calls are required to register with an event source (port_associate() or event source specific call) and to collect events(port_get()/port_getn()), making the usage of the API flexible.

The Event ports framework is suitable for Event Driven programming or Event based programming model. The event loop will be, to wait on the event port, where events from different subsystems(event sources) are collected and call  relevant event handler routines to process the events. A pool of work threads can be created to execute this event loop, dispatching events(calling event handlers).  Having more subsystems providing event notifications under event ports framework will be helpful. We are looking at adding signal event notification to Event Ports.

You can read more about Event Ports(Called event completion framework) with some examples in this article.  Also refer Bart Smaalders's  blog entry which has a good example using event ports. Bart has been very influential in the Event Ports architecture. For API details, refer man page documentation for the Event Ports API functions - port_create(3C), port_associate(3C), port_dissociate(3C), port_get(3C), port_alert(3C), port_send(3C).

File Events Notification - API:

Event types

     Watchable events:
  •  FILE_ACCESS             /* Monitored file/directory was accessed */
  •  FILE_MODIFIED        /* Monitored file/directory was modified */
  •  FILE_ATTRIB             /* Monitored file/directory's ATTRIB was changed */
  •  FILE_NOFOLLOW     /* flag to indicate not to follow symbolic links */
     Exception events - cannot be filtered:
  •  FILE_DELETE                 /* Monitored file/directory was deleted */
  •  FILE_RENAME_TO        /* Monitored file/directory was renamed */
  •  FILE_RENAME_FROM /* Monitored file/directory was renamed */
  •  UNMOUNTED                /* Monitored file system got unmounted */
  •  MOUNTEDOVER           /* Monitored file/directory was mounted on */

Event types are defined in '<sys/port.h>'.

The File Events Notification source is identified by  'PORT_SOURCE_FILE', defined in '<sys/port.h>', when calling event port interface routines.

An event port is where the events from the different event sources are collected. The port is a file descriptor returned by port_create(3c) function call.

To start watching a file for change events, it needs to be registered with the FEN source. The port_associate(3C) call is used to register a file.

The events are collected using the port_get(3C)/port_getn(3C).

The port_dissociate(3C) call can be used to cancel a registered file watch.

Each event source defines an object type. The object type for the FEN source is

typedef struct file_obj {
        timestruc_t     fo_atime;     /* Access time got from stat() */
        timestruc_t     fo_mtime;   /* Modification time from stat() */
        timestruc_t     fo_ctime;    /* Change time from stat() */
        uintptr_t        fo_pad[3];   /* For future expansion */
        char               *fo_name;   /* Pointer to a Null terminated path name */
} file_obj_t;

Which is defined in sys/port.h

The object 'file_obj_t' is initialized with the name of the file to be monitored and the time stamps of the file collected with a stat(2) call. The object pointer is passed to port_associate(3C) to register the file to be watched. The object pointer is the handle, that can be used to cancel the file events watch registration if necessary, by passing it to port_dissociate(3C).

At the time a file is registered to be monitored, the time stamps of the file, passed in the file_obj_t, are compared with its current times tamp. If they have changed relevant events are generated. This is how it ensures that change events are not missed between the time the file is processed and it is registered to be watched.

While the file object 'file_obj_t' is in use and it gets reused to register a different file, the earlier file watch gets canceled and the new file specified will be watched.

Example :

This program takes the path name of the files to be monitored and prints the change events received. It reads path names from standard input(stdin). Any number of file names can be entered.

In another session, modify the files that are being monitored and see the change events being printed.

The program exits upon reading EOF(^D).

Note -  On NFS filesystem, file events due to operations local to the client only, are delivered. Change events occurring on the server will not be reported.
 * fen.c
 * cc  fen.c -o fen
#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <unistd.h>
#include <fcntl.h>
#include <strings.h>
#include <port.h>
#include <errno.h>
#include <pthread.h>
#include <thread.h>

struct fileinfo {
        struct file_obj fobj;
        int events;
        int port;

printevent(int event, char *pname)
        printf("%s :",pname);
        if (event & FILE_ACCESS) {
                printf(" FILE_ACCESS");
        if (event & FILE_MODIFIED) {
                printf(" FILE_MODIFIED");
        if (event & FILE_ATTRIB) {
                printf(" FILE_ATTRIB");
        if (event & FILE_DELETE) {
                printf(" FILE_DELETE");
        if (event & FILE_RENAME_TO) {
                printf(" FILE_RENAME_TO");
        if (event & FILE_RENAME_FROM) {
                printf(" FILE_RENAME_FROM");
        if (event & UNMOUNTED) {
                printf(" UNMOUNTED");
        if (event & MOUNTEDOVER) {
                printf(" MOUNTEDOVER");

 * event handler for file events source.
process_file(struct fileinfo *finf, int revents)
        struct file_obj *fobjp = &finf->fobj;
        int port = finf->port;
        struct stat sb;

        if (!(revents & FILE_EXCEPTION) &&
                stat(fobjp->fo_name, &sb) == -1) {
                fprintf(stderr, "Failed to stat file : %s - errno %d\n",
                        fobjp->fo_name, errno);

         * Add what ever processing that needs to be done
         * here. Process received events.
        if (revents) {
                printevent(revents, fobjp->fo_name);
                 * If exception, no need to re-register.
                if (revents & FILE_EXCEPTION) {
         * re-register.
        fobjp->fo_atime = sb.st_atim;
        fobjp->fo_mtime = sb.st_mtim;
        fobjp->fo_ctime = sb.st_ctim;
        if (port_associate(port, PORT_SOURCE_FILE, (uintptr_t)fobjp,
                 finf->events, (void *)finf) == -1) {
                 * Add error processing as required, file may have been
                 * deleted/moved.
                fprintf(stderr, "Failed to register file :%s - errno %d\n",
                        fobjp->fo_name, errno);

 * worker threads wait here for event.
void *
waitforevents(void *pn)
        int port = *((int *)pn);
        port_event_t pe;

        while (!port_get(port, &pe, NULL)) {
                 * Can add cases for other sources if this
                 * port is used to collect events from multiple sources.
                switch (pe.portev_source) {
                case PORT_SOURCE_FILE:
                        /* Call file events event handler */
                        process_file((struct fileinfo *)pe.portev_object,
                        perror("Event from unexpected source");
        printf("worker thread exiting\n");

#define STRLEN 256
char str[STRLEN];

main ()
        struct fileinfo *finf;
        int port;
        char *stp;
        pthread_t tid;
        if ((port = port_create()) == -1) {

         * create a worker thread to process events. Can add as many
         * worker threads as required.
        pthread_create(&tid, NULL, waitforevents, (void *)&port);

        while(1) {
                if(fgets(str, STRLEN, stdin) == NULL) {
                        /* EOF */
                if ((stp = strstr(str, " ")) != NULL ||
                        (stp = strstr(str, "\n")) != NULL) {

                finf = malloc(sizeof(struct fileinfo));
                if (finf == NULL) {
                        perror("memory alloc");

                if ((finf->fobj.fo_name = strdup(str)) == NULL) {

                 * Event types to watch.
                finf->events = FILE_ACCESS|FILE_MODIFIED|FILE_ATTRIB;
                finf->port = port;

                 * Start monitor this file.
                process_file(finf, 0);
         * close port, will de-activate all file events watches associated
         * with the port.

         * wait for threads to exit.
        while (thr_join(0, NULL, NULL) == 0)




« August 2007