Oracle Linux kernel developer Rahul Yadav kicked off a new project in LXC this year, called libresource. In this blog post, he talks about how to use libresource to effectively read system statistics in a stable manner. This project is hosted on github at https://github.com/lxc/libresource
System resource information, like memory, network and device statistics, are crucial for system administrators to understand the inner workings of their systems, and are increasingly being used by applications to fine tune performance on different environments.
Getting system resource information on Linux is not a straightforward affair. Many tools like top, free and sar can gather system statistics. The best way is to collect the information from procfs or sysfs, but getting such information from procfs or sysfs presents many challenges. Each time an application wants to get a system resource information, it has to open a file, read the content and then parse the content to get actual information. Over time, the format in which information is provided might change and with that each application has to change its own code to read the data in the correct manner. Libresource tries to fix few of these problems by providing a standard library with set of APIs through which we can get system resource information e.g. memory, CPU, stat, networking, security related information. Find libresource on github at https://github.com/lxc/libresource
Libresource provides following benefits:
- Ease of use: Currently applications needs to read this info mostly from /proc and /sys file-systems. In most of the cases complex string parsing is involved which is needed to be done in application code. With the library APIs application can get the information directly and all the string parsing, if any, will be done by library.
- Stability: If the format in which the information is provided in /proc or /sys file-system is changed then the application code is changed to align with those changes. Also if a better way to get information comes in future, like through a syscall or a sysconf then again application code needs to be changed to get the benefit of it. Library will take care of such changes and the application will never have to change the code.
- Virtualization: In cases where DB is running in a virtualized environment using cgroup or namespaces, reading from /proc and /sys file-systems might not give correct information as these are not cgroup aware. Library API will take care of this e.g. if a process is running in a cgroup then library should provide information which is local to that cgroup.
Interfaces to libresource
Reading a single resource ID
/* This is to read a resource information. A valid resource id should be * provided in res_id, out should be properly allocated on the basis of * size of resource information, hint should be given where needed. * Currently pid and flags are not used, they are for future extensions. */ int resread(int resid, void out, void hint, int pid, int flags); /* Available Resource IDs */ RES_MEM_ACTIVE Total amount of buffer or page cache memory, in kilobytes, that is in active use. RES_MEM_INACTIVE Total amount of buffer or page cache memory, in kilobytes, that are free and available RES_MEM_AVAILABLE An estimate of how much memory is available for starting new applications, without swapping. RES_MEM_FREE The amount of physical RAM, in kilobytes, left unused by the system. RES_MEM_TOTAL Total amount of physical RAM, in kilobytes. RES_MEM_PAGESIZE Size of a page in bytes RES_MEM_SWAPFREE Total amount of swap free, in kilobytes. RES_MEM_SWAPTOTAL The total amount of swap available, in kilobytes. RES_KERN_COMPILE_TIME Kernel compile time RES_KERN_RELEASE Kernel version RES_NET_ALLIFSTAT Network stat for all interfaces on system. RES_NET_IFSTAT Network stat for an interface RES_MEM_INFOALL All Memory related information
Reading multiple resources in one call
If an application wants to read multiple resource information in one call, it can call res_*_blk APIs to do so which are described below.#define RES_UNIT_OUT_SIZE 256
/* This union is used to return resource information of various types */
union r_data {
int i;
size_t sz;
long l;
char str[RES_UNIT_OUT_SIZE];
void *ptr;
};
/* In case of res_read_blk, each resource information will be represented by
* following structure.
*/
typedef struct res_unit {
int status;
unsigned int res_id;
void *hint;
union r_data data;
} res_unit_t;
/* In case of bulk read (res_read_blk), this structure will hold all required
* information needed to do so.
*/
typedef struct res_blk {
int res_count;
res_unit_t *res_unit[0];
} res_blk_t;
/* It allocates memory for resources and initiates them properly. * res_ids holds an array of valid resource ids and res_count holds * number of resource ids. It also initializes struct fields properly. */ extern res_blk_t *res_build_blk(int *res_ids, int res_count); /* Reading bulk resource information. Memory must be properly allocated and * all fields should be properly filled to return error free resource * information. res_build_blk call is suggested to allocate build res_blk_t * structure. */ extern int res_read_blk(res_blk_t *resblk, int pid, int flags); /* Free allocated memory from res_build_blk */ extern void res_destroy_blk(res_blk_t *resblk);
Some Examples
Reading total memory
size_t stemp = 0;
res_read(RES_MEM_TOTAL,&stemp,NULL, 0, 0);
printf("MEMTOTAL is: %zu\n", stemp);
Reading network interface related statistics for interface named “lo”
res_net_ifstat_t ifstat;
res_read(RES_NET_IFSTAT,&ifstat, (void *)"lo",0, 0);
printf("status for %s: %llu %llu\n", ifstat.ifname,
ifstat.rx_bytes,
ifstat.rx_packets
);
Reading multiple resources in one call
res_blk_t *b = NULL;
int a[NUM] = {RES_MEM_PAGESIZE,
RES_MEM_TOTAL,
RES_MEM_AVAILABLE,
RES_MEM_INFOALL,
RES_KERN_RELEASE,
RES_NET_IFSTAT,
RES_NET_ALLIFSTAT,
RES_KERN_COMPILE_TIME
};
b = res_build_blk(a, NUM);
b->res_unit[5]->hint = (void *)"lo";
res_read_blk(b, 0, 0);
printf("pagesize %ld bytes,\n memtotal %ld kb,\n memavailable %ld kb,\n"
" memfree %ld kb,\n release %s,\n compile time %s\n",
b->res_unit[0]->data.sz,
b->res_unit[1]->data.sz,
b->res_unit[2]->data.sz,
((res_mem_infoall_t *)(b->res_unit[3]->data.ptr))->memfree,
b->res_unit[4]->data.str,
b->res_unit[7]->data.str
);
res_net_ifstat_t *ip = (res_net_ifstat_t *)b->res_unit[5]->data.ptr;
printf("stat for interface %s: %llu %llu\n", ip->ifname,
ip->rx_bytes,
ip->rx_packets
);
int k = (int)(long long)b->res_unit[6]->hint;
res_net_ifstat_t *ipp = (res_net_ifstat_t *)b->res_unit[6]->data.ptr;
for (int j=0; j< k; j++) {
printf("stat for interface %s: %llu %llu\n", ipp[j].ifname,
ipp[j].rx_bytes,
ipp[j].rx_packets
);
}
free(ipp);
res_destroy_blk(b);
res_blk_t *b = NULL;
int a[NUM] = {RES_MEM_PAGESIZE,
RES_MEM_TOTAL,
RES_MEM_AVAILABLE,
RES_MEM_INFOALL,
RES_KERN_RELEASE,
RES_NET_IFSTAT,
RES_NET_ALLIFSTAT,
RES_KERN_COMPILE_TIME
};
b = res_build_blk(a, NUM);
b->res_unit[5]->hint = (void *)"lo";
res_read_blk(b, 0, 0);
printf("pagesize %ld bytes,\n memtotal %ld kb,\n memavailable %ld kb,\n"
" memfree %ld kb,\n release %s,\n compile time %s\n",
b->res_unit[0]->data.sz,
b->res_unit[1]->data.sz,
b->res_unit[2]->data.sz,
((res_mem_infoall_t *)(b->res_unit[3]->data.ptr))->memfree,
b->res_unit[4]->data.str,
b->res_unit[7]->data.str
);
res_net_ifstat_t *ip = (res_net_ifstat_t *)b->res_unit[5]->data.ptr;
printf("stat for interface %s: %llu %llu\n", ip->ifname,
ip->rx_bytes,
ip->rx_packets
);
int k = (int)(long long)b->res_unit[6]->hint;
res_net_ifstat_t *ipp = (res_net_ifstat_t *)b->res_unit[6]->data.ptr;
for (int j=0; j< k; j++) {
printf("stat for interface %s: %llu %llu\n", ipp[j].ifname,
ipp[j].rx_bytes,
ipp[j].rx_packets
);
}
free(ipp);
res_destroy_blk(b);