Strange, strange Linux

Take following piece of code:
#include <stdlib.h>
#include <stdint.h>
#include <unistd.h>
#include <sys/mman.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <sys/shm.h>
#include <sys/ipc.h>
#include <fcntl.h>
#include <string.h>
#include <errno.h>

static int create_sharedmem_resources(const char\* dirname, const char\* filename, 
                                      int size) {
  char buf[1024];
  int fd;

  snprintf(buf, sizeof buf, "%s/%s", dirname, filename);
  fd = open(buf, O_RDWR|O_CREAT|O_TRUNC, S_IREAD|S_IWRITE);

  if (fd == -1) {
    perror("open");
    return -1;
  }

  if (ftruncate(fd, (off_t)size) == -1) {
    perror("ftruncate");
    return -1;
  }

  return fd;
}


int run(int size) {
  char\* mapAddress;
  int fd;

  fd = create_sharedmem_resources("/tmp", "shared", size);
  
  if (fd == -1) {
    return 1;
  }

  mapAddress = (char\*)mmap(NULL, size, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0);

  if (mapAddress == MAP_FAILED) {
    perror("mmap");
    return 1;
  }

  //memset(mapAddress, 0, size);

  getchar();

  memset(mapAddress, 0xff, size);

  return 0;
}



int main() {
  run(8192\*10000);
  return 0;
}
Then let's make small filesystem, like this:
dd if=/dev/zero of=/export/aa bs=1024 count=1000
mke2fs /export/aa
mount /export/aa  /tmp/ -o loop
Let's run this program, and what we'll see:
[root@XXX nike]# df -kh /tmp/
Filesystem            Size  Used Avail Use% Mounted on
/export/aa            979K  979K     0 100% /tmp
[root@XXX ni81036]# ls -lh /tmp/shared 
-rw------- 1 nike wheel 79M Jun  5 19:25 /tmp/shared
I'd call this miracle....

Update: In fact du -sh gives more sane results:

du -sh /tmp/shared 
962K    /tmp/shared
But what's make me really feel bad, is following:
cat /tmp/shared |wc -c
81920000
(so we can read 79M of this file) and even worse, if I replace getchar(); memset() with
  srand(239);
  for (i=0; i<size; i++) {
    mapAddress[i] = (char)rand();
  }
And do md5 on resulting file I'll get:
md5sum /tmp/shared 
f908d51422f2e1b15da9a930f51d3560   
md5sum /export/users/nike/shared 
f908d51422f2e1b15da9a930f51d3560  
In second case I stored same file on regular huge filesystem, so we managed to keep 79M of random data intact on seemingly 1M loopback disk. Moreover even if I unmount and mount back loop fs I'm still getting same file with same checksum, even although underlying loopback file disk usage is around 1M. Really interesting where data get stored.
Comments:

Neat trick. Ubuntu (Feisty) behaves the same. Must be some compression thing? About 80MB of zero's will compress fine to a lot less.

Posted by SwitchBL8 on June 05, 2007 at 03:34 PM MSD #

try "du" instead of "ls", it give the real disk space used. try "sync".

Posted by SDiZ on June 06, 2007 at 04:03 AM MSD #

Just plain sparse files :)

Posted by lumag on June 19, 2007 at 08:06 PM MSD #

Those files aren't sparse, memset(mapAddress, 0xff, size) or mapAddress[i] = (char)rand() files whole file with data.

Posted by nike on June 20, 2007 at 12:10 AM MSD #

Post a Comment:
  • HTML Syntax: NOT allowed
About

nike

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today