Thursday Sep 29, 2011

btrfs compression

Another day another btrfs feature :) Compression

btrfs has built-in compression. (support for both lzo and zlib) This allows you to automatically compress data for the entire filesystem, subvolumes or even down to individual files. A mount option is all that's needed to specify which compression type to use: compress=lzo,zlib.

To try this out, I took a simple setup where I used a linux kernel tree as a testcase. Here are the results:

mount the volume on /mnt and untar a linux-2.6.32.tar.bz2 tree onto /mnt
# btrfs filesystem df /mnt
Data, RAID1: total=1.00GB, used=369.81MB
Data: total=8.00MB, used=0.00
System, RAID1: total=8.00MB, used=4.00KB
System: total=4.00MB, used=0.00
Metadata, RAID1: total=1.00GB, used=57.75MB
Metadata: total=8.00MB, used=0.00
Now, lets do this again,
# mount -o compress=lzo /dev/sdf /mnt

# btrfs filesystem df /mnt
Data, RAID1: total=1.00GB, used=188.95MB
Data: total=8.00MB, used=0.00
System, RAID1: total=8.00MB, used=4.00KB
System: total=4.00MB, used=0.00
Metadata, RAID1: total=1.00GB, used=41.37MB
Metadata: total=8.00MB, used=0.00
So, 188.95M instead of 369.81M - pretty darned cool... go lzo. but.. what about zlib!
# mount -o compress=zlib /dev/sdf /mnt

# btrfs file df /mnt
Data, RAID1: total=1.00GB, used=128.75MB
Data: total=8.00MB, used=0.00
System, RAID1: total=8.00MB, used=4.00KB
System: total=4.00MB, used=0.00
Metadata, RAID1: total=1.00GB, used=35.45MB
Metadata: total=8.00MB, used=0.00
Down to 128M! Of course from a filesystem point of view you don't have to do anything, just pass a mount option or set the attribute. As I mentioned earlier, btrfs lets you do this at a file level, a directory level (and inherit down for that directory) or all the way to the top level.

More cool bits : The compression uses kernel threads and will make use of as many threads as there are cpus. So compression gets loadbalanced/spread out across all threads in a server even if it's a single big file, we will split up the big files into 128kb chunks and compress in parallel.
Another cool bit : if you have an existing uncompressed filesystem, and want to compress it, or even just compress a file on it, you can do that with btrfs filesystem defragment. The defragment command has an option -c that lets you specify zlib or lzo.
About

Wim Coekaerts is the Senior Vice President of Linux and Virtualization Engineering for Oracle. He is responsible for Oracle's complete desktop to data center virtualization product line and the Oracle Linux support program.

You can follow him on Twitter at @wimcoekaerts

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
9
10
11
12
13
14
15
16
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today