Fragmentation occurs when a file is stored on disk in a non-contiguous manner. This can have several causes and consequences for filesystem performance. Filesystem fragmentation can arise from various factors:
- Frequent Creation and Deletion of Small Files: When small files are created and deleted frequently, the free space on the disk becomes scattered across many locations. This dispersed free space can cause new or large files to be stored in non-contiguous chunks.
- High Levels of File Modification and Growth: As files are modified and grow, their data may not fit into the originally allocated space. This can lead to fragmentation as the file’s data is spread across different parts of the disk.
- Copy-on-Write Operations: Features like reflink, which capture a snapshot of a file by marking its extents (a contiguous block of storage space allocated for a file – see extent allocation in xfs for more details) as shared, can contribute to fragmentation. When a file is reflinked and its contents are modified, the extents can be split or altered, which may further fragment the filesystem.
When a filesystem becomes fragmented, it means that the available or allocated space is divided into many small, non-contiguous chunks. This can impact performance in several ways such as slow access time and increased reflink creation time.
Files often start small and may grow or shrink over time, with various file operations occurring in between. As a result, the file data is not always physically contiguous. It is possible to have a large file with few extents and a small file with many extents. The example below illustrates this: it shows a large file with few extents and a small file with many extents. A small file may have many extents if free space is scattered in different locations on the disk.
# ls -lh -rw-r--r--. 1 root root 20G Jun 14 06:20 file1 -rw-r--r--. 1 root root 4G Aug 6 10:06 file2 # xfs_bmap file1 | wc -l 359 # xfs_bmap file2 | wc -l 261588
Here is an illustration to help visualize how a file’s contents can be spread across the filesystem. While this may not accurately reflect the actual extent allocation process, but it helps to illustrate fragmentation. Some files may span multiple extents scattered across the disk, leading to fragmentation.
How to check your XFS filesystem’s Fragmentation Rate
XFS includes a utility called xfs_db for monitoring and editing the XFS filesystem. You can use the xfs_db command with the frag option to get a metric on the fragmentation rate. Note that this tool calculates fragmentation based on the number of files and extents, providing an average percentage. However, this metric may not accurately reflect the actual fragmentation of your filesystem in practice. But, it is useful for determining whether your filesystem might need defragmentation. More details on defragmentation will be covered later in the blog.
Consider the following example: Suppose you have a 2 GB filesystem with two files, each 782 MB in size, and these files have 4 and 5 extents respectively, the extents are quite large. It would be reasonable to assume that the filesystem is in good shape. However, using xfs_db, it reports a fragmentation factor of 60.00%, accompanied by a warning message: “Note, this number is largely meaningless,” this may be misleading. In this case, the large size of the extents means that I/O operations should still be efficient, despite the high fragmentation factor reported.
# ls -lhs total 1.6G 782M -rw-r--r--. 1 root root 782M May 27 06:50 file1 782M -rw-r--r--. 1 root root 782M May 27 06:51 file2 # xfs_bmap -pvvvl file1 file1: EXT: FILE-OFFSET BLOCK-RANGE AG AG-OFFSET TOTAL FLAGS 0: [0..399999]: 192..400191 0 (192..400191) 400000 000000 1: [400000..799999]: 1048656..1448655 1 (80..400079) 400000 000000 2: [800000..1199999]: 2302032..2702031 2 (204880..604879) 400000 000000 3: [1200000..1599999]: 3145808..3545807 3 (80..400079) 400000 000000 # xfs_bmap -pvvvl file2 file2: EXT: FILE-OFFSET BLOCK-RANGE AG AG-OFFSET TOTAL FLAGS 0: [0..248383]: 800192..1048575 0 (800192..1048575) 248384 000000 1: [248384..248431]: 80..127 0 (80..127) 48 000000 2: [248432..641095]: 400192..792855 0 (400192..792855) 392664 000000 3: [641096..1282255]: 1448656..2089815 1 (400080..1041239) 641160 000000 4: [1282256..1599999]: 2702032..3019775 2 (604880..922623) 317744 000000 # xfs_db -r -c frag filesystem.img actual 5, ideal 2, fragmentation factor 60.00% Note, this number is largely meaningless. Files on this filesystem average 2.50 extents per file
Consider another scenario where the filesystem becomes fragmented due to the creation of a large number of small files. After deleting some of these small files, you create two new files, file1 and file2, each 235 MB in size. Each of these files ends up with around 50,000 extents, with most extents spanning only a single block. This indicates that the filesystem is highly fragmented.
Despite this fragmentation, the frag option from xfs_db may not accurately reflect the situation. The presence of a large number of small files can skew the calculation. As a result, the fragmentation factor reported by the frag option might be as low as 0.37%, and the average number of extents per file could be reported as 1. This discrepancy highlights that the frag options’s metric may not fully capture the extent of fragmentation in all cases.
# ls -lhs total 486M 236M -rw-r--r--. 1 root root 235M May 27 07:35 file1 236M -rw-r--r--. 1 root root 235M May 27 07:35 file2 16M drwxr-xr-x. 2 root root 12M May 27 07:34 small-files # ls -l small-files/ | wc -l 284203 # xfs_bmap file1 | wc -l 51721 # xfs_bmap file2 | wc -l 57517 # xfs_db -r -c frag v2-filesystem.img actual 285410, ideal 284342, fragmentation factor 0.37% Note, this number is largely meaningless. Files on this filesystem average 1.00 extents per file
When to defragment
During defragmentation, files are rearranged to improve access efficiency. The xfs_fsr tool performs defragmentation by making a copy of the original file and then comparing the number of extents in the new file to those in the original. If the new file has fewer extents, xfs_fsr replaces the original file’s extent map with that of the copy, freeing the extents from the old map. Since xfs_fsr creates a complete copy of the file, it may not be effective if free space is highly fragmented.
To assess whether there is sufficient free space for defragmentation, use xfs_db or xfs_spaceman (note that xfs_db operates on the device file, while xfs_spaceman runs on the mount point). Reviewing this data will help you determine if there is enough contiguous free space to successfully perform defragmentation. Below is an example of a filesystem with a high count of small blocks and a low count of large blocks. In this case, the filesystem has fragmented free space, and defragmentation might not yield the expected results.
# xfs_db -r -c freesp v2-filesystem.img
from to extents blocks pct
1 1 45012 45012 27.54
2 3 55 127 0.08
4 7 14 76 0.05
8 15 65 569 0.35
16 31 17 395 0.24
32 63 3 157 0.10
64 127 23 2380 1.46
128 255 20 3938 2.41
256 511 5 1511 0.92
512 1023 13 9430 5.77
1024 2047 15 24409 14.94
2048 4095 13 39534 24.19
4096 8191 5 26725 16.35
8192 16383 1 9167 5.61
Defragmenting your filesystem
Fortunately, xfs_fsr can help defragment your filesystem. The effectiveness of this tool depends on the available space on your machine. xfs_fsr works by creating a temporary copy of a file and then swapping the extents with those of the original file. It can be used to defragment individual files as well as entire filesystems. By default, xfs_fsr runs for up to 2 hours and saves progress to a temporary file in /var/tmp. If the tool is run again, it can resume from where it left off.
However, xfs_fsr has some limitations: – If free space is fragmented, xfs_fsr may not be effective. If it finds that the number of extents in the target is more than the source, xfs_fsr aborts with the message “No improvement will be made”. – If the file being defragmented is modified during the process, the defragmentation will be aborted.
Note that you cannot defragment holes or unwritten extents. In cases where real extents are between holes or unwritten extents, defragmentation is not very helpful.
After freeing up some space and running this tool on the second example, we observed the following results. After using the defragmentation tool, we observed a significant reduction in the number of extents for both file1 and file2. With repeated runs, there is a possibility that the tool could further reduce the number of extents.
# xfs_fsr tmp_mnt/ /tmp_mnt start inode=0 # xfs_bmap file1 | wc -l 20 # xfs_bmap file2 | wc -l 134
Conclusion
In real-life filesystems, fragmentation often results from uneven file growth, creation, and deletion activities. This fragmentation can cause files to be scattered across different areas of the disk, which may impact performance, as accessing or modifying these files could require multiple disk operations. To optimize performance, it is beneficial to reduce fragmentation by consolidating file extents and keeping related data together. Regular maintenance and defragmentation can help ensure that files remain contiguous, maximize storage efficiency, and enhance overall system performance and reliability.