Snapping every minute

I'm not sure when ZFS was being designed it's target was going to be laptops. However on mine it is looking great. So good I upgraded my Dell this afternoon.

A short script and I have a snapshot every minute which I can keep for an hour and then the hourly snaphshots kept for a day. The daily snapshots kept for a month and the monthly snaphosts kept for a year.

It will be ineteresting to see how the disk space usage works out over the longer term. This all comes from this script:

#!/bin/ksh -p

function take_snap
{
        if zfs list -H -o name $1 >/dev/null 2>&1
        then
                zfs destroy $1
        fi
        zfs snapshot ${1}
}

case ${1:-boot} in
        "boot")
                snap=$(date '+%F-%T')
                ;;
        "minute")
                snap=minute_$(date +%M)
                ;;
        "hour")
                snap=hour_$(date +%H)
                ;;
        "day")
                snap=day_$(date +%e)
                ;;
        "month")
                snap=month_$(date +%m)
                ;;
esac

for fs in $(zfs list -H -o name -t filesystem)
do
        take_snap ${fs}@${snap}
done

And this entry in cron:

\* \* \* \* \* /export/home/local/snapshot minute
0 \* \* \* \* /export/home/local/snapshot hour
1 1 \* \* \* /export/home/local/snapshot day
2 1 1 \* \* /export/home/local/snapshot month

All those snapshots:

: sigma IA 9 $; ls ~/.zfs/snapshot/
2005-11-20-17:00:26  minute_20            minute_46
hour_21              minute_34            minute_47
hour_22              minute_35            minute_48
minute_00            minute_36            minute_49
minute_11            minute_37            minute_50
minute_12            minute_38            minute_51
minute_13            minute_39            minute_52
minute_14            minute_40            minute_53
minute_15            minute_41            minute_54
minute_16            minute_42            minute_55
minute_17            minute_43            minute_56
minute_18            minute_44            minute_57
minute_19            minute_45
: sigma IA 10 $;

listing out the snapshots it really does seem that they are free:

# zfs list | grep cjg
home/cjg              33.0M  8.52G  30.5M  /export/home/cjg
home/cjg@2005-11-20-17:00:26  30.5K      -  29.4M  -
home/cjg@hour_21       296K      -  30.6M  -
home/cjg@minute_34    42.0K      -  31.0M  -
home/cjg@minute_35    22.0K      -  31.0M  -
home/cjg@minute_36    50.5K      -  31.0M  -
home/cjg@minute_37    37.0K      -  31.0M  -
home/cjg@minute_38        0      -  31.0M  -
home/cjg@minute_39        0      -  31.0M  -
home/cjg@minute_40        0      -  31.0M  -
home/cjg@minute_41        0      -  31.0M  -
home/cjg@minute_42    6.00K      -  31.0M  -
home/cjg@minute_43    68.0K      -  31.0M  -
home/cjg@minute_44    31.5K      -  31.0M  -
home/cjg@minute_45    37.5K      -  31.0M  -
home/cjg@minute_46        0      -  31.0M  -
home/cjg@minute_47        0      -  31.0M  -
home/cjg@minute_48        0      -  31.0M  -
home/cjg@minute_49        0      -  31.0M  -
home/cjg@minute_50        0      -  30.3M  -
home/cjg@minute_51        0      -  30.3M  -
home/cjg@minute_52        0      -  30.3M  -
home/cjg@minute_53        0      -  30.3M  -
home/cjg@minute_54        0      -  30.3M  -
home/cjg@minute_55        0      -  30.3M  -
home/cjg@minute_56        0      -  30.3M  -
home/cjg@minute_57        0      -  30.3M  -
home/cjg@hour_22          0      -  30.4M  -
home/cjg@minute_00        0      -  30.4M  -
home/cjg@minute_11        0      -  30.5M  -
home/cjg@minute_12        0      -  30.5M  -
home/cjg@minute_13        0      -  30.5M  -
home/cjg@minute_14        0      -  30.5M  -
home/cjg@minute_15        0      -  30.5M  -
home/cjg@minute_16        0      -  30.5M  -
home/cjg@minute_17        0      -  30.5M  -
home/cjg@minute_18        0      -  30.5M  -
home/cjg@minute_19        0      -  30.5M  -
home/cjg@minute_20        0      -  30.5M  -
home/cjg@minute_21        0      -  30.5M  -
home/cjg@minute_22        0      -  30.5M  -
home/cjg@minute_23        0      -  30.5M  -
#

I can feel an open letter to the admin of my home directory server comming along the lines of:

I want a snapshot every minute and I want it now.

Tags:

Comments:

No need for that letter Chris. As soon as our 'new' storage arrives we will be using Nevada and ZFS.

Posted by Paul Humphreys on November 21, 2005 at 12:37 AM GMT #

I hope it arrives soon, very soon.

Posted by Chris Gerhard on November 21, 2005 at 04:23 AM GMT #

Hi Chris, thanks for the great script. I did find a bug in it though - the "date %e" should be "date %d" such that single-digit days of the month produce snapshots (i.e. the day will be "07" instead of " 7"). I'm going to include the modified script in my March 06 SysAdmin Magazine column about ZFS if that's okay...

Posted by Peter Baer Galvin on January 07, 2006 at 08:26 AM GMT #

Thanks for the bug fix. Feel free to publish the script more widely.

Posted by Chris Gerhard on January 07, 2006 at 02:20 PM GMT #

Great stuff! You don't mention having an RC script such as this one:

$ cat /etc/rc3.d/S99take.snapshot
#!/sbin/sh

# ident "@(#)take-snapshot      1.0     07/10/21 dbryant"

# Could use some more sanity checking here...
[ ! -d /usr/bin ] && exit
[ ! -d /root/bin ] && exit

# Take a snapshot at boot time

SCRIPT="/root/bin/snapshot.ksh"

case "$1" in
'start')
        if [ -x ${SCRIPT} ];
        then
           ${SCRIPT} boot
        else
           echo "$0 failed to find ${SCRIPT} at boot." | /usr/bin/mailx -s root
        fi
        ;;

\*)
        echo "Usage: $0 { start }"
        exit 1
        ;;
esac
exit 0
# EOF

Your method has a nice "automatic" retention feature. But I like the idea of using unformatted dates such as "date +%Y%m%d%H%M". Since this allows for easier incremental transfers (think zfs send -i), and "date based" math (blow away everything older than N days|hours|minutes). Keep up the good work! David

Posted by David Bryant on January 14, 2007 at 07:22 PM GMT #

RC scripts are so yesterday!;-). SMF is what you want.

I have moved on since this take a look at this blog entry http://blogs.sun.com/chrisg/entry/snapshot_on_boot

Then I have been taking the snapshots via samba and using a more descriptive name which I have carried over into the current version of the snapshot script. See http://blogs.sun.com/chrisg/entry/samba_meets_zfs.

Finally I have a clean up script that runs to delete the old snapshots: http://blogs.sun.com/chrisg/entry/a_faster_zfs_snapshot_massacre.

Posted by Chris Gerhard on January 15, 2007 at 07:49 AM GMT #

Post a Comment:
Comments are closed for this entry.
About

This is the old blog of Chris Gerhard. It has mostly moved to http://chrisgerhard.wordpress.com

Search

Archives
« March 2015
MonTueWedThuFriSatSun
      
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
     
Today