Advanced ZFS Snapshots 

Earlier in this series, we discussed the Basics of ZFS Snapshot Management: how to create OpenZFS snapshots, restore files from a snapshot, and delete snapshots. Today’s article dives a bit deeper into OpenZFS snapshot management with snapshot holds, clone creation and promotion, and assigning permissions to snapshot-related operations. 

Holding Snapshots 

On a FreeBSD 13.0 test system, I created a series of snapshots of the home directory in the default zroot pool. Here is a listing of the available snapshots:

zfs list -t snapshot 
NAME                       USED  AVAIL     REFER  MOUNTPOINT 
zroot/usr/home@snapshot1    80K      -       96K  - 
zroot/usr/home@snapshot2    72K      -     1.99M  - 
zroot/usr/home@snapshot3    64K      -     2.03M  - 
zroot/usr/home@snapshot4    64K      -     2.00M  - 
zroot/usr/home@snapshot5   512K      -      260M  - 
zroot/usr/home@snapshot6     0B      -      259M  - 

By default, as the root user, I can destroy any of these snapshots. Here I delete and verify the deletion of snapshot1: 

zfs destroy zroot/usr/home@snapshot1 
zfs list -t snapshot 
NAME                       USED  AVAIL     REFER  MOUNTPOINT 
zroot/usr/home@snapshot2    72K      -     1.99M  - 
zroot/usr/home@snapshot3    64K      -     2.03M  - 
zroot/usr/home@snapshot4    64K      -     2.00M  - 
zroot/usr/home@snapshot5   512K      -      260M  - 
zroot/usr/home@snapshot6     0B      -      259M  - 

And here goes snapshot4: 

zfs destroy zroot/usr/home@snapshot4 
zfs list -t snapshot 
NAME                       USED  AVAIL     REFER  MOUNTPOINT 
zroot/usr/home@snapshot2    72K      -     1.99M  - 
zroot/usr/home@snapshot3    72K      -     2.03M  - 
zroot/usr/home@snapshot5   520K      -      260M  - 
zroot/usr/home@snapshot6     0B      -      259M  - 

But what if I have a snapshot that I don’t want even the root user or an errant deletion script to destroy? Perhaps that snapshot represents a baseline point-in-time that I might wish to return to. Enter zfs-hold(8): Hold a snapshot to prevent it being removed with the zfs destroy command. 

To create a hold, specify a name for the hold (known as a tag) and the name of the snapshot. Here I’ll create a hold tag of keepme on snapshot3: 

zfs hold keepme zroot/usr/home@snapshot3 

You can use any value you want for the tag that reminds you why you created the hold, as long as the snapshot doesn’t already have a tag using that same value. 

To see the list of holds on a snapshot, specify the snapshot name: 

zfs holds zroot/usr/home@snapshot3 
NAME                      TAG     TIMESTAMP 
zroot/usr/home@snapshot3  keepme  Thu Aug 19 11:04 2021 

Now that this snapshot has a hold, I am unable to delete the snapshot: 

zfs destroy zroot/usr/home@snapshot3 
cannot destroy snapshot zroot/usr/home@snapshot3: dataset is busy 

If I no longer need to hold this snapshot and actually want to delete it, I’ll need to first release its hold. To do so, specify the name of the hold tag and the name of the snapshot: 

zfs release keepme zroot/usr/home@snapshot3 
zfs holds zroot/usr/home@snapshot3 
NAME                      TAG     TIMESTAMP 

Now the snapshot deletion will succeed: 

zfs destroy zroot/usr/home@snapshot3 
zfs list -t snapshot 
NAME                       USED  AVAIL     REFER  MOUNTPOINT 
zroot/usr/home@snapshot2   140K      -     1.99M  - 
zroot/usr/home@snapshot5   528K      -      260M  - 
zroot/usr/home@snapshot6     0B      -      259M  - 

Creating Clones 

A snapshot, by definition, is a read-only copy of a file system. This is a good thing for file restoration purposes as it allows you to access a file’s version from a known point in time. However, there are times when it is useful to have a read-write copy of a file system. zfsconcepts(8) defines a clone as: a writable volume or file system whose initial contents are the same as another dataset.  As with snapshots, creating a clone is nearly instantaneous, and initially consumes no additional space. 

As an example use case, consider a dataset containing a directory of virtual images that are each 2 GB in size and which are used for testing by multiple users. If each tester copies an image to their testing directory whenever they need to perform a test, the amount of consumed storage will increase by 2 GB for every test. Contrast that with instead creating a clone for each user. Since clones are created from snapshots, there is a known point in time, a definite benefit in testing. Better yet, the only storage consumed will be any edits made to the clones. This more efficient use of storage can apply to any use case where multiple users are working on large files (another example would be a directory containing videos that need to be edited for production). And, any edits to the clone are independent of the snapshot (your original baseline) as well as the “production” file system the snapshot is based on. 

It’s important to note that clones can only be created from a snapshot and that a snapshot cannot be destroyed as long as it has any clones. Interestingly, clones do not inherit the properties of the snapshot, meaning that different properties can be specified when creating the clone. The clone can be located anywhere in the ZFS hierarchy as long as it is in the same pool. 

To create a clone from a snapshot, specify the snapshot name followed by the clone location and name. In this example, I’ll verify which snapshots still exist before creating a clone of snapshot5 named clone1 and locate it on the tmp file system in the zroot pool: 

zfs list -t snapshot 
NAME                       USED  AVAIL     REFER  MOUNTPOINT 
zroot/usr/home@snapshot2   140K      -     1.99M  - 
zroot/usr/home@snapshot5   528K      -      260M  - 
zroot/usr/home@snapshot6     0B      -      259M  - 
 
zfs clone zroot/usr/home@snapshot5 zroot/tmp/clone1 
 

Since a clone is a readfully writable file system, zfs list doesn’t distinguish between file systems and a cloned snapshot: 

zfs list 
NAME                 USED  AVAIL     REFER  MOUNTPOINT 
zroot               1.17G  11.9G       96K  /zroot 
zroot/ROOT           932M  11.9G       96K  none 
zroot/ROOT/default   932M  11.9G      932M  / 
zroot/tmp             96K  11.9G       96K  /tmp 
zroot/tmp/clone1       0B  11.9G      260M  /tmp/clone1 
zroot/usr            260M  11.9G       96K  /usr 
zroot/usr/home       260M  11.9G      259M  /usr/home 
zroot/usr/ports       96K  11.9G       96K  /usr/ports 
zroot/usr/src         96K  11.9G       96K  /usr/src 
zroot/var            632K  11.9G       96K  /var 
zroot/var/audit       96K  11.9G       96K  /var/audit 
zroot/var/crash       96K  11.9G       96K  /var/crash 
zroot/var/log        152K  11.9G      152K  /var/log 
zroot/var/mail        96K  11.9G       96K  /var/mail 
zroot/var/tmp         96K  11.9G       96K  /var/tmp 

However, the origin property will indicate that it is a clone and give the value of the parent snapshot: 

zfs get origin zroot/tmp/clone1 
NAME              PROPERTY  VALUE                     SOURCE 
zroot/tmp/clone1  origin    zroot/usr/home@snapshot5  - 

Now that I have a clone, let’s see what happens if I try to delete the parent snapshot, snapshot5: 

zfs destroy zroot/usr/home@snapshot5 
cannot destroy 'zroot/usr/home@snapshot5': snapshot has dependent clones 
use '-R' to destroy the following datasets: 
zroot/tmp/clone1 

The error message clearly indicates that the dataset (the clone) needs to first be destroyed. Once I am finished with the clone, I can delete the clone and then successfully delete the parent snapshot: 

zfs destroy zroot/tmp/clone1 
zfs destroy zroot/usr/home@snapshot5 
zfs list -t snapshot 
NAME                       USED  AVAIL     REFER  MOUNTPOINT 
zroot/usr/home@snapshot2   140K      -     1.99M  - 
zroot/usr/home@snapshot6     0B      -      259M  - 

Promoting Clones 

But what if you like the changes in the clone more than the data in the original snapshot? A common use case for creating clones is to test changes, and it is desirable to push them into production once testing is complete. 

In this case you can promote the clone which reverses the parent-child dependency relationship. In other words, the clone is now a “real” file system and the original snapshot is now a clone with all the old snapshots intact. 

Let’s see this in action by creating a clone on one of the remaining snapshots: 

zfs list -t snapshot 
NAME                       USED  AVAIL     REFER  MOUNTPOINT 
zroot/usr/home@snapshot2   140K      -     1.99M  - 
zroot/usr/home@snapshot6     0B      -      259M  - 
 
zfs clone zroot/usr/home@snapshot2 zroot/tmp/promote-me 
 
zfs destroy zroot/usr/home@snapshot2 
cannot destroy 'zroot/usr/home@snapshot2': snapshot has dependent clones 
use '-R' to destroy the following datasets: 
zroot/tmp/promote-me 

So far, so good. Let’s see what happens when I promote the promote-me clone: 

zfs promote zroot/tmp/promote-me 
zfs get origin zroot/tmp/promote-me 
NAME                  PROPERTY  VALUE   SOURCE 
zroot/tmp/promote-me  origin    -       - 

As expected, promote-me is no longer dependent on snapshot2. However, this might surprise you: 

zfs list -t snapshot 
NAME                             USED  AVAIL     REFER  MOUNTPOINT 
zroot/tmp/promote-me@snapshot2     0B      -     1.99M  - 
zroot/usr/home@snapshot6           0B      -      259M  - 

snapshot2 now resides on the zroot/tmp/promote-me file system. Is there a remaining dependency? Let’s check: 

zfs get origin 
NAME                            PROPERTY  VALUE                           SOURCE 
zroot                           origin    -                               - 
zroot/ROOT                      origin    -                               - 
zroot/ROOT/default              origin    -                               - 
zroot/tmp                       origin    -                               - 
zroot/tmp/promote-me            origin    -                               - 
zroot/tmp/promote-me@snapshot2  origin    -                               - 
zroot/usr                       origin    -                               - 
zroot/usr/home                  origin    zroot/tmp/promote-me@snapshot2  - 
zroot/usr/home@snapshot6        origin    -                               - 
<snip> 
 

Now things look much better, don’t they? Enabling compression is good in most cases. The same test can be done changing the recordsize. Let's try with a small one just to see how one single setting can affect space efficiency in a dramatic way. I will disable compression to better understand the numbers:

root@geroda:/testpool # zfs set compression=off testpool 
root@geroda:/testpool # zfs set recordsize=4K testpool
root@geroda:/testpool # /root/test_recordsize.sh 
1 K size -> 3 K alloc 
2 K size -> 3 K alloc 
3 K size -> 5 K alloc 
4 K size -> 5 K alloc 
5 K size -> 13 K alloc 
6 K size -> 13 K alloc 
7 K size -> 13 K alloc 
8 K size -> 13 K alloc 
9 K size -> 17 K alloc 
23 K size -> 29 K alloc 
24 K size -> 29 K alloc 
25 K size -> 33 K alloc 
31 K size -> 37 K alloc 
129 K size -> 137 K alloc 
254 K size -> 264 K alloc 
255 K size -> 264 K alloc 
256 K size -> 264 K alloc 

And what happens if I try to delete snapshot2? 

zfs destroy zroot/tmp/promote-me@snapshot2 
cannot destroy 'zroot/tmp/promote-me@snapshot2': snapshot has dependent clones 
use '-R' to destroy the following datasets: 
zroot/usr/home@snapshot6 
zroot/usr/home 

Well, I probably don’t want to destroy zroot/usr/home... This is what zfs-promote(8) means when it says: the origin file system becomes a clone of the specified file system. 

Before creating a clone that you eventually plan on promoting to be a “real” file system, you want to give careful thought to where that file system should live. Example 10 in zfs(8) gives a real-world example of replacing a production file system with a testing clone that is now ready for running in production. Note that the example uses zfs rename to change file system names between beta, legacy, and production after performing the promote. You will want to follow a similar workflow in order to differentiate which file system is which when changing to a new production file system and eventually destroying the legacy file system. 

Assigning Permissions 

Basics of ZFS Snapshot Management gives several examples of assigning a particular user specific permissions to ZFS operations on a specified file system. To see if any non-root users or groups have been assigned permissions to ZFS operations, specify the file system to check. Here I’m checking ZFS operations permissions on the zroot pool of a default FreeBSD installation: 

zfs allow zroot 

If I only get the prompt back, no permissions have been assigned and most ZFS operations can only be performed by the root user. The following permissions are related to snapshot operations: 

• clone    
• create 
• destroy 
• diff 

• hold 
• promote 
• release 
• rename 
• snapshot 

Permissions can be granted (with allow) and revoked (with unallow) to users and groups. Keep in mind that ZFS operations are powerful and act on data, one of your most valuable assets. Be judicious when allowing operational access and periodically review permissions to determine if they are still appropriate. Refer to zfs-allow(8) for syntax guidance. 

Conclusion 

This article covered some advanced OpenZFS features that can help you get even more out of your snapshots. However, if your needs are more complicated and demand qualified services, the experts at Klara are available to help you with your OpenZFS support.