Should I Upgrade to OpenZFS 2.1?

FreeBSD 13.1-RELEASE is just around the corner (expected late April, 2022), and with it comes support for the most recent stable release of OpenZFS (version 2.1.2). This means that new FreeBSD 13.1 installations will start with the latest version of OpenZFS and all of its features.

When you upgrade an existing FreeBSD installation to 13.1, the new OpenZFS features are not yet available to existing pools and zpool status will indicate “Some supported features are not enabled on the pool.” This is by design as it allows the administrator to determine when the pools are “upgraded”—the assumption is that users will first research the new features and determine if any features will cause any compatibility issues within their environment.

This write-up provides an overview of some of the new features in the OpenZFS 2.1 series. We’ll then discuss what to consider before upgrading your pools.

What’s New in OpenZFS 2.1

FreeBSD 13.0 shipped with OpenZFS 2.0.0 (see our article on Demystifying OpenZFS 2.0). You can see what has changed in OpenZFS since that version by perusing the OpenZFS Releases page.

In addition to a lot of bug fixes, improved man pages, performance enhancements, and some new zfs and zpoolcommands, the 2.1 series introduced three major new features:

• Distributed Spare RAID (dRAID): This new variant of raidz is the OpenZFS 2.1’s biggest new feature. dRAID provides integrated distributed hot spares designed for faster resilvering. Full redundancy can be restored to the pool in a fraction of the time normally required to do a full disk replacement. You can learn more about how this feature works and how to use it in this section of the  OpenZFS Documentation. Jim Salter’s Ars Technica article discusses draid’s use case and how it compares to traditional vdevs.

• Compatibility Property: This new property is for administrators that need to create portable pools and maintain pool compatibility between differing OpenZFS versions and platforms. When enabled, this property reads a configuration file containing a list of features to enable and only enables those features during pool upgrades. This property must be enabled during pool creation. Details can be found in the Compatibility feature sets section of zpool-features(7).

• InfluxDB Support: The new zpool influxdb command parses ZFS statistics into an InfluxDB time-series database. This new feature also includes integration examples for the telegraf statistics aggregator and a grafana dashboard template. See this page for usage information, examples, and the template.

In addition to these major features, OpenZFS 2.1 adds the following improvements:

• Support for memory and CPU hotplugging: This means that in virtualized environments, you can now adjust the resource allocation of a VM without rebooting it.
• Optimized prefetch for parallel workloads: Previously, storage servers which needed to handle deep client request queues from their clients had to either serialize consequential reads or execute incoming requests as-is with almost no prefetch from ZFS. This improvement should result in much better iSCSI throughput with almost 100% prefetch hit rate.
• Reduced pool import time: Parallelization of the taskqueue reduces the time waiting for serialized disk I/O. Additionally, if the system contains zvols and the  vfs.zfs.vol.recursive sysctl is set to 0, the import will bypass zvols, which can greatly reduce import time.
• Reduced fragmentation from ZIL blocks: ZFS now allocates one embedded slog metaslab from each top-level vdev to be preferentially used for ZIL allocations. From an allocation perspective, this is similar to having a dedicated log device.  The space required for the embedded slog metaslabs is usually between 0.5% and 1.0% of the pool, and comes out of the existing 3.2% of "slop" space that is not available for user data.
• Improved zfs receive performance with lightweight write: This change improves performance when receiving streams with small compressed block sizes.

Considerations Before Upgrading Pools

It’s great to have access to the latest OpenZFS features, but there are a few things to consider before upgrading an OpenZFS pool. Some of these considerations apply to any pool upgrade, and some are specific to upgrading to the OpenZFS 2.1 series.

For OpenZFS 2.1 in particular, there are operating system minimums:

• FreeBSD 12.2 (or higher)
• FreeBSD 13.0 (or higher)

In a mixed environment with Linux systems, the compatible kernel versions are: 3.10 - 5.13

Meeting the minimum operating system or kernel requirements answers the question “Can I upgrade to OpenZFS 2.1?”.

To answer the question “Should I upgrade to OpenZFS 2.1?”, keep in mind that once you upgrade the pool, you can’t downgrade it to an earlier pool version. Or, to put that another way: you can’t use an upgraded pool on an operating system kernel version that doesn’t understand the features provided by that version of the upgraded pool. This is primarily an issue for administrators who might want to change the operating system beneath an existing pool, whether by installing a new OS or by physically pulling the ZFS disks and putting them in a different machine with a different OS already installed.

Upgrading an OpenZFS pool makes the OpenZFS features already built into the operating system’s kernel available for use on the pool. This means that the risk to consider is whether or not this will cause any compatibility issues with other systems that may need to import that pool.

This is where you need to consider your systems:

• Do you have systems running older operating system or kernel versions? If so, do you foresee a need to import an upgraded pool on any of those systems before their operating system or kernel version is updated?
• Do you have boot environments that you still need to boot into that were created from an earlier version of the operating system? If so, you probably want to wait on upgrading the pool until those boot environments are no longer needed.
• Do you foresee having to import the pool on any FreeBSD systems with an old boot loader? Consider that you would need to first update the operating system and bootloader before you could boot into a pool with a newer OpenZFS version.

In an ideal world, all of your systems would run the same operating system version and the latest OpenZFS version supported by that operating system. Since this is often not the reality, knowing your OS versions and which systems are integral to serving the data in your pools is important.

Going forward, the new OpenZFS compatibility property may be useful in mixed environments that require pool portability. In this case, you will want to consider which features apply to all of your systems requiring portability so that you can design your configuration.

More Information

You can always determine which features are available on your current FreeBSD version by running zpool upgrade -v | more. To learn more about these features, read man zpool-features.

To determine the current OpenZFS version, run zfs version.

zpool status will include this message in its output whenever a newer OpenZFS version is available:

• status: Some supported features are not enabled on the pool. The pool can still be used, but some features are unavailable.

• action: Enable all features using 'zpool upgrade'. Once this is done, the pool may no longer be accessible by software that does not suppor the features. See zpool-features(5) for details.

Once you are ready to upgrade your pools, run zpool upgrade on the pool you wish to upgrade, or use -a to upgrade all pools on the system. The output from the command will indicate which new features were enabled on the specified pools.