Your AI Configured Your ZFS. Are You Sure That’s Right?

Over the past year, a new kind of pool has started showing up in our audits. The layout looks reasonable. The properties appear to be set following best practices. Then you run zdb and find ashift=9 on brand-new 4K drives, and when you ask who built the pool, the answer is a chatbot.

A good craftsperson doesn't blame the tools. ZFS is hard to configure from only a simple description of the workload. The vdev layout is the most important; it is what will define your IOPS ceiling. The difference between the native sector size and the pool ashift will define the level of write amplification. The sync policy governs what will be lost when the power drops out. The feature flags decide which machines can import the pool when you need to recover. Many of these parameters are fixed the moment you run zpool create. Others stay hidden until a crash or a restore.

You might get them right by investigating the system, its hardware, and having read some ZFS best practices. However, just pasting three commands from a chat window skips the investigation step and ends in tears.

What the Model Can't See

Every one of those decisions depends on facts the model never has information on:

• The physical sector size of your drives, which the drives routinely misreport
• The application's natural block size, and how much of its I/O is synchronous
• How much RAM the ARC can take before the workload starts to suffer
• The OpenZFS version on every host that imports or receives from the pool
• The RPO and RTO the business is counting on for survival

The model has your prompt. Where the prompt is silent, it fills in whatever looked typical across a decade of mailing lists, forum threads and blog posts covering Solaris, illumos, FreeBSD and Linux. A tunable removed in 2014 comes back with the same confidence as one added last year. And the model never runs zdb, never watches zpool iostat under load, and never asks what version or OS the replica is running.

What We've Seen Go Wrong

ashift=9 on 4K drives

Left to its own devices, ZFS will believe what the drive says. But the drive lies, for the sake of Windows XP of all things. The prompt you wrote didn’t specify, so the LLM assumes the drives have 512-byte sectors, so the answer it generated was:

zpool create -o ashift=9 tank raidz2 da0 da1 da2 da3 da4 da5

In reality, the drives were 512e: 512-byte logical sectors on top of 4 KiB physical ones. Every small write turned into a read-modify-write inside the drive. Random write throughput fell well short of what the hardware could do. ashift is fixed per vdev at creation, so the only fix is a new pool and a full copying of the data. This mismatch also prevents you from using vdev removal on mirror vdevs later: OpenZFS refuses to remove a top-level vdev when ashift differs across the pool.

sync=disabled as a database fix

One of the most common atrocities we see is sync=disabled. The original question was, “Why are PostgreSQL commits so slow on ZFS?” The answer generated by the AI was: zfs set sync=disabled tank/pgdata, and commit latency dropped immediately.

With sync disabled, fsync returns before the data reaches stable storage. The writes sit in memory until the next transaction group commits, every 5 seconds by default. The pool stays consistent through a crash. However, since the data is only in memory, but we told PostgreSQL it was safe, the database transactions that PgSQL already acknowledged to its clients do not survive it. After one power event, the database and the application no longer agreed on which orders existed.

The real bottleneck was sync write latency on the data vdevs, and zpool iostat -l showed it clearly. The right answer is adding a mirrored SLOG backed by power-loss-protected NVMe drives and a recordsize matched to PostgreSQL's 8K pages. With those changes, you get the performance back, without the risk, and can keep sync=standard as it should be.

zpool upgrade and the DR host

After updating the version of OpenZFS on the primary, the AI-generated maintenance procedure ended with zpool upgrade -a. That enables every new feature the release supports on every pool, and many feature flags cannot be turned back off. The DR host was still on the previous release. Now, if there is a failover, the DR host could no longer import the pool, and the developer machine normally meant to recover it was also locked out. Worst, if you boot from ZFS, if you didn’t carefully ensure your boot loader supports the new features, and has been updated on all of the hosts that might import the boot pool, you can end up rebooting into a void of errors, with no easy recovery.

What an Investigation Looks Like

None of this needs special tooling. It needs someone to look before changing anything. This is the first pass we make on a system before we touch a layout or a tunable:

Before any upgrade, set the compatibility property so zpool upgrade only enables what your oldest importing host can read:

zpool set compatibility=openzfs-2.2 tank

Then test the change where failure is cheap: a clone, a scratch pool, or the DR host during a planned drill.

ZFS Configuration Takeaways

• Treat any AI-generated ZFS commands as a draft. Check every value against your hardware and workload before you run it. Make sure it applies to the version of ZFS you are using. Make sure you have considered your entire environment, not just this single host.
• Read ashift with zdb -C before a pool goes into production. You cannot change it later.
• Keep sync=standard on anything holding data you promised to keep. Fix slow sync writes with a proper SLOG or metadata vdev and the right recordsize.
• Set compatibility on every pool that might be used on a second host, be that for HA, DR, or just recovery, and never run zpool upgrade -a without checking the other hosts first.
• Run zfs get -s local and make sure someone can explain every change it prints out.

If you want a second set of eyes on your pools, our Quarterly ZFS Health Check covers all of this, plus a timed recovery. Get in touch at klarasystems.com.