Managing Boot Environments

Managing Boot Environments

In Basics of ZFS Snapshot Management, we demonstrated how snapshots and clones can be used to access data from a given point in time. In this article, we’ll learn how ZFS boot environments (BEs) are used to boot the operating system itself into a previous point in time.

A ZFS boot environment is a bootable clone of the datasets needed to boot the operating system. Creating a BE before performing an upgrade provides a low-cost safeguard: if there is a problem with the update, the system can be rebooted back to the point in time before the upgrade.

The FreeBSD boot loader was rewritten for 12.0 to add BE support. Additionally, the default ZFS layout in the FreeBSD installer understands BEs.

This article demonstrates how to use the bectl utility to manage BEs and provides examples on how to update packages, apply security patches, and upgrade the operating system using BEs.

Using bectl(8)

Prior to FreeBSD 12.0, the sysutils/beadm port was available for managing BEs. This shell script was rewritten in C and included with the operating system as bectl beginning withFreeBSD 12.0. If you’ve used beadm in the past, the options are the same in bectlbectl adds a few more options (such as jail support) which we won’t cover today.

This command lists available BEs. Here is an example from a freshly installed 12.2 system:

bectl list
BE 		Active 	Mountpoint 	Space 	Created
default 	NR 		/ 			906M 		2021-06-10 04:29

By default, there is one BE named default. The N in the Active column indicates that this BE is active now; in other words, this is the currently booted BE. The R means that this BE is automatically selected on reboot, unless the boot process is interrupted.

The create option creates a new BE, which is a clone of the currently booted BE:

bectl create test
bectl list
BE 		Active 	Mountpoint 	Space 	Created
default 	NR 		/ 			906M 		2021-06-10 04:29
test 		- 		- 			8K 		2021-06-10 05:11

The test BE has no Active letters as it is not the currently booted BE and it isn’t configured to be the default on next boot.

If I reboot this system, the boot menu now has a new 8. Boot Environments entry:

Press 8 or e to enter the BE sub-menu:

Press 2 as often as needed to toggle through the list of BEs. Once you find the one you want to boot into, press a or enterto boot into the selected BE. Here I’ve selected the test BE:

Once booted, the listing changes to:

bectl list
BE 		Active 	Mountpoint 	Space 	Created
default 	R 		/ 			906M 		2021-06-10 04:29
test 		N 		- 			404K 		2021-06-10 05:11

Since the R is still on the default BE, the system will continue to boot into the default BE unless I interrupt the boot menu. To permanently set the test BE as the new default, use the activate switch:

bectl activate test
Successfully activated boot environment test
bectl list
BE 		Active 	Mountpoint 	Space 	Created
default 	- 		- 			424K 		2021-06-10 04:29
test 		NR 		/ 			906M 		2021-06-10 05:11

Did you know?

Getting your ZFS infrastructure up to date has never been easier!

Our team provides consistent, expert advice tailored to your business.

Example: Updating Packages in BE

In this example, I have a 12.2 laptop with a lot of packages installed and it’s been a while since I’ve updated them. I’ll start by creating a BE with a useful name:

bectl create pkg_update_`date +%y%m%d`

If I mount that BE and perform the package update within that BE, I can reboot into the new BE to test the updated packages. To mount the BE, specify its name:

bectl list | grep pkg
pkg_update_210610 	-	 -	 496K	 	2021-06-10 05:11
bectl mount pkg_update_210610
Successfully mounted pkg_update_210610 at /tmp/be_mount.EFae

Make note of the mount point (tmp/be_mount.EFae) and specify it to the pkg command to update the repos and perform the pkg upgrade:

pkg -r /tmp/be_mount.EFae update -f
Updating FreeBSD repository catalogue...
FreeBSD repository update completed. 30504 packages processed.
All repositories are up to date.

pkg -r /tmp/be_mount.EFae  upgrade
Checking for upgrades (626 candidates): 100%
Number of packages to be installed: 44
Number of packages to be upgraded: 394
Number of packages to be reinstalled: 37
The process will require 179 MiB more space.
1 GiB to be downloaded.
Proceed with this action? [y/N]: y

Once the upgrade is complete, boot into that BE to test the packages. Once you’re satisfied everything is working as expected, you can activate that BE.

Example: Deploying a Security Patch

The same process can be used to patch the operating system. In this example, I’ll update the default FreeBSD 12.2 installation to the latest patch level. Start by creating a BE, mounting it, and noting the name of the mount point:

bectl create security_update_`date +%y%m%d`
bectl list |grep security
security_update_210610 	-	 -	 8K	 2021-06-10 06:15
bectl mount security_update_210610
Successfully mounted security_update_210610 at /tmp/be_mount.7pMM

Next, ensure that freebsd-update doesn’t use its default database in the currently booted BE by removing that directory:

rm -Rf /var/db/freebsd-update

Run the freebsd-update utility and specify the BE mount point as the basedir (-b) and the BE’s database as the workdir (-d). First, fetch the updates:

freebsd-update -b /tmp/be_mount.7pMM -d /tmp/be_mount.7pMM/var/db/freebsd-update fetch
<output snipped>
The following files will be removed as part of updating to
<SNIP> (use page down to read through and q to quit)

Next, repeat that command with install:

freebsd-update -b /tmp/be_mount.7pMM -d /tmp/be_mount.7pMM/var/db/freebsd-update install
<output snipped>

Before rebooting into the patched BE, let’s verify the version of the currently booted BE:

freebsd-version -u

Repeat that version check after rebooting into the security_update_210610 BE:

freebsd-version -u

Once you’ve tested the BE, you can activate it.

Did you know?

Want to learn more about ZFS? We consistently write about the awesome powers of OpenZFS in our article series.

Example: Testing an OS Upgrade

For this example, I’ll test updating the patched 12.2-RELEASE-p8 BE to 13.0. Again, create a new BE and mount it:

bectl create 13.0
bectl mount 13.0
Successfully mounted 13.0 at /tmp/be_mount.JO5Y

Specify the release version (-r 13.0-RELEASE) to upgrade toin the freebsd-update command:

freebsd-update -b /tmp/be_mount.JO5Y -d /tmp/be_mount.JO5Y/var/db/freebsd-update -r 13.0-RELEASE upgrade
<output snipped)
The following file will be removed, as it no longer exists in
FreeBSD 13.0-RELEASE: /etc/motd
Does this look reasonable (y/n)? y
<snip> (page down through list of files and q to quit)
To install the downloaded upgrades, run "/usr/sbin/freebsd-update install".

Next, repeat the command, this time with install:

freebsd-update -b /tmp/be_mount.JO5Y -d /tmp/be_mount.JO5Y/var/db/freebsd-update -r 13.0-RELEASE install
Installing updates...
Kernel updates have been installed. Please reboot and run
"/usr/sbin/freebsd-update install" again to finish installing updates.

Using the ZFS bootonce feature we developed at Klara, you can instruct the system to boot a different boot environment, one time only, with the -t (temporary) flag.

bectl activate -t 13.0

Reboot and the 13.0 BE will be loaded, and the temporary flag will be cleared. Since you are now rebooted into that BE, you don’t have to mount it and can just issue the install command to complete the installation process:

freebsd-update install

Once you are finished testing the upgrade, you can activate the BE.

bectl activate 13.0

If the upgrade isn’t working, or you don’t activate the 13.0 BE, rebooting will return to the previous default BE.

If Something Goes Wrong

Most OS upgrades “just work”. Occasionally you will hit a gotcha that requires configuration file edits after booting into the new OS version. Once these are resolved, you can activate the BE.

Sometimes you get the dreaded mountroot> prompt that halts the OS boot process. In this case, simply boot into a good BE, then mount the problematic BE, noting its mount point. As you research which update gotcha bit you, you can make any modifications within the mount point. For example, if the BE mount point is /tmp/be_mount.ON3E and you need to edit the BE’s fstab file, make your edits to /tmp/be_mount.ON3E/etc/fstab

<strong>Meet the author:</strong> Dru Lavigne
Meet the author: Dru Lavigne

Dru Lavigne is a retired network and systems administrator, IT instructor, author, and international speaker. Dru is author of BSD Hacks, The Best of FreeBSD Basics, and The Definitive Guide to PC-BSD.

Like this article? Share it!

You might also be interested in

Getting expert FreeBSD advice is as easy as reaching out to us!

At Klara, we have an entire team dedicated to helping you with your FreeBSD projects. Whether you’re planning a FreeBSD project, or are in the middle of one and need a bit of extra insight, we’re here to help!

More on this topic

FreeBSD or Linux – A Choice Without OS Wars

Uncover the key differences between FreeBSD and Linux as we break down their features and use cases, helping users make well-informed decisions based on their specific requirements.

FreeBSD History – Understanding the Origins of DTrace

DTraceis a powerful tool for system administrators to diagnosis system issues without unduly impacting performance. DTrace became part of FreeBSD with the release of FreeBSD 7.1 in 2009—two years before Oracle began porting DTrace, and nine years before Oracle eventually solved the inherent CDDL vs GPL license conflict.

A Quick Look at the History of Package Management on FreeBSD

Pkgng became FreeBSD’s official package manager in FreeBSD 10 in 2014. Applications can be easily installed from either pkg—a system managing precompiled binary packages—or the ports tree, which automates building and installation of packages directly from their source code.

6 Comments on “Managing Boot Environments

  1. Pingback: Valuable News – 2021/07/19 | 𝚟𝚎𝚛𝚖𝚊𝚍𝚎𝚗

  2. I did not know this tool and how to use it, in this article I have learned how to do it. Another reason to continue with FreeBSD – recovering a system after a failed upgrade. Thank you. Great job.

  3. Pingback: Manipulating a Pool from the Rescue System | Klara Inc.

  4. Pingback: Evaluating FreeBSD CURRENT for Production Use | Klara Inc.

  5. Pingback: Valuable News – 2022/06/06 | 𝚟𝚎𝚛𝚖𝚊𝚍𝚎𝚗

  6. While this is super cool, the upgrade from 13.0 to 13.1 failed for me:

    I followed the instructions up to the point where I would reboot into 13.1 (“bectl activate -t 13.0”, I have replaced 13.0 with 13.1 for the RELEASE and for the BE of course).

    After rebooting into 13.1, I couldn’t run the second “freebsd-update install” which would run on the currently running system, without mounting the BE. It would tell me that I have to fetch updates first, so apparently the system wasn’t aware that it was in the middle of upgrading to 13.1.

    The reason might be that the new BE runs on the new kernel, but on the old Userland, without being able to proceed with the upgrade:

    Checking userland and kernel versions:
    Userland and kernel are not in sync
    Userland version: 1300139
    Kernel version: 1301000

    Any ideas?

    (BTW, in the meantime, I have rebooted into the old 13.0 BE, destroyed 13.1, and started from scratch, but this time directly booting into 13.1, and then performing the upgrade as usual.)

Tell us what you think!