OpenZFS – OpenZFS For HPC Clusters
Discover how OpenZFS can provide cost-effective and reliable storage for high-performance computing (HPC) workloads in this comprehensive write-up.
Not surprisingly, the periodic scripts live in the /etc/periodic directory. This directory organizes the scripts into 4 subdirectories: daily, weekly, monthly, and security:
Each of the built-in scripts starts with a number, indicating the order of script execution. The name portion of the script summarizes the purpose of the script. These are commented, Bourne shell scripts, some of which contain configurable variables. As an example, here is the beginning comment in /etc/periodic/daily/100.clean-disks: Remove garbage files more than $daily_clean_disks_days days old.
The periodic.conf(5) man page provides a description of each script and any available configurable options.
By default, some scripts are already enabled (configured to automatically run) and some are not enabled (meaning that they will not run unless you change the script’s enable value to YES in the periodic configuration file). The default values, indicating which scripts are set to run and their configurations, are listed in the /etc/defaults/periodic.conf file.
If we grep for the YES value on my FreeBSD 13.0 system and pipe the output as a word count, we see that 54 items are set to YES by default. We’ll then pipe the output to head to see the first 10 items:
Note that some of the scripts have options in addition to enable. In the example above, the daily scripts clean_preserve and clean_rwho have a verbose option to indicate whether or not the script should list the files it deleted.
You will want to review this file yourself to see which scripts interest you and whether you want to enable some, disable others, or change any of the default configurable values.
NOTE: While you will want to review /etc/defaults/periodic.conf, do not edit this file as it is overwritten when the system is updated. Instead, create an overrides file named /etc/periodic.conf that contains the values that you want to differ from the defaults, as demonstrated in the next section.
In this example, we’re interested in the ZFS scripts so we start by appending the default zfs values from /etc/defaults/periodic.conf to a file we’ll save as /etc/periodic.conf:
After comparing these default values to their descriptions in periodic.conf(5), we decide that we want to:
You might also be interested in
At Klara, we have an entire team dedicated to helping you with your FreeBSD projects. Whether you’re planning a FreeBSD project, or are in the middle of one and need a bit of extra insight, we’re here to help!
Since /etc/periodic.conf is meant to just contain the overrides to the default settings, our file will be shorter once we remove the defaults we want to keep. In the lines that remain, we change each NO to YES; we also changed the default scrub threshold from 35 days (every 5 weeks) to 28 (every 4 weeks):
Your file will differ from ours, as it will contain the defaults that you are interested in customizing. The main point is that your file should only contain the overrides. This makes it easy to determine how a system’s custom configuration differs from the built-in defaults.
By default, the output from running the periodic scripts is emailed to the root user on the local system. You can use the built-in mail program to read these emails. Here is an example from one of our systems, demonstrating the emails sent by the monthly run, weekly run, daily security run, and daily run. We typed 1 to read the first listed message for the monthly run, then typed q to exit out of the mail program.
If you’ve never used mail to read the local mail on a FreeBSD system that has been up for some time, it is likely that there are dozens or even hundreds of unread email messages from periodic. Fortunately, it is easy to change this default.
Here is the section of /etc/defaults/periodic.conf that configures the output which was sent to the root user’s local mail:
In fact, each of the 4 types of scripts (daily, weekly, monthly, security) has a section for configuring where and how much output to report. Here is the section for the daily scripts:
To send the output to an email address, replace root with the email address. You can specify multiple email addresses by separating them with a space:
daily_output=”address1@me.com address2@me.org”
To instead send the output to a log file, replace root with the name of the file. Note that the following locations are understood by newsyslog so these log files will automatically rotate:
daily_output=/var/log/daily.log
weekly_output=/var/log/weekly.log
monthly_output=/var/log/monthly.log
NOTE: remember to replace root in each section you want to configure. While this might seem like a hassle, it offers the flexibility of configuring output so that it is sent to the appropriate person or file. Here is a quick way to see the available options for configuring output being sent to an email address or log file:
Additionally, these options are available to reduce the output in the report:
Again, the * indicates that these are configurable in each section (daily, weekly, etc.).
Finally, the security script options also include the ability to control how often the security scripts are run:
If you take a closer look at the periodic scripts themselves, you will see that they all begin and end with the same code—the only part that differs is the middle of the script where the actual commands in the script are run. An example can be seen in this script; we’ve added line numbers to the output:
Lines 1-15 begin every periodic script and tell periodic to read the default configuration file to determine the default values, including the name of the custom configuration file, and then look to see if there are any overrides in the custom configuration file. As we’ve seen in this article, the configuration values allow periodic to determine if the script should be run, when it should be run, and where the output, and how much output, should be sent.
Line 14 contains the name of the script’s enable variable. In this example, if weekly_locate_enable is set to YES in either the default or custom periodic.conf, the commands in the following lines are executed during the weekly run. When creating your own script, give your variable a name that begins with when it should be run (daily, weekly, monthly, or security) followed by a summary of what the script does, and ending with enable.
Lines 16-30 do the actual work of this particular script: updating the locate database. If you are creating your own periodic script, you would put the commands you want your script to run here. If you wish your script to have any configurable options (such as the daily_scrub_zfs_default_threshold we saw in the previous section), create a variable case for each configuration option.
Line 32 ends every periodic script.
If you create your own periodic script, place it in the default subdirectory location of /usr/local/etc/periodic/. If pkg is installed on the system, it will have already created a subdirectory structure within /usr/local/etc/periodic/ for its daily, weekly, and security scripts. If not, you can create the subdirectory that matches when you would like your script to run. Don’t forget to enable the script variable in /etc/periodic.conf, as well as indicate the values for any additional variables you created within your script.
In summary, when creating your own periodic scripts:
FreeBSD provides a highly configurable mechanism for running built-in and custom maintenance scripts. It is well worth the time to review the available scripts and to create a customized configuration that meets the needs of the system. Don’t forget to read the emails or log files from these scripts as they are meant to keep you on top of what is happening with the system.
Discover how OpenZFS can provide cost-effective and reliable storage for high-performance computing (HPC) workloads in this comprehensive write-up.
The most common category of ZFS questions is “how should I set up my pool?” Sometimes the question ends “… using the drives I already have” and sometimes it ends with “and how many drives should I buy.” Either way, today’s article can help you make sense of your options.
OpenZFS is a cost-effective open source storage solution for Educational institutions. Learn how universities & system administrators can maximize its benefits.
Pingback: Valuable News – 2022/01/31 | 𝚟𝚎𝚛𝚖𝚊𝚍𝚎𝚗
You write really great articles and it would be super useful if RSS contained whole articles, not just first few lines. Thank you!
Pingback: Part 2: Tuning Your FreeBSD Configuration for Your NAS | Klara Inc.
Pingback: Valuable News – 2022/09/12 | 𝚟𝚎𝚛𝚖𝚊𝚍𝚎𝚗