OpenZFS and storage in general is a complex and important part of any project’s architecture. It should be planned thoughtfully and ideally, ahead of time! In this article, we’ll talk about how to understand, measure, and plan for your storage performance needs.
FreeBSD Developer Summit 2021
FreeBSD Developer Summit 2021
CamCorder by Warner Loss
Warner Losh presented work he has been developing at Netflix to provide a tcpdump compatible tracing framework for CAM, the FreeBSD storage device subsystem. This allows for the capture of CCBs, the command blocks used by the storage subsystem, and provides improved observability and debugging capabilities for storage.
IPv6 on FreeBSD by Hiroki Sato
Hiroki Sato of the FreeBSD Foundation presented a session about FreeBSD’s IPv6 stack. FreeBSD has supported IPv6 since the release of FreeBSD 4.0 in 2000. The goal of this session was to gather information on various organizations experience with IPv6 and ways to improve that experience on FreeBSD. Topics included configuration, the lack of a default DHCPv6 client, and improving application support for link-local addresses.
IFLIB by Sai Rajesh Tallamraju
NetApp developer Sai Rajesh Tallamraju presented about his team’s experience working with IFLIB, FreeBSD’s framework for making the writing of network device drivers easier. By providing common functionality and idioms as a library, IFLIB aims to make writing drivers easier and less error prone, while at the same time providing a more consistent configuration and management experience to the user. Between FreeBSD 11 and 12, the Intel network drivers were rewritten using IFLIB, and NetApp discussed their experience with porting their customizations to IFLIB, including some advanced features that were missing from the Intel IFLIB drivers, such as Adaptive Interrupt Moderation (AIM).
FreeBSD Core Team
Next, the FreeBSD core team presented some of the things they have been working on since they took office in the summer of 2020. Topics including the expansion in use of SPDX (License) tagging, work by members of the core team to upstream years’ worth of improvements to the bsd-user features of QEMU, and then recruiting new members for various FreeBSD teams including Release Engineering, Cluster Administration, Security Team, Ports Manager, the Documentation team, and the CI team. The core team then spent the rest of their time answering questions from the live stream.
FreeBSD by the Fire with Kirk McKusick
What better way to end the day than story time with Kirk McKusick. Kirk spent 90 minutes regaling the audience with stories from the early days of BSD, including the AT&T/USL lawsuit and the TCP/IP wars.
Icelandic DNS Registry by Einar Bjarni Halldórsson
Einar Bjarni Halldórsson from ISNIC, the Icelandic TLD authority, presented about their infrastructure which has been based on FreeBSD for over 20 years. They detailed how their small team (3 developers and 3 sysadmins) develop and maintain their infrastructure. Einar detailed their technology stack, which includes BIND, Unbound, OpenDNSSec, Apache, PHP, Java, PostgreSQL, HAProxy, and ZFS.
Boot Code by Allan Jude
Next, Klara’s own Allan Jude chaired a session on the FreeBSD boot code, including the legacy bootstrap and loader, as well as the EFI loader. Discussion focused the need for a modern updating mechanism and accompanying documentation, but the session also covered design goals for a failsafe update mechanism, special considerations for ZFS, and general boot performance. In addition to discussing the future direction of the boot code, and ways to improve and expand the existing testing and CI of the boot code, the session spawned a working group to document the current state and design of the various boot codes, and develop a design and requirements document for some of the future development in order to grow the number of people who can contribute to this part of FreeBSD.
RAID-Z Expansion by Matt Ahrens
Matt Ahrens of the OpenZFS project presented on his FreeBSD Foundation funded work to implement RAID-Z expansion, the ability to add an additional disk to an existing RAID-Z vdev to make it wider and increase its capacity. This is one of the most often requested features in ZFS, and it is now complete and undergoing review. Matt provided an overview of how it works, and details some of the challenges faced during its 4 years of development.
Project Workflow Planning by Warner Losh
Warner Losh of the FreeBSD core team presented an update on the various workflows in the FreeBSD project, and current and future efforts to improve them. The first topics were the completion of the migration from SVN to GIT for source control, and the migration of the project’s documentation from XML Docbook to AsciiDoctor. Then Warner discussed the outcome of a pair of round-table sessions he held on improving the contributor and committer workflows, looking at ways to reduce friction for new and existing contributors, and improve the project’s ability to triage, review, and integrate submitted changes. Types of friction include needless extra time or steps in the process, project policies leading to undesired outcomes, project practices failing to leverage available resources, or simply contributions going unnoticed due to clutter in the bug tracker or review system.
FreeBSD 14.0 Planning Session
The next session is a mainstay of FreeBSD developer summits, the “Have/Need/Want” session. Developers and vendors that use FreeBSD come together to list recent developments they have that they intent to integrate into FreeBSD in the near future, as well as present work they will need over the next year to help foster collaboration, and then go on to make a list of items that users and vendors want, to try to find others who may also be interested in those features, and to gauge how much desire there is for a particular feature. This session built on a similar session from the developer summit in November of 2020, providing status updates on many projects, and noting those that were completed.
Works in Progress
- To end the day, various attendees presented their in-progress projects:
- Kevin Bowling presented his work on iflib and the Intel drivers.
- Arka Sharma presented on OCSSD, a standard software flash translation layer they implemented for FreeBSD.
- Charlie Li presented on efforts to maintain ports of GNU Radio and various desktop software.
- FreeBSD Foundation Interns Cyril and Yang presented on runj (OCI container runtime for FreeBSD) and a prototype of a browser-driver installer respectively.
- Konstantin Belousov presented on EFI runtime services, boot to kernel handoff, and Kernel ASLR.
- Finally, Ed Maste presented on recent work to improve lldb, the llvm debugger, on FreeBSD.
Like this article? Share it!
Data security is about preventing data from being disclosed, ensuring that only the correct people can access it. Data integrity ensures the data is correct, that it has not become corrupt due to hardware failure or other issues. With ZFS, you can get both.
If you’re getting ready to close the year, we’ve got you covered with some of the best content that we put out in the past year. Check out our top ZFS and FreeBSD content from 2022 and go down the open source rabbit hole for the holidays!