FreeBSD/arm64 is the FreeBSD port to the 64-bit ARM architecture, also known as AArch64 or ARMv8. All supported FreeBSD releases include support for ARMv8 and there are many packages and ports (3rd party applications) available to support the software you normally deploy with FreeBSD.
arm64 extends and modernizes the ARM architecture, adding specialized instructions and accelerators, as well as modern virtualization capabilities. The arm64 hardware, on which FreeBSD runs, ranges from small single board computers to mammoth multi-processor server machines with more than 100 cores on a single socket.
arm64 supports two models for describing hardware. Most single board computers (such as the raspberry pi or ROCK64) use flattened device tree (FDT) to describe the hardware, while other devices use the same ACPI (Advanced Configuration and Power Interface) as x86 CPUs. arm64 scales up to server platforms and the ARM specifications intend to make it possible to have images that ‘run anywhere’ through the SBSA (Server Base System Architecture). This standardizes the interface the machine uses to boot and interface with hardware. With SBSA and SBBR (Server Base Boot Requirement) arm64 hosts are able to boot using UEFI and discover hardware through ACPI. This makes it easier to deploy single images that work on many different hardware and virtual platforms, and brings arm64 deployment in line with amd64.
FreeBSD/arm64 has had support for server class platforms for a long time. The FreeBSD project uses Ampere servers to build packages for ARMv7 and ARMv8. These machines can be expensive and difficult to acquire, hence for experimentation it has been easier to pick up single board computers that are available for less than $50.
If you aren’t an embedded device vendor, then single board computers are not a proof-of-concept platform for more than the smallest experimentation. Thankfully FreeBSD/arm64 runs really well in the cloud and we can use cloud services to show that FreeBSD/arm64 is just as capable a server platform as any other of the supported architectures.
Cloud Services
arm64 is still breaking into the world of cloud hosting for virtual machines. AWS is the major host of virtual machines that will run arm64 images. At the time of writing Azure is experimenting with offering deployment pipelines on arm64, and other providers have experimented with arm64 virtual machines and full arm64 machine provisioning.
FreeBSD runs very well on the Amazon Web Services (AWS) with their Graviton 2 hosts. Graviton 2 is the second generation of general access arm64 servers that amazon has offered in the cloud.
AWS introduced their Graviton platform in November 2018 with their first generation of hardware using Annapurna Labs processors. This platform featured 16 Cortex-A72 cores organized as four quad-core clusters, all operating at 2.3 GHz.
At ReInvent, in December 2019, AWS introduced the Graviton2 platform built on top of ARM Neoverse N1 processors. These processors quadruple the number of cores compared to Graviton 1 from 16 to 64 and claim to offer up to 7x the performance of the first generation with the newer 7nm process.
Graviton2 processors power EC2 T4g, M6g, C6g and R6g instances, and their variants offer high performance NVMe-based storage and high-performance networking with Elastic Network Adapters (ENA).
Did you know?
Until the 30th of June 2021 AWS is offering free trial of the Graviton 2 t4g.micro instance size with 750 hours of free time available each month of the trial.
This is the perfect time to try running FreeBSD/arm64 in EC2.
Adding SSH Keys
We are going to access the virtual machine using ssh. EC2 can generate a key pair for each AMI as we launch it or we can upload our own. For most users with ssh keys, using an existing public key will be the easiest option.
If you do not already have a ssh key pair you can generate one with the ssh-keygen command:
[user@computer] $ ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/home/user/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/user/.ssh/id_rsa.
Your public key has been saved in /home/user/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:M5nvBGhF6uWTnBseST7X75Qd0Cld6e5S2xD1t27rrCw [email protected]
The key's randomart image is:
+---[RSA 3072]----+
| . o|
| o o.+|
| . + o.=.|
| . O * . +.o|
| + S . . .oo|
| . . % ..*.|
| o o B.+|
| o E.+.=.|
| . .o*+.|
+----[SHA256]-----+ We need to add the ssh public key (id_rsa.pub here) to AWS so EC2 can add the key to the virtual machine when it is created. We do so by visiting the EC2 console and selecting key pairs and selecting import key pair:
It is good practice to give key pairs meaningful names especially if they are generated for particular hosts (you might have a different key for a laptop and a desktop). I call this key ‘arm64 testing’ as that is what it is used for. I prefer to paste the public key into the box:
Running FreeBSD on EC2
EC2 t4g instances offer great general workload performance with the ability to burst performance of CPU, storage and network. CPU burst credits are determined by instance size with all sizes of instance being able to burst network performance up to 5 Gigabit/s.
The current release of FreeBSD (12.2) is available as an official image produced by the FreeBSD Release Engineering team and offered on the Amazon Market Place by the FreeBSD Foundation.
Launching a t4g.micro instance is a matter of hitting the launch button from the EC2 web interface, searching for FreeBSD AMIs, selecting FreeBSD/ARM 12 AMI type and launching an instance.
When we launch the virtual machine we need to select the key pair we added:
After a few minutes the machine will be available in the instance dashboard and you will be able to ssh to the virtual machine with your ssh key. FreeBSD AMIs are configured with a user named ec2-user which is available for use with ssh. Once you connect you will be shown the motd as on any other FreeBSD host:
[user@computer] $ ssh [email protected]
Last login: Mon Mar 15 10:21:09 2021 from tmux(2897).%0
FreeBSD 12.2-RELEASE r366954 GENERIC
Welcome to FreeBSD!
Release Notes, Errata: https://www.FreeBSD.org/releases/
Security Advisories: https://www.FreeBSD.org/security/
FreeBSD Handbook: https://www.FreeBSD.org/handbook/
FreeBSD FAQ: https://www.FreeBSD.org/faq/
Questions List: https://lists.FreeBSD.org/mailman/listinfo/freebsd-questions/
FreeBSD Forums: https://forums.FreeBSD.org/
Documents installed with the system are in the /usr/local/share/doc/freebsd/
directory, or can be installed later with: pkg install en-freebsd-doc
For other languages, replace "en" with a language code like de or fr.
Show the version of FreeBSD installed: freebsd-version; uname - a
Please include that output and any error messages when posting questions.
Introduction to manual pages: man man
FreeBSD directory layout: man hier
Edit /etc/motd to change this login announcement.
To easily configure your installed FreeBSD use bsdconfig(8).
-- Lars Engels <[email protected]>
ec2-user@freebsd:~ $ uname - a
FreeBSD freebsd 12.2-RELEASE FreeBSD 12.2-RELEASE r366954 GENERIC arm64
The ec2-user is in the wheel group and you can elevate to root by running the su command without a password.
Software Side
A lot of work has gone into FreeBSD/arm64 from both src and ports committers to make the platform a high-quality experience. Because of all of this hard work, using FreeBSD/arm64 in the cloud should not be very different than using amd64 based cloud images in any provider.
I ran some tests services with nginx web workloads and some iperf3 network performance tests without any issue on the FreeBSD/arm64 EC2 instances. The places you may encounter architecture related issues will be when software has not been written with portability or where FreeBSD/arm64 is not well supported upstream.
The second area of where you may encounter issues is when ports have been marked as amd64 only because in the past amd64 was the main platform these tools would be run. With the SBSA, arm64 is much more similar to amd64 server hardware, and most of these tools will run without any problem once the port metadata has been updated to include arm64.
t4g Burstable Instances
EC2 t4g instances are burstable, when the CPU, storage (EBS) or network (ENA) is not running above a baseline capacity the instance accrues credits which can later be used when there is an increase in workload. When the instance runs above the baseline capacity it will spend accrued credit.
The free trial t4g.micro instance we deployed in this example will accrue CPU credits when the CPU load is below the baseline of 10% utilization per vCPU. When below the baseline the instance will accrue 12 credits per hour up to a maximum of 288 credits. Credits are spent at the millisecond interval by EC2 and information about credits is made available at 5-minute intervals through cloudwatch.
Starting with ENA version 2.3.0 t4g-network, FreeBSD supports exposing ENA network metrics via sysctl. You first have to start collecting metrics by setting the sysctl that governs the metrics interval:
# sysctl dev.ena.1.eni_metrics.sample_interval=10
# sysctl dev.ena.network_interface.en1_metrics
I was unable to test this feature; all of the instances I started came up with ENA version 2.2.0.
Disk and network are also burstable and are throttled to a base line when their credit has been exhausted. The t4g burstable instances are a great way to experiment with FreeBSD/arm64 in the cloud, but don’t expect them to be able to handle sustained load as well as non-burstable instance types.
Conclusion
FreeBSD/arm64 is a growing platform, and I would expect to see more cloud providers offering services on arm64 and more and more powerful server hardware to come on to the market. arm64 is already able to provide significant power savings and new server offerings from companies such as Ampere offer very high core counts as well.
There is ongoing discussion in the FreeBSD project to promote FreeBSD/arm64 to a Tier 1 platform in FreeBSD 13. There is clear enthusiasm from FreeBSD developers for the arm64 architecture and it looks to have a great future in the project.
Embedded