Core Infrastructure: Why You Need to Control Your NTP

The Ease of Self-Run Servers

In a previous article, we discussed the Domain Name Service (DNS) and how you might run your own DNS server for your office or small business, and more importantly, why you would want to. Among the important benefits of running your own DNS server are:

• privacy considerations,
• reduction in internet traffic,
• performance benefits to each client machine, and
• joy of learning about a key internet protocol.

Though you probably will not notice much speed or privacy benefit in doing so, there are considerable security benefits to running your own Network Time Protocol (NTP) server. It is rather easy to do, and you will be surprised how much pointless internet traffic is eliminated.

Introduction to Network Time Protocol (NTP)

There are numerous reasons why accurate measurement of the time-of-day is desired in modern networks. Any number of synchronous communication needs will depend on the proper knowledge of the exact time (often to extreme precision). Yet even the day-to-day running of networked equipment will need accurate time for logging events and managing automated processes. The entire PKI system that underpins the security of HTTPS relies on accurate time to validate certificates. Incorrect time is the number one cause of issues loading websites. You may even have simple IoT devices that, for some reason, insist on knowing the time down to millisecond accuracy, and constantly check that they have it. For a while now, we have standardized the Network Time Protocol (NTP) to provide this information.

Early Time Protocols and Their Limitations

The history of getting the time on networked devices is quite complicated. Early versions of time-related protocols operated exactly as you would expect. For example, the “Time Protocol”^[1], classically on port 37 listens for either a UDP or TCP connection (just like DNS on port 53) and simply provides the number of seconds (as a 32-bit signed integer) since 1 January 1900 UTC.

There is also a “Daytime Protocol”^[2], classically on port 13, that responds with a freeform textual string describing the date and time to all comers^[3]. In addition to their poor resolution (a second is an eternity for precise timing), these are also very unreliable sources of the time as they do not account for network delays and other effects, nor is there any mechanism to validate that the source, itself, is even aware of the correct time. Accordingly, they are not suitable where high accuracy time-of-day information is needed.

How NTP Solves These Issues

These problems (and more) are solved with NTP, which classically operates on port 123. This crucial protocol is born of the genius of Dr. Keith Marzullo’s Ph.D. dissertation in electrical engineering at Stanford in 1984 which outlined an algorithm for getting timing information from “noisy” sources. Later it was implemented in software by the late internet pioneer Dr. David Mills. After several updates and tweaks, NTP has become a mature mechanism. Today, it is coupled with a globe-spanning network of authoritative time servers. Any device on the internet or inside a LAN now knows the exact time to an accuracy of a millisecond^[4], typically.

How Does NTP Work?

NTP Stratum Levels

The backbone of NTP is the time server, analogous to the nameserver that we discussed in the DNS article. While we generally do not track in DNS our distance from the authoritative, non-recursive nameserver that ultimately informed our answers, this piece of information is central to NTP, and is called the stratum. A time server whose time was set by a physical precision time device (such as an atomic clock, or the GPS constellation) is known as a stratum 1 server. A time server whose time was set by a stratum 1 server is known as a stratum 2 server. One set by that server, of course, is a stratum 3 server, and so on. Naturally, as you synchronize to progressively higher strata, the precision of time-of-day measurement accumulates error. However, it usually does not accumulate so much that typical hosts need to be very concerned.

Ultimately, most end-of-the-line hosts and routers might be synchronized to relatively high strata. For example, I recently read a question in a Cisco forum where a customer indicated his enterprise routers were at stratum 6. I think this is unusually high, and expect most people are synchronizing to NTP servers in strata 2, 3, and 4 (though as it happens, my NTP server is synchronized to a stratum 1 at the moment, and thus my LAN is getting stratum 2 timing; you just never know, as will be made clear later in this article). Most are probably reliably within a couple milliseconds of the authoritatively true standard world time regardless of the specific stratum.

How NTP Clients Query Servers

Clients query time servers with the NTP or the related simplified SNTP protocol. The choice of which servers to query varies: it may be set by default in the host operating system (for example, Windows system probably queries time.windows.com^[5]), manually configured in FreeBSD or Linux operating system, or—more frustratingly—hard-coded into certain IoT or voice assistant devices. In many cases, routers and similar devices obtain their time servers via DHCP.

Optimizing NTP Queries

Modern NTP client implementations (assuming they are configured in a reasonable way) will adjust the frequency of their queries according to how accurate the timekeeping has recently been. At the moment, my NTP server is synchronizing itself upstream approximately every 9 hours and believes itself to be within 0.0001 seconds of the real time. All of the devices on my LAN are receiving their time information from it.

Except for a stratum 1 server which gets its time from an authoritative physical time source, a properly implemented NTP server will typically query several (say, at least three, but often far more) reliable servers for time information. Through the magic of algorithms for deducing correct time from lagged and noisy sources (which of course you may read about if curious), your server will calculate an accurate time-of-day, and generally use that result to set its own system time and (if you configure it to allow this) will gladly offer this time to others if asked. Every time a new calculation is made, the system time will be adjusted.

Setting Your System Time

FreeBSD Timecounters

The system time is maintained in FreeBSD by various timecounters. You may view available timecounters by polling the oid kern.timecounter.choice with sysctl(8):

root@daneel:/etc/ntp # sysctl kern.timecounter.choice
kern.timecounter.choice: TSC-low(1000) ACPI-fast(900) i8254(0) HPET(950) dummy(-1000000)

There are at least four usable timecounters for this server, each shown with some vague scalar in parentheses that indicates their expected “quality” compared to other counters. “Quality” can mean any combination of general reliability and stability and resolution. Should any have a negative quality value, this indicates that the operating system has determined that the timecounter is “broken”, as you can see for the so-called dummy timecounter in the above example.

Selecting the Best Timecounter for Your System

Typically, the operating system will choose the highest quality available timecounter, even though there is no guarantee that it is, in fact, the highest quality timecounter^[6]. Timecounters are affected by a motley collection of all kinds of environmental variables and conditions. The counter in use is set by sysctl(8) (or, if desired, in /etc/sysctl.conf) with oid kern.timecounter.hardware:

root@daneel:/etc/ntp # sysctl kern.timecounter.hardware 
kern.timecounter.hardware: HPET

root@daneel:/etc/ntp # cat /etc/sysctl.conf | grep timecounter 
kern.timecounter.hardware=HPET

For each timecounter, you may explore its specifications:

root@daneel:/etc/ntp # sysctl kern.timecounter.tc.HPET 
kern.timecounter.tc.HPET.quality: 950 
kern.timecounter.tc.HPET.frequency: 24000000 
kern.timecounter.tc.HPET.counter: 381644536 
kern.timecounter.tc.HPET.mask: 4294967295

As you might imagine, “frequency” indicates the fraction of the second each tick ideally represents^[7], and “counter” represents the current reading of the timer odometer. For example,

root@daneel:/etc/ntp # sysctl kern.timecounter.tc.HPET.counter ; sleep 1 ; sysctl kern.timecounter.tc.HPET.counter 
kern.timecounter.tc.HPET.counter: 1951503309 
kern.timecounter.tc.HPET.counter: 1975687194 

is close enough to the 24000000 ticks per second that we expected.

Similarly, system events such as interrupt signals are maintained by event timers. There will be a collection of possibilities here. Generally different from the timecounters (although a few, and especially the high-precision event timer, or HPET, is likely available to both), though these are beyond the scope of preparing to run an NTP server.

root@daneel:/etc/ntp # sysctl kern.eventtimer.choice 
kern.eventtimer.choice: LAPIC(600) HPET(550) i8254(100) RTC(0) 

Timekeeping in Virtual Machines

This gets even more complicated when this hardware is emulated, such as inside a Virtual Machine guest. Ensuring accurate time inside a VM is important, and due to the variability of the emulated counters is more likely to drift from the clock of the host. Some hypervisors do implement a paravirtualized clock, that provides the guest OS with direct access to read the clock of the host. It is still best practice to run NTP inside each VM.

Should You Run Your Own NTP Server?

Key Benefits

If you are going to run your own NTP server for all of the devices on your network, the correct procedure is to designate a single one of your LAN hosts to be an NTP server which uses the NTP pool to get the time (more on this shortly). It’s important to have each of your hosts on the LAN side synchronize themselves to your NTP server. There are several advantages to this setup:

• Have very consistent time across all your devices.
• Be able to sync dozens or hundreds of devices on your LAN at the cost of only one set of NTP internet traffic.
• Limit needless internet traffic, and more importantly, use the donated resources of the NTP pool as sparingly as possible.
• Have a timeserver inside the LAN that continues to provide time information in case of an internet outage.
• Enjoy the intellectual connection to one of the most important services of the internet and may even find the awareness of timekeeping itself to be satisfying.

A couple of these reasons bear some emphasis. Consider the second and third points, that internet traffic would be reduced and resources respected. It is not an inconsiderable amount of traffic, second only in abject waste to the uncontested winner, DNS.

The Impact of IoT and Excessive NTP Queries

For example, my home, which has the usual number of computer hosts, cell phones^[8], tablets, voice assistants, smart plugs, bulbs, televisions, and appliances, generates many thousands of NTP time requests every single day. Very little of it is my servers or computers; most are all the other silly devices. I am not sure why devices such as a random smart plug that turns on a small decorative lamp at certain times of the day are so very insistent on knowing what the time of day is down to sub-millisecond accuracy and then checking that they have it several times per hour. But this is the reality–our homes, to say nothing of more complicated commercial setups, generate a largely useless flood of NTP synchronization packets. Most are resolved on the open internet, at volunteer expense.

Security Considerations for NTP Servers

There are also security considerations, if an observer can detect  how many devices are on your network by examining the volume of NTP queries made from your connection. Most of the time, we will not have fine, or even any, control of the NTP behavior of our devices. However, we can still stand between them and the bandwidth and resources of internet volunteers.

The continuance of semi-reliable timing information in the event of an internet outage is more important than you might think. For example, you might be surprised how quickly a device’s system time can go astray without NTP updates. Worse yet, I have a few energy monitoring devices in my home made by one company (that shall remain nameless) with the unfortunate propensity to shut down the power. It’ll then brutally cycle on-and-off every couple minutes, in perpetuity, whenever they receive anomalous NTP behavior^[9]. Needless to say, this behavior is very sub-optimal when the smart plug is attached to what is supposed to be an always-on FreeBSD backup server.

Granted, the various advantages are not so impressive as they can be with hosting your own DNS server. They are still significant, especially when compared to the relative ease of setting up and managing your own NTP services in your LAN. You will find there is almost no measurable load on your server for running the type of NTP server we will discuss. My NTP server, servicing thousands of queries per day, consumes just about 0.1 seconds of CPU time daily, an almost immeasurably small amount.

But, where will your server get its time from?

The NTP Pool Project

It is natural to ask who the ultimate custodian for the correct time is. Could there not be competing standards for what time it is? Entire books have been written about the history of such matters^[10]. These days, however, there is wide agreement and an international body of time servers that are considered the de facto standard for typical consumer devices. These are the servers of the NTP Pool Project, and at the moment of this writing, they are 5,255 servers spread across the globe and largely run by volunteers and organizations. Though strata vary, the majority of servers in this pool are very low stratum. Entities hosting these servers run the gamut of governments, universities, ISPs, small businesses, and even a few individual hobbyists with bandwidth to donate.

In the simplest implementation, one merely need resolve pool.ntp.org^[11] to get some perfectly fine IP addresses of servers in the NTP pool. Though you will not be manually specifying these IP addresses, you will see that it is useful to know that typically four are provided in the DNS response, and these rotate and change with regularity. Unless you do some digging, you do not know who the servers belong to, or where they’re located:

root@daneel:/etc/ntp # host -t A pool.ntp.org 
pool.ntp.org has address 50.205.57.38 
pool.ntp.org has address 74.208.72.129 
pool.ntp.org has address 162.159.200.1 
pool.ntp.org has address 173.73.96.68 

And just a few moments later, an almost(!) completely different set:

root@daneel:/etc/ntp # host -t A pool.ntp.org 
pool.ntp.org has address 216.31.16.12 
pool.ntp.org has address 104.167.241.253 
pool.ntp.org has address 144.202.66.214 
pool.ntp.org has address 162.159.200.1

This resolution should automatically provide geographically optimal servers. However, if you prefer time servers from a specific continent or country, you can easily specify that.

root@daneel:/etc/ntp # host -t A ca.pool.ntp.org 
ca.pool.ntp.org has address 23.133.168.245 
ca.pool.ntp.org has address 162.159.200.123 
ca.pool.ntp.org has address 207.34.48.31 
ca.pool.ntp.org has address 23.133.168.244 

root@daneel:/etc/ntp # host -t A oceania.pool.ntp.org 
oceania.pool.ntp.org has address 192.107.172.20 
oceania.pool.ntp.org has address 162.159.200.1 
oceania.pool.ntp.org has address 159.196.178.7 
oceania.pool.ntp.org has address 103.106.65.219

Feel free to explore the different configuration options available in the NTP Pool zones.

Other NTP servers exist, but if you're setting up a LAN NTP server, there's little reason to use anything other than the NTP Pool Project’s servers.

Setting Up Your Own Server

Installing and Configuring Chrony

There is an old, matronly, reliable software package for NTP servers, called ntpd. It’s present by default on any FreeBSD (probably also Linux) installation as ntpd(8) with configuration file located at /etc/ntp.conf. Like DNS’s BIND software, it is often not the choice these days for small deployments by hobbyists and power users. In our article on DNS, you may recall that we installed unbound(8), which was a smaller, modern, agile, and frankly more interesting DNS resolver than the classical one. The NTP software analogous to unbound, and of course the one we will be demonstrating in this article, is called chrony, and is easily installed from packages or ports on any FreeBSD server. The easiest way to install the software is with pkg(8):

pkg install net/chrony

This installs the software (and any needed dependencies) with default options. Some options include supporting “network time security” which is generally beyond the scope of this article and almost certainly unnecessary to you. The configuration file is located at /usr/local/etc/chrony.conf and contains quite a bit of commentary outlining some of the options.

Note: a pristine copy of the configuration file is also placed in the directory as chrony.conf.sample, should you ever need to revert to a clean version. The full documentation for all configuration commands, at least for the current version of chrony, can be viewed on the Chrony Project’s manual. In addition, you should begin at once to familiarize yourself with its many possibilities.

Here is a very minimal configuration file that will get you started. Of course, you will want to learn about the power and diversity of chrony’s capabilities and create a configuration that works well for you.

root@giskard:/usr/local/etc # cat chrony.conf 
pool pool.ntp.org iburst maxpoll 16 minpoll 6 
server 1.ca.pool.ntp.org iburst maxpoll 16 minpoll 6 
minsources 3 
driftfile /var/db/chrony 
log measurements statistics tracking 
logdir /var/log/chrony 
allow 0.0.0.0/0 
local stratum 10

I have used the pool directive to get the multiple servers returned by pool.ntp.org as discussed earlier in the document. Also, I have used the server directive (which is essentially the same as pool, but retrieves only one server) to add a random Canadian timeserver. minpoll specifies the fastest rate at which timeservers can be pinged as a power-of-2 number of seconds.

Responsible server admins should not set minpoll lower than 6. They can set maxpoll (which defaults to 10) as high as possible. A value of 16 is very high, nearly a whole day, but it will only choose long time intervals when timing conditions are perfect.

Starting and Monitoring Your NTP Server

Then, to manually start your server:

service chronyd onestart

and verify that it has started by searching for a daemon listening on port 123:

root@giskard:/var/log/chrony # sockstat | grep ":123"
chronyd  chronyd    20466 7   udp4   *:123                  *:*

In addition to the log files in /var/log/chrony that you will doubtlessly find illuminating, the chrony daemon chronyd(8) is paired with a client program chronyc(1) for inquiring about the running timeserver. Two very useful commands here are tracking and sources:

root@giskard:/var/log/chrony # chronyc tracking 
Reference ID	: 2D3F360D (LAX.CALTICK.NET) 
Stratum     	: 3 
Ref time (UTC)  	: Thu Feb 13 01:26:08 2025 
System time 	: 0.000286188 seconds fast of NTP time 
Last offset 	: +0.000304798 seconds 
RMS offset  	: 0.001287244 seconds 
Frequency   	: 19.205 ppm fast 
Residual freq   	: +0.009 ppm 
Skew        	: 1.460 ppm 
Root delay  	: 0.063127175 seconds 
Root dispersion 	: 0.000833148 seconds 
Update interval	: 129.7 seconds 
Leap status 	: Normal 

root@giskard:/var/log/chrony # chronyc sources -v 

  .-- Source mode  '^' = server, '=' = peer, '#' = local clock. 
 / .- Source state '*' = current best, '+' = combined, '-' = not combined, 
| /         	'x' = may be in error, '~' = too variable, '?' = unusable. 
||                                             	.- xxxx [ yyyy ] +/- zzzz 
||  	Reachability register (octal) -.       	|  xxxx = adjusted offset, 
||  	Log2(Polling interval) --.  	|      	|  yyyy = measured offset, 
||                            	\ 	|      	|  zzzz = estimated error. 
||                             	|	|       	\ 
MS Name/IP address     	Stratum Poll Reach LastRx Last sample 
=============================================================================== 
^* LAX.CALTICK.NET           	      2   6   377     1   -169us[  -84us] +/-   34ms 
^+ nu.binary.net             	      2   7   177    63  +1050us[+1133us] +/-   55ms 
^+ time2.tritan.host         	      2   7   367   130  -8663us[-8279us] +/-   73ms 
^+ ns2.fortrockdc.com        	      2   6   377     0  +1781us[+1781us] +/-   45ms 
^+ muug.ca                   	      2   7   365   125  -7180ns[ +377us] +/-   64ms 

In the above you can see, for example, that we are a stratum-3 time server, and have been synchronized based on the contributions of five different stratum-2 time servers, and that we believe ourselves to be within a couple milliseconds of the true time. Not only is this server now tracking the time, but it also has set the system clock^[12], and is listening (thanks to the permissive allow directive in the configuration file) for any NTP requests on port 123. It is also interesting to note that the built-in timecounter, as shown, is ticking at a rate that is off by approximately 20 parts per million^[13]. This is enough for a drift of more than a second per day left unchecked; not the end of the world, of course, but probably worse than you imagined.

Once you have everything the way you want it, why not make sure the service starts by default:

sysrc chronyd_enable=YES

Now start having the LAN side clients use your NTP server (instead of one over the internet) by either inputting this IP address. Alternatively, using DNS to resolve something like timeserver.home to this server, and then placing this information in routers and clients.

Taming IoT’s NTP Noise

Of course, one thing you won’t be able to do without another step is to curtail the rude behavior of countless IoT devices that use hard-coded hostnames or IP addresses to do their own nearly incessant NTP queries out to the internet. For this, you will need to put on your networking hat to detect and redirect egressing UDP traffic^[14] on port 123 at your WAN-facing router, and manually redirect it to your local NTP server. Watch in awe as literally thousands of NTP packets that would have cluttered the internet and volunteer run NTP servers, each day, are now easily handled within the LAN.

Please remember that this is an example configuration. You should learn more about exactly what configurations options are best for you and explore all of the interesting information chronyd can provide.

Wrapping Up: Taking Control of Your Network Time

By now, it should be clear that running your own NTP server isn’t just a theoretical exercise—it’s a practical and responsible way to improve time synchronization, reduce unnecessary internet traffic, and minimize reliance on volunteer-run servers.

With a little configuration, you can bring order to the relentless time queries of IoT devices, ensure consistent timekeeping across your network, and enhance overall efficiency. Whether for security, reliability, or simply better network management, taking control of your NTP is a worthwhile investment.

If your organization relies on FreeBSD and needs expert support for time synchronization, networking, or broader system optimization, Klara’s FreeBSD Development & Support services can help.

─────────────────────────────

Footnotes

^[1] This has been standardized in RFC 868, dating back to early 1983.

^[2] Unsurprisingly, this has also been standardized (more accurately, loosely described) in the immediately preceding RFC 867.

^[3] If you’re looking for these protocols today, they do still exist; in FreeBSD (and presumably other operating systems), these services are not listening for security reasons, by default. You may, of course, enable them. These and many other older protocols are controlled by the inetd(8) daemon, which you may explore at your leisure and risk. Its configuration file may be found at /etc/inetd.conf.

^[4] Of course, it is not hard to imagine in a world with some networks now dealing with speeds measured in Tbps that one could require something better than the millisecond precision offered by NTP.  Indeed, the Precision Time Protocol (PTP), beyond the scope of this article, can achieve, potentially, precision in the nanoseconds.

^[5] In operating systems geared toward average end users, there is the expectation that few of its users are even aware of NTP, much less interested in having fine-grained control over how the computer clock is synchronized.  In these cases, you may find installing additional NTP software to be a necessity.  I have had very happy results on Windows with the free and open source NetTime, despite its codebase having been moribund for some years now.

^[6] In fact, in my system, HPET appears to be more stable than TSC-low, despite the relative “quality” scores, and so I have forced its use.

^[7] But of course, does not, which is why the system time must be maintained at intervals.

^[8] Many cell phones do not actually use the NTP protocol for time-of-day.  Instead, they may use NITZ, which uses the cell network itself, not its data connection, for time-of-day and time zone information.

^[9] Before you ask, yes, I have tried many times to contact them about it.  But finding anyone–even a technical person–that has any familiarity with NTP can be nearly impossible. This becomes even more difficult when the NTP implementation is blindly imported as an IoT dependency into their product, as is the case here.

^[10] For example, A Brief History of Timekeeping, the Science of Marking Time from Stonehenge to Atomic Clocks, by Chad Orzel (BenBella Books, 2002, ISBN: 1953295606)

^[11] For all of these ntp.org addresses, you may specify a 0, 1, 2, or 3, to get guaranteed different sets of addresses, if you wish and have the need.  For example, 0.pool.ntp.org.

^[12] Speaking of which, you will not be able to run chronyd in a FreeBSD jail. It requires access to the running kernel to set the system clock, which would defeat the purpose of a jail anyway.

^[13] And who knows what it will be tomorrow, when the temperature and humidity change.

^[14] Except, of course, those emanating from your own server who is keeping time for you by going out and fetching time from the NTP pool…