Early on, developers working on Unix created a set of ideals that acted as a roadmap for the programs they wrote. They didn’t always follow these ideals, but they set the tone for the Unix project. Keep programs simple, design programs to work together, test early and often – are only some of these ideals. To this day, the Unix Philosophy impacts many projects.
History of FreeBSD – Part 4: BSD and TCP/IP
History of FreeBSD
Part 4: BSD and TCP/IP – How a Game Winning Team Was Formed
Today, FreeBSD is used by many companies and individuals to manage network traffic or build embedded systems in one form or another. BSD was created many years before the idea of networking computers across the country was even possible. In this new article from the “History of FreeBSD” series, we will look at how networking was first added to BSD.
DARPA Comes Knocking
Before the Internet that you are using right now, there was a little thing called . ARPANET was one of the first attempts to network computers across the country. It got its name because it was created and run by the US Department of Defense’s Advanced Research Projects Agency. (The agency is now called Defense Advanced Research Projects Agency and will be referred to by the acronym DARPA going forward.)
In the early 1980s, DARPA decided to standardize its computer network. At the time, DARPA had many projects around the country and each of those projects ran on different systems with different hardware. Many of them didn’t even share a computer language. This made it very difficult for those projects to share their results. DARPA picked the VAX system from the Digital Equipment Corporation for the hardware side of things.
Once DARPA decided what computer they were going to use, they had to pick an operating system. At the time, there were two competing options: DEC’s own VAX/VMS or a Unix from Berkeley that was growing in popularity.
At the time, some wanted “a real operating system with real vendor support”. Others demanded that DARPA pick BSD because they were already using it. Unsurprisingly, there was a lot of discussion on the two choices.
A man from the Stanford Research Institute named David Kashtan wrote a paper comparing the performance of VAX/VMS versus BSD in a series of benchmarks. The benchmarks included: “how fast can you do getpid() and how fast can you write a byte back and forth over a pipe between two processes to measure context switch times”. The results showed that BSD lagged behind VAX/VMS in every area.
At the time, Bill Joy was leading the BSD project. (Joy is also known as the creator of the vi editor and a co-founder of Sun Microsystems.) He was not happy when he caught wind of Kashtan’s paper. Joy proceeded to spend the next couple of months adding features and improving the performance of the BSD kernel. When he was done, he published his own paper “showing that Kashtan’s benchmarks could be made to run as well on Unix as they could on VMS”.
Did you know?
Improve the way you make use of ZFS in your company
Did you know you can rely on Klara engineers for anything from a ZFS performance audit, to developing new ZFS features to ultimately deploying an entire storage system on ZFS?
The updated system was released as 4.1BSD in June 1981. The original plan had been to call the release 5BSD, but AT&T objected. They feared their commercial customers would confuse 5BSD with AT&T’s own System V. Instead, Berkley decided that “the naming scheme for future releases to stay at 4BSD and just increment the minor number”.
Time to Add TCP/IP
Once DARPA had chosen BSD, they told Berkley that they wanted several features added. One of these was networking support, specifically TCP/IP. At the time, BSD had basic networking features through Unix-to-Unix Copy or UUCP. But DARPA wanted TCP/IP.
According to Kirk McKusick, DARPA didn’t trust what they considered students at Berkeley to understand how to do real networking. So, DARPA hired a company named Bolt Beranek and Newman (BBN) to write the actual TCP/IP protocols. Joy was supposed to write the API or what we call the socket interface today.
Joy didn’t know much about networking but was assisted by Sam Leffler. Leffler had more experience with networking and made several suggestions that improved the code.
Joy had a reputation for writing code very quickly. McKusick said that what Joy could do in a month would take him a year. So, it didn’t take him long to write his code for the networking stack.
Joy contacted BBN to see if they had finished their part of the project. They replied that they were not done yet. Joy asked if they at least had a prototype they could send him because he needed to test the API he had written. BNN sent them what they had and Joy plugged it in.
The BBN code worked, but there were some performance issues. BBN has tested the code on a 50-kilobit network. This was the backbone of ARPANET at the time. Berkley had access to an experimental 3-megabit network from Xerox PARC. The BBN code could not take advantage of the extra network capabilities.
Joy then modified and rewrote portions of the BBN code so that it could take advantage of the faster network. According to Robert Kahn (one of the original creators of TCP/IP), “Bill Joy just didn’t feel like this [the BBN code] was as efficient as he could do if he did it himself. And so Joy just rewrote it. Here the stuff was delivered to him, he said, “That’s a bunch of junk,” and he redid it. There was no debate at all. He just unilaterally redid it.”
The BSD team started shipping out 4.1aBSD with this new networking support because people wanted it. At the time, the networking part was hacked together. But very shortly users started sending in bug fixes and improvements to the networking stack. Based on this, the BSD team continued to tune it. This resulted in the release of 4.2BSD in August of 1983.
Eventually, BBN finished their TCP/IP software and sent it to Joy to add to the system. Joy refused, saying that they already had networking up and running. BBN continued to insist.
At a meeting about TCP/IP integration, Joy was asked how he came up with his networking code. He said, “It’s very simple — you read the protocol and write the code.” Rob Gurwitz, the BBN TCP/IP programmer at the meeting, disagreed with this story, but people who knew Joy said that this sounded like something he’d say.
Not too long after this, Joy left Berkley to start a new venture. As McKusick put it, “Bill’s response was to blow them off by going off to found Sun Microsystems.” He did return to do some work at Berkeley, but he was no longer steering the project.
The Network Test
With Joy gone, Mike Karels took over the project. He evaluated the finished BBN code and determined that the best solution would be to add “the good ideas from the BBN code into the Berkeley code base, but not to replace the Berkeley code base”. After all, the BSD code had actually been used out in the wild, and people were used to it. As a compromise, he offered to include the BBN code, as well. BBN was happy with this arrangement. After all, McKusick said that they need to justify the fact that they got paid seven times what Berkley did to do a third of the work.
DARPA however was not happy with the decision because it could lead to user confusion. They decided to send both TCP/IP implementations to Mike Muuss of the Ballistics Research Laboratory, as a neutral third party.
One thing that DARPA was interested in was how the networking stack would operate under military conditions. In other words, how well does the software handle dropping packets?
After several months of testing, Muuss releases his findings. He said that “the Berkeley code was more efficient but that the BBN code handled congestion better”. In fact, the Berkeley code handled all of the tests without any issue, but the BBN code would panic under some stressful conditions. Based on this, DARPA decided to keep the BSD code. Shortly after this report, 4.3BSD was released.
Like this article? Share it!
You might also be interested in
Getting expert FreeBSD advice is as easy as reaching out to us!
At Klara, we have an entire team dedicated to helping you with your FreeBSD projects. Whether you’re planning a FreeBSD project, or are in the middle of one and need a bit of extra insight, we’re here to help!
We continue our series of articles on the history of Unix with the events led to the creation of BSD. Find out about the first Unix editions, how C evolved, and how Unix was first licensed.
In his 1999 book In the Beginning… Was the Command Line, Neal Stephenson said the following about Unix: “Windows 95 and MacOS are products, contrived by engineers in the service of specific companies. Unix, by contrast, is not so much a product as it is a painstakingly compiled oral history of the hacker subculture. It is our Gilgamesh epic.”
Read more about how the story of UNIX actually goes.