Programming Network Systems - Systems Approach

7 min read Original article ↗

Regular readers of our books will know that we like to use code to concretely illustrate the concepts we are explaining. When asked to define “The Systems Approach” we often turn to the fact that our approach focuses on implementation of real-world systems. Finding a bug in one of our code snippets recently led to some reflection on the role that implementation plays in teaching computer science.


I recently fielded a Pull Request (PR) to our book repo fixing some bugs in a snippet of code that we were using to illustrate how the sliding window algorithm works. I recognized the snippet as coming from the x-kernel, a research OS that my students and I worked on at the same time Bruce and I were writing the First Edition. I went back to my copy of that edition to see if the bug had been there from the beginning, and was reminded that 1E included an entire Chapter 2 on Implementing Network Software, with that chapter drawing heavily from the x-kernel. 

The x-kernel was designed to facilitate implementing and composing network software—the rationale being that the whole point of an OS kernel is to provide a means to connect your computer to the rest of the world, so it should be optimized for that purpose—so leveraging that experience made sense at the time. But the x-kernel was not widely adopted, so our editor persuaded us to remove the x-kernel specifics from the second (and subsequent) editions. And Chapter 2 was reduced to a single section in the introductory Chapter 1. 

Returning to the PR, it seems that bugs were introduced in the process of removing x-kernel specifics. We turned the remaining code into generic snippets–effectively pseudo-code that borrows C syntax. Divorced from the x-kernel, these snippets did not actually get executed and tested.

I had forgotten how implementation-heavy the First Edition (published in 1996) was, which is all the more surprising when you consider that the books we’ve written recently—on SDN, Cloud Operations, Private 5G, and TCP Congestion Control—all draw heavily from implementations. Those books focus on concepts and principles, but they are all informed by systems the authors implemented and used. And further connecting the dots to the first edition, here is Jennifer Rexford’s Foreword to our Private 5G book:

When I was a graduate student studying parallel computing in the early 1990s, the World Wide Web exploded onto the scene, and I wanted to understand how the Internet made such rapid advances possible. I picked up a copy of the first edition of Computer Networks: A Systems Approach, and I started reading. I was delighted to learn not only the Internet’s history and main concepts, but also to see examples of real code that illustrated how the protocols actually worked and how they could be changed. I was hooked. I loved the idea of working in the computer networking field, where the technologies we build can help bring people together and lower the barriers to innovation so that anyone who can write software can contribute.

For me, then, the story comes full circle. Private 5G networks are something you can touch, code, and deploy yourself. Armed with the knowledge of how 5G access networks work, and with hands-on experience with open source software, just imagine the places you’ll go!

It’s nice, every now and then, to be reminded that others see value in your core principles, even when marketing data and anonymous online comments suggest otherwise. 

Programming III: Network Software

This whole experience got me thinking about an idea that’s been brewing in my mind for a while. The traditional CS curriculum starts with two semesters of programming, with Programming I introducing basic language constructs such as types, loops, and conditionals, and Programming II then venturing into modest algorithms and data structures. There are assorted third semester courses focused on programming—for example, introducing students to other languages they’ll need in their upper division classes—but I think there’s a strong case for using “Programming III” as an opportunity to introduce networking concepts, by having students write network software. 

For those of you hardened by curriculum battles, I ask for a short suspension of disbelief, and a consideration of what might be valuable content for such a course, independent of how you would realistically wedge even a subset of the topics into your current curriculum. You should also feel free to broaden the scope from networking to all of “Systems Software”, which is how I would probably implement the idea at Princeton—by adjusting CS217: Introduction to Programming Systems. This is an intro class typically taken by freshmen or sophomores.

I would build such a course around two central themes: (1) client/server communication and (2) packet forwarding. The first takes an end-to-end perspective of the network, establishes the Socket API as its cornerstone, and is straightforward to translate into a set of programming assignments. The second takes a switch-centric perspective of the network, establishes Match-Action Rules as its central programming construct, and could map onto a set of P4 programming assignments. Both themes provide ample opportunity to introduce many of the fundamental concepts of networking.

With that foundation, I can imagine two possible directions to go next: one focused on “protocol internals” and one focused on “assembled tools”. The first would expose students to manipulating headers, scheduling timeouts, demultiplexing incoming packets, and managing session state. It’s easy to see how this track aligns with traditional networking courses, with ample opportunity to talk about both concepts and implementation details. (This was the focus of Chapter 2 in the First Edition.) The second would explore the plethora of tools programmers use when building network systems (e.g., iptables/nftables, Linux bridges, Kubernetes CNIplugins, eBPF). This track has obvious practical value, but with the right framing, could also reinforce the fundamental concept of networking. This is especially true when viewed through the lens of “programming construct” or “design pattern” (e.g., these tools can be treated as specialized instances of the forwarding paradigm).

Of course, it’s far easier to throw out ideas like this than it is to construct a coherent course, but I think there are some important takeaways from such a thought experiment. One is that networking topics should not be isolated in a 400-level class that only some students will take. They are important enough that we should look for opportunities to teach them early and often. A second is that knowing how to program with send/receive and match/action constructs is arguably just as important as understanding, say, iterators or symbol tables. They should be treated as first-class programming capabilities. A third is that there is substantial intellectual value in understanding networking’s “tools of the trade”. Such topics should not be dismissed as simply “training” (any more than picking Python as your programming language in the first semester should limit you to “Python training”).

And last but certainly not least, there is a substantial movement to reclaim the open, distributed nature of the Internet from the centralized walled gardens. Teaching networking as an essential aspect of programming is an important part of that movement. In my view, we are doing a good job introducing students to the tools of analysis (of what already is), but it is important that we also teach the tools of synthesis, preparing them to create something new.


We keep finding articles from Cory Doctorow that are relevant to our own work, and this recent piece on the sorry state of security in large companies is a good one. It provides a counterpoint to our own optimistic view in the last newsletter, pointing to the structural problems that make good security hygiene take a back seat. We try not to be too envious of Cory’s productivity in producing a daily newsletter and a range of fiction and non-fiction books.

Another good read this week: A Eulogy for DevOps. The analysis of how DevOps didn’t magically fix all the problems of building, deploying and operating software systems resonates well with our own experience.