PlanD: Plan9 from Containerspace
github.com> One of these fundamental ideas was the notion of a system composed of microservices communicating via a well established system of network API requests encoded as 9P messages.
I'm still reading but this strikes me as a misservice.
Messages I recognize are lower in the abstraction stack & this isn't an inherently incorrect statement. But I think it mispercieves the nature of 9p. 9p is a remote filesystem, that so happens to be built out of messages. The author here emphases operations- messages- but 9p is really what's in those messages, what the messages represent, which is structure & state.
The 9p spirit isn't about sending messages. It's about exposing your well structured state to the world. Rather than as most programs do hiding it close to your chest... and only exposing complex artisinally-homegrown inter-face abstracted atop that.
If everyone just exposed their state well, that'd be great.
What I think I'm reading here is that the abstraction is misplaced.
How then would one leverage 9p filesystems to solve the containerization problem?
I've had vague intuitions that Plan9 would have elegantly solved some of our problems that Docker, Terraform, Kubernetes, etc. were all created to be solutions for. But I'm unsure as to the proper shape of the solution.
> How then would one leverage 9p filesystems to solve the containerization problem?
In Plan 9, processes are sandboxed by default, so they only see what is exposed to them. You utilize mounts to attach filesystems to processes, and since all APIs are done through the filesystem, you can trivially wrap/extend/replace those APIs with different implementations.
Essentially, everything-is-a-filesystem + 9p gave you a kind of capability-based security.
The sandboxing is pretty important to keeping things isolated, but I've also wondered about the right way to ensure my writes to files are "atomic" in 9p, and never really understood the guarantees or lack thereof.
In POSIX-land, I can write to a temp file, sync it and the parent directory, and rename it over the original and be pretty well guaranteed anyone accessing the file by name will never see a partially written to file.
How the heck does this work in 9p servers? For a "synthetic" file server one could decide to update the state depending on receiving full payloads I presume, and discard anything that's nonsense or incomplete. (like changing the baud on a serial port through the control file - you need to know the format of the commands and send the correct payload)
But what does one do about regular files? Write to temporary files and rename over them? Do the Plan 9 file systems have any atomic guarantees (HJFS, Fossil, KenFS etc?) and do I need to understand each of them?
I reallyike both csande17 and smasher164's comments about capabilities.
You question I admit left me a little stuck. Seeing their answers provided some clues.
I think there's a bit of a distinction, that Docker, Terraform, Kubernetes tend more to be for a simpler part of orchestration: summoning resources.
Rather than figuring out what to beget, plan9's universe was more about figuring out what context to spawn resources (effectively their own chroots with some volumes) in. I think that is more the clear lesson here.
There's semi orthogonal concerns. So many of modern tools help us spawn resources, but figuring out what capabilities/affordances to expose to them is actually quite a bit back from where we've already tread.
It turns out exposing state as a filesystem, then mounting subtree of state as volumes in other apps is a quite composable powerful general purpose paradigm, frankly surpassing what we are up to now. And we already have some very nift weird volume mount stuff going on in kube, but it's still not this comprehensively useful: apps just don't make a practice of exposing their state in the universal way (as a filesystem) as the 9p practice did.
The version of this idea that I've heard is that, in the idealized Plan9 "everything is a file" universe, the only containerization primitive you need the kernel to provide is chroot. Then you can implement things like network namespaces entirely in userspace, as filesystems that get mounted at the appropriate location in the container.
> How then would one leverage 9p filesystems to solve the containerization problem?
Well, there's:
https://wiki.qemu.org/Documentation/9psetup
Then again, there's also an alternative citing overhead with 9p being a network protocol:
> the general "zeitgeist" of the software developer community has enthusiastically moved towards fully distributed, eventually consistent systems
For their sake, as well as of the rest of us, I really hope they turn back from the cliff.
As far as general uses of network topologies and system architectures are concerned, there is no advancement I am aware of in the past 10+ years that has become a "better" topology or system design than the rest. There is "newer", there is "different", but not "better". Fundamentally all the designs we have are from the early 2000s or before, with some newer algorithmic pixie dust that hasn't changed the end result substantively. In the end you still need to pick a design based on the purpose and use case.
Plan9 was an excellent beginning design, and I would argue it's due to its combination of simplicity and loosely-coupled interfaces. Distributed or not, consistent or not, its beauty was in how it made certain functionality easier. You could do some things that used to be crazy hard, in a much easier and simpler way. It's that end result, not some theoretical design principle, that made it so interesting.
Containers are an overcomplicated abstraction (and they're not even really "one thing", they're more an idea). The operating system should provide primitives and first class features that make it easier to run what we today call a containerized application. Kernel devs want everything pushed into user space, and that has worked okay, but it has also made a bit of a mess. The Kubernetes ecosystem only exists because the kernel and core system utilities won't do what we want it to do out of the box. We are still living in a world of virtualized hot-rods, cars that have been chopped up and had new parts bolted on to turn it into a Frankenstein that approximates what we really want.
It would be nice to go back to something like Plan9; something so stupid simple and universal that everything becomes easier. But it doesn't look like that will happen anytime soon. Maybe in another 30 years we will have rewritten all the code so many times people forget about Plan9 and then accidentally reinvent it, and it will hit HN, people will think it's new and cool, and the hype cycle will begin. I'll be 70 years old. Maybe the kid who'll create it was born today.
(also, mods: please add (2020) to the title)
The defining featureset of K8s is orchestration/provisioning/control, not mere containerization or even distribution over a network. Plan9 addresses the latter in a rather clean and elegant way but says nothing specifically about the former. The Linux kernel has grown namespacing features and user-space implementations of kernel-side API's that are, on the whole, quite reminiscent of Plan9 already. Of course, this has been done in a rather ad hoc, inelegant and arguably incomplete fashion.