Sanstun: A sans-IO, parallel-friendly WireGuard implementation in Rust
firezone.devIt was all going great until I read
> The design also avoids problems like function coloring and plays well with Rust's ownership rules.
Async is not I/O. It can be used for I/O, but it's just a way of writing pure, deterministic state machines that keeps a logical flow together instead of scattered over a bunch of transitions.
Even if you choose to manually compile your async flows into explicit state machines, you still have to deal with ‘function colouring’ (a.k.a. typing): a flow that crosses several state transitions (and may have other disjoint parts of the state updated in the meantime, via other transitions) now has to be rewritten as a (series of) state transitions, and can't be called from a blocking function. If anything, rewriting all your state machines without `async` makes the function colouring ‘problem’ worse, because multi-transition flows are even more different from normal functions.
In principle, you are right. Async is not I/O. And yes, `async` functions compile to a deterministic state machine.
The problem is, the moment you use `async`, your state machine reduces down to one API: `poll`. What if I want to change some state in-between two state transitions (i.e. between two `await` points)? Now I have to wire-in a channel to supply that data at the right point or access some shared data via a Mutex.
Rust's `async` implementation is great if you know everything upfront before you kick off your state machine. The moment you need to do many things concurrently, it can become a real mess.
Firezone's sans-IO core handles many things concurrently: inbound IP, inbound UDP, WebSocket control messages, TURN allocations, timers, user commands, etc. Having essentially one big struct that I can just call `&mut self` functions on for any one of these inputs makes this very easy to understand, compose and test.
A few things like TURN request-response handshakes might be easier as an `async` function but in reality, it is not a big deal and the composability you get from it outweighs it in my opinion.
> What if I want to change some state in-between two state transitions (i.e. between two `await` points)?
I think as phrased what you said is an _advantage_ of the Rust model: the state of your state machine is entirely contained within the dedicated state-machine type and you know its behaviour can't depend on surprise outside state.
But I think the spirit of your criticism is correct: you do really want to be able to type processes with intermediate inputs and outputs, in general, and Rust makes this hard (though not impossible: see Munksgaard's work on session types in Rust, for example). I would not have had a problem if the original article had decided that Rust's futures are unfit for its purposes, and it would have been an interesting case study in what exactly real-world code needs from the API; my complaint is just that they get categorized as a type of I/O, which couldn't be further from the truth.
ha - wh33zie and I have had this discussion before about sans-io on hn at https://news.ycombinator.com/item?id=40872020
Isn't this the idea behind generalizing `async` to `Generator`? You can set the `R` type parameter to whatever type you want to resume the computation with.
Correct! I played around with this in the past. It helps less than you'd think. As soon as you have a generic resume parameter, you can no longer drive the generator with a generic runtime.
If I am now writing my own runtime / eventloop to advance my state machine anyway, what exactly am I getting from doing that through an abstraction like the Generator trait?
On top of that, the resume and yield abstraction can get convoluted because a lot of times, you only want to do one or the other, meaning either case needs to support a no-op because a single `yield` always does both.
Lastly, with all input going through a single type, your state machine then needs to handle every possible input at every yield point. That is a lot of `match` statements and branching.
I am sure there are better ways to do this and having generators to express custom iterators would be amazing. But for generic state machines, I don't see much of a benefit.
I think at the end of this long road lies algebraic effects. You can write your computations with typed effects and, once you've handled every effect (or all but a finite subset, if you want to leave some to the runtime), you can drive your computation generically. Because the effects are typed, you get to narrow the types of the effect arguments at each yield, so you don't have to handle every possible effect at every yield (on either side of the channel).
Nit: the relevant feature is now called `Coroutine`, not `Generator`
> Async is not I/O. It can be used for I/O, but it's just a way of writing pure, deterministic state machines that keeps a logical flow together instead of scattered over a bunch of transitions.
I argue that it is exactly opposite. Blocking threads are a much more natural way of thinking about concurrency.
Quote from the erlang creator Joe Armstrong's thesis. He argues that the only way to understand multi-threaded programs is writing strictly sequential code for every thread and not muddling all the code for all the threads in one place:
"The structure of the program should exactly follow the structure of the problem. Each real world concurrent activity should be mapped onto exactly one concurrent process in our programming language. If there is a 1:1 mapping of the problem onto the program we say that the program is isomorphic to the problem. It is extremely important that the mapping is exactly 1:1. The reason for this is that it minimizes the conceptual gap between the problem and the solution. If this mapping is not 1:1 the program will quickly degenerate, and become difficult to understand. This degeneration is often observed when non-CO languages ["non concurrency-oriented", looking at you JavaScript!] are used to solve concurrent problems. Often the only way to get the program to work is to force several independent activities to be controlled by the same language thread or process. This leads to a inevitable loss of clarity, and makes the programs subject to complex and irreproducible interference errors." [0]
Your comment is phrased as a rebuttal, but I actually agree with it wholeheartedly, with one minor quibble.
> Blocking threads are a much more natural way of thinking about concurrency.
The (ideal) programming model for writing a thread vs an async task is identical, and indeed the most natural way of writing sequential flows that may incidentally be temporally separated. A blocking operation and an await are equivalent (in the absence of shared memory; in its presence they differ in that an async task on a single thread gets to assume atomicity of the code between awaits, but that's not too relevant here). I think what Joe is arguing against here is exactly splitting up your business flows into a bunch of explicit state machine transitions.
The difference is only in the caller. Callers of async tasks (à la Rust) have the right and responsibility of controlling how (and whether) the task they create resumes, while threads/BEAM processes (and ‘stackful’ futures à la Go, JavaScript, etc.) are answerable only to the great global event loop in the sky (sometimes known as the OS scheduler or the BEAM multiplexer, etc.). This makes it easier to avoid bugs that are about forgetting to drive a task, but harder to avoid bugs that are about forgetting to stop a task, or stopping a task wrongly; see structured concurrency. It also hurts composability a bit, because by construction there can be only one event loop and there's no way to decompose it.
You can see Erlang (or I suppose we should talk about BEAM more generally now people like Elixir) in terms of OS processes as an analogy, but it's just as valid and perhaps a little more accurate to think about it as a pool of async tasks with a global event loop, à la Go, where all blocking-looking code is actually async. It shares a common problem of such languages that because it can't interrupt real blocking code, e.g. FFI, you have to be very careful not to accidentally starve your other processes with such a call. This is made more obvious (but not really fixed) by languages that colour async functions differently, because the async colour acts as a promise (sadly not enforced by the language) that the function will be a good neighbour and not block for too long (whatever that means).
It's interesting that we've gone from cooperative to preemptive to cooperative multitasking again.
I agree that blocking threads is generally more natural / obvious. Cooperative multitasking was always done for performance reasons and more recently for simplicity (in single thread implementations with no locking) and/or scaling reasons.
Message passing concurrency is going even further though, and never really seems to have taken off wholistically. Again, primarily for performance reasons. When we do pass messages (across networks, too/from RAM or disk, etc) we usually add caching and other optimizations which make concurrency even harder to reason about.
Message-passing is popular in HPC (e.g. MPI) because it more closely models how hardware works. But at a higher level people mostly prefer to build abstractions on top of it like async/await because message-passing requires you to manually compile your flows into per-process state machines, and that's not how most people think about flows (even though business people do sometimes: https://link.springer.com/chapter/10.1007/978-3-540-72863-4_...).
I'm no stranger to this but there is nuance. Thoughts on my other comment below?
The programming model and the implementation are not 1:1. This thread is mostly about the model; we can talk about implementation below :)
Message passing on virtual threads on the JVM allows you to run millions of threads and it has no performance downside to async.
Every copy is a performance downside. There's plenty of applications where it is essential to use shared memory. You could not rewrite Linux to avoid shared memory between threads.
The line is fuzzy though.
You might say that each RAM access is really a message over a bus, and so is each cache access. Conversely, you could write a message passing style key value store and create race conditions by using like shared memory.
Transactional memory is also 'message passing' like -> you bundle up access of shared memory into a single transaction 'message'.
A lot of shared memory code uses constructions like FIFO/queues which are really not too distinct from message passing except when ownership is shared or goes back and forth. Perhaps that's just an 'implementation' of message passing.
Message-passing doesn't necessarily involve naïvely copying the data around. If you have access to shared memory then you can pass a pointer; the only thing you really need to ‘pass’ is the capability to access the message data. This is how seL4 does it, for example.
Of course if you don't have shared memory then you have to copy anyway :) And because of CPU caching architecture the benefits of having an owned copy of the data can sometimes outweigh the cost of copying it, even if shared memory is available.
Ok if you define message passing that broadly then I would agree, but I think that definition is also broad enough to bring in the downsides of shared memory such as deadlocking.
We're also now talking about patterns which are common in other contexts, but I don't have a problem with that. io_uring for example - is it shared memory, asynchronous, message passing, or all three (probably yes)?
> wholistically
You made me curious about the spelling of 'holistic', given that it really is based on the concept of wholeness. It turns out:
- Holistic is formed from the Greek root hol-, meaning "whole". Like hologram.
- It's even closer to the English than it looks; modern whole descends from Old English hal (cognate to the Greek word), which also gives us hale and health. The W in the modern spelling is a spurious addition unrelated to the etymology of the word. You'd expect "hole".
- Who is different, and the W in that word is etymologically correct. (Old English hwa.)
> flow that crosses several state transitions (and may have other disjoint parts of the state updated in the meantime, via other transitions) now has to be rewritten as a (series of) state transitions, and can't be called from a blocking function
This is incorrect. You can easily call that flow from a blocking function, just call it in a loop.
What normally prevents a blocking function from "simply" calling an async one is that the async function needs runtime support for certain operations: the simpliest of which is suspending until it's notified, with more complex ones being async I/O primitives.
A Sans-IO library needs none of these. It can simply be called in a loop and as long as the caller provides more inputs it will make progress. An blocking caller can simply call the std's `Read::read` and pass the received bytes in, while an async caller can use async I/O primitives to do the same, and since the Sans-IO library yields once it finishes consuming the current input it won't block the async library either.
It's the best of both world but as always you have a tradeoff, and in this case it's the ability to express the state machine as if it was a straightforward function. Maybe once Rust gets support for generic coroutines this will get better.
> This is incorrect. You can easily call that flow from a blocking function, just call it in a loop.
Sure, the same can be said of an async library — so long as you're okay with none of your other flows being able to make progress in the meantime. You can just poll in a loop.
> It's the best of both world but as always you have a tradeoff, and in this case it's the ability to express the state machine as if it was a straightforward function.
‘The best of both worlds’ means that it shares the advantages of both, but the main (arguably only) advantage of async/await is that you get to write your temporally separated flows sequentially. (And maybe also that they look a bit like functions, I guess.) That's not true here.
> A Sans-IO library needs none of these.
To be clear I'm not by any means arguing that ‘sans-IO’ (or, as we used to call it, pure functional programming) is a bad thing. I'm very in favour of it. I'm just saying that a ‘sans-IO’ design shouldn't need to eschew async/await, which you can write in terms of pure functions.
(As a sibling comment points out, there are languages where that is not true, i.e. async/await inherently introduces some kind of side effect around concurrency or a global event loop. Thankfully that's not the case in Rust.)
Without actual code solving an actual problem discussing this is all a bit wishy washy, but I think you're mixing 'levels'.
The sans-io design here is about building reusable parts that any runtime or thread can drive and incorporate into the colored interface they expose.
Function coloring is about (the annoyance when) composing interfaces designed around those different runtimes/thread that might use the same sans-io library internally.
Yes async compiles down to a statemachine, but we're talking about the elegance of the language syntax/composition. Most languages require _a lot_ of ritual (or overhead) if you want to use the async-statemachine as a pure statemachines and sidestep the build in runtime-side-effect channels.
> Function coloring is about (the annoyance when) composing interfaces designed around those different runtimes/thread that might use the same sans-io library internally.
No, it's about the difficulty of calling async functions (or functions with other typed effects that can't be handled locally) from pure functions (or more generally functions that aren't themselves typed with that effect).
> Yes async compiles down to a statemachine, but we're talking about the elegance of the language syntax/composition. Most languages require _a lot_ of ritual (or overhead) if you want to use the async-statemachine as a pure statemachines and sidestep the build in runtime-side-effect channels.
That's true, which I would consider a limitation of those languages. Thankfully Rust futures are exactly state machines; there are some rough edges on their API still, but I'd have preferred a discussion about those to throwing the baby out with the bathwater and claiming that those state machines are a type of I/O.
Sweet, this is basically how I design my protocol implementations in rust as well.
Way more testable, the bulk can be shared between async and blocking IO, it's a lot easier to completely ban unsafe in your dependencies (excepting std and core), etc.
Mullvad spun their own implementation off of boringtun as well (called gotatun[0])
One of the fundamentally great things about the Wireguard protocol is the simplicity. A small codebase can implement it, and that’s lead to some great libraries and innovation.
Another great (and subtle) thing is the monotonically increasing sequence number. Note that it isn’t incrementing, like DTLS. This small thing makes it so much easier to implement with efficient concurrency, and event external state (i.e. distributed concurrency).
That latter bit we’ve done in our own library.
Edit: typo, don’t -> done
I've recently been developing a VPN Manager implementation using Rust, and the underlying architecture also uses WireGuard. Currently, I'm using WireGuard gotatun developed by Mullvad.
I wonder if you've tested any related data, such as performance curves, connection jitter, and so on, as well as things like security implementations.
From their website about their product:
> Hardware does the heavy lifting.
> Your operating system and network card take over the repetitive jobs whenever they can. Your CPU gets room to breathe, and your battery stays full longer.
Where does this operating system run? Where does the power come from to run it if not battery?
Pure rubbish ai slop.
While the software might actually be ok upon inspection but this kind of ai slop nonsense copy doesn’t inspire confidence in using their software considering how security critical the vpn is.
There is actually more thought in it than that.
The quoted section is meant to explain in layman's terms that we are doing hardware offloading where possible. That means vectorized instructions for encryption and checksum-offloading and packet segmentation in the kernel / hardware driver where supported.
Being more efficient per packet means that we using less power to move the same amount of data, hence less power draw on mobile platforms.
Having said that, I agree that it could be better while still not be filled with technical jargon.
I don’t think it has much to do with technical jargon. You are making a product for technical people, jargon is fine.
You should not write things with AI that you expect people to read
The quoted section makes sense to me. It’s like video decoding: the more you can do in hardware, and the fewer user mode / kernel mode switches, the less CPU you use and the less power you consume.
I imagine they’re talking about handling network protocols in the kernel or the NIC instead of in userland code, and they’re generally right. It runs off the same battery, but the kernel implementation and especially the (hardware-accelerated) NIC implementations of protocols like TCP and TLS are generally more energy efficient than doing the same amount of IO in userland code.
I think the point is not if the text can be interpreted charitably, but that the usage of AI text in what is intended for humans to read causes mistrust about where else corners could have been cut, on what is security-critical software
And even the charitable reading is not a good indicator. There is reason things are getting pulled out of kernel and more and more userland services are used, it is because we have finally come to terms that secure software is hard and better hack as little of it trusted with the whole system as possible.
I've wanted Wireguard on some MCUs. Great!