WebAssembly from the Ground Up
wasmgroundup.comHi HN! Co-author of the book here, happy to answer any questions you have.
Beyond the sample chapters which are linked from the landing page, we also have a couple blog posts which may be interesting:
- A WebAssembly Interpreter: https://wasmgroundup.com/blog/wasm-vm-part-1/
- An older blog post, "A WebAssembly compiler that fits in a tweet" (https://wasmgroundup.com/blog/wasm-compiler-in-a-tweet), was also on HN earlier this year: https://news.ycombinator.com/item?id=42814948
It looks good! I may pick up a copy. Besides the convenience, why do you recommend your book over reading the WASM spec as you implement your basic compiler? I find that WASM has a beautifully readable spec. One of its best features!
Either way, I’ll likely buy a copy to support the hard work on a piece of tech that I am very fond of
> I find that WASM has a beautifully readable spec. One of its best features!
Disclaimer: I'm one of the guys whose face is advertising this book, as someone who bought the early access version and loved it enough to help a bit with proofreading.
I'm a self-taught programmer who essentially started from the lowest level with Z80 assembly on the TI-83+. I just wanted to know how to fit the bytes together directly without dealing with the rest of the toolchain.
I've tried reading the spec multiple times and what it revealed to me is that my lack of formal training in the subject matter is really holding me back here. I feel like I can follow 90% of it, but that doesn't matter really. It's the remaining 10% I don't understand that does.
The spec is written as a reference and gives you all the pieces, but doesn't really do a great job at fitting all the pieces together.
Everyone I know who does have some relevant background to compiler writing agrees with you though. So I think that for them it's obvious how to fit the pieces together.
Speaking for myself though, this is the first book that made the bytecode "click" as a whole.
Having said that, I think this book and the spec together are the real combo to go for. The book covers the core, and understanding that foundation makes all the extensions easy to grasp from spec alone.
Thank you!
I would 100% agree that the spec is quite readable. At the top of our Minimum Viable Compiler chapter, we say:
> The binary module format is defined in the WebAssembly Core Specification. You’ll notice that we back up many of our explanations with reference to the relevant part of the spec. One of our goals with this book is to convince you that the spec is a valuable resource that’s worth getting familiar with.
I think the spec is great as reference material, but we wrote the book to be more of a tutorial. We've talked to many people who say they've looked at the spec, but find it too overwhelming. For those people, we hope the book provides a good structure that ultimately helps them become comfortable with the spec!
Doing is a far better way to learn than just reading.
And is there anything you’ve done that has helped you learn WebASM?
Does it cover tail calls?
Edit: changed slightly to provide a more useful answer.
No, it doesn't — not this version of the book at least. We only cover WebAssembly 1.0.
That said, as my co-author says below, there's really not much to tail calls. Once you've worked through the book, you'd be able to grok tail calls pretty quickly.
As an aside — 2.0 was announced just a few weeks after we launched the book, and 3.0 a few months ago. And with 3.0 (which added tail calls), the spec has more than doubled in size vs 1.0, so it would be hard to cover everything.
We've talked about doing a new chapter to cover some of the interesting parts of 2.0 (e.g. SIMD), but covering everything in 3.0 (garbage collection, typed reference, exception handling, tail calls…) feels almost like an entire 2nd book!
Co-author here.
if you are interested in tail calls you just need to understand the call instruction which we cover in the book and then replace it with either:
- return_call <funcidx>, the tail-call version of call
- return_call_indirect <tableidx> <typeidx>, the tail-call version of call_indirect
More info here: https://github.com/WebAssembly/tail-call/blob/main/proposals...
Thank you both for the replies.
Wasm is such a cool technology. The spec though for me leaves a lot to be desired. Oh, the first few chapters are fine, but when you get to the binary and text formats that's when it all breaks down for me.
For whatever reason, Wasm loves OCaml. This wouldn't really be a bad thing if they didn't come up with their own custom language to denote syntactic elements of both formats instead of using EBNF or similar. I discussed this with them (because before this change they were using raw MathML for all the productions, and screen readers and MathML are... Erm... Hit and miss) and they noted that they needed an attribute grammar instead of just either BNF or an extension of it. So what they have now (SpecTec) is better than what they did have, and I like that I can now just open the raw grammar files and dive in. The problem is the way they chose to express it. And it could just be me, because ML languages (and functional languages in general) don't really come all that easy to me. (they're just... Really difficult for me to mentally follow, which is odd since I can follow most others just fine.)
Like with many things, the reference interpreter is in ocaml because one sufficiently motivated insider wanted it to be.
I think ocaml (ml-y languages in general perhaps) lend themselves to interpreters quite nicely. Similarly with Rust.
Maybe there's an intersection between PL nerdery and interpreter authoring, and I fall into that bucket and am biased.
Yes, I have had the same experience with the specification. It really is quite difficult to follow :c
Their SpecTec system is fancy and neat but I don't think that auto-generated specifications produce something worth reading. Perhaps in the future when there's less churn, there might be a hand-written specification? In the mean time I've needed to jump into their Discord to ask clarification questions about the high-level stuff. Once understanding that and the grammar conventions and the like, the specification becomes much more readable, though still not great.
Certainly nothing like an RFC. But maybe I have too high standards...
(It doesn't help that the syntax is *weird*. You've got your choice of an S-expression Scheme syntax or a stack-oriented ML syntax, *and* you can use both together. And there's at least one undocumented de facto syntax floating around AFAIK, though I believe the standard merged support for the main features it was used for, so hopefully test suites and the like will switch away from it at some point.)
Despite not being an ML programmer, I found the spec pretty easy to read for the most part. One of the least intimidating specifications I have ever read, surprisingly.
I wish I could say the same. I don't know what wall I'm hitting which causes it not to click for me, otherwise I'd go off and write my own Wasm interpreter just for the fun of it lol
I don't remembering submitting it today; it must be from the second-chance pool. Good to see my submission on the frontpage, though :-)
Yup, as suspected, I had submitted it 5 days ago [1], but here it shows as submitted 2 hours ago. But I don't see it in the second-chance pool [2], perhaps because it has graduated out of there to the frontpage.
[1]: https://news.ycombinator.com/submitted?id=gurjeet [2]: https://news.ycombinator.com/pool
Personally, I learned Web Assembly by reading through the spec. Can't recommend it more. It's extremely well written.
I agree, it really is quite approachable.
I'm not gonna lie, this Little course book thing is incredibly fantastic. Well written, well structured code. It really helped me to connect the spec to the binary format in my brain.
Also because it uses Ohm.js to write the Parser/Lexer from a BNF definition, almost 100% of the focus is on WebAssembly and how to compile it, instead of flexing and parsing.
Go buy it.
Just went and bought it!
I'm in a process where application-level programming isn't cutting it anymore (I still have a lot to learn, but it's in the diminishing returns).
I've been looking to understand the entire stack at a deeper level (from how requests are made to how they're parsed), and this seems like the next natural step!
Thanks a bunch!
Awesome! You're exactly the kind of person we were thinking of when we wrote the book…experienced programmers who are interested in understanding things at a lower level.
Let us know how it goes! You can find us in the book Discord, or email us at hello@wasmgroundup.com.
You might also want to look at this thread - https://news.ycombinator.com/item?id=45862591
WebAssembly seems like a big workaround for JavaScript only supporting doubles, strings, and objects/arrays. Its big features are to allow using a byte array as stack/heap storage memory, and having actual integer types, along with allowing C code to be compiled to WebAssembly.
Oddly enough, Zig is the nicest C to WebAssembly compiler I've used so far.
The "official" wasi-sdk is quite similar, a prepackageded clang toolchain to compile c/c++ code to wasi. What makes emscripten more complicated is the browser integration (eg compiling into a ready-to-run html/js/wasm bundle and providing C API wrappers for various web apis), and Zig also only manages the "compile to wasm" part but lacks Emscripten's JS interop features (that's why in my zig+wasm experiments I use the Emscripten SDK as sysroot and linker).
What the Zig toolchain provides that neither the wasi-sdk nor emsdk have is a package manager and build system which are both much nicer than what the c/c++ ecosystem offers.
Great work, would love to learn more about Wasm.
I can't help but notice that in the editor screenshots there's type information in *.js files.
Ah, good catch! I see them in one of the screenshots. Those are just inlay hints, they're not in the source code. (The editor is https://zed.dev)
What's the state of wasm for porting multi-process code?
I want to run something that execs a command line tool, both in the browser. Doable yet?
> I want to run something that execs a command line tool, both in the browser. Doable yet?
If it's not possible from Javascript, it's also not possible from WASM, it's as easy as that.
If your command line tool can be compiled to WASM and works within the restrictions of the browser sandbox, it's trivial. But if you want to start a native command line tool from within the browser, it's pretty much impossible (and for good readons).
There's also a grey zone if you need it to work in a specific runtime environment. For instance VSCode extensions allow to run POSIX command line tools compiled to WASI, and those can safely access parts of the filesystem (like reading and writing files in the current project directory).
I meant: I want a program compiled to wasm capable of running in the sandbox calling another command line tool also compiled to wasm.
Right now the call is through an exec system call, but that can be changed.
In VSCode extensions this is trivial, this is how you create the 'executable':
https://github.com/floooh/vscode-kcide/blob/main/src/wasi.ts
...and this is how you run it:
https://github.com/floooh/vscode-kcide/blob/2dfc621aade4a2be...
The asmx.wasm file is a vanilla POSIX cmdline tool which loads and saves files via fopen/fread/frwrite/fclose, and the tool has been compiled with the WASI SDK: https://github.com/WebAssembly/wasi-sdk
The resulting VSCode extension (https://marketplace.visualstudio.com/items?itemName=floooh.v...) then even runs in the VSCode browser version (https://vscode.dev/)
But AFAIK there's currently no easy way to get a similar easy to use WASI wrapper in browsers (it's definitely possible though because the VSCode browser version does it - VSCode basically has a filesystem abstraction which works for native filesystems as well as virtualized web filesystems like github repositories).
We have a research project called WALI (WebAssembly Linux Interface) here: https://github.com/arjunr2/WALI
It's experimental, but there is a toolchain that can compile most C/Posix programs and run them on the prototype implemented in the WAMR engine. And yes, exec works! In fact, we are able to run bash and Lua and memcached, among other things.
Neat! Let me take a look.
most of that is happening above the WebAssembly spec: https://wasi.dev/
More specifically for command line tools:
Is wasi .3 still planned to finish soon like the roadmap said? so wasi 1.0 is the the next goal?
wasi 1.0 feels like when it will take off after all these years of dev
timeline at the bottom of this page: https://wasi.dev/roadmap
See also https://wasix.org/
Check https://wanix.sh/ you may like it :)
As much as I love a free resource, I’ll happily pay for a high quality complete resource. Looking forward to reading this
Also a nice resource: https://developer.mozilla.org/en-US/docs/WebAssembly
yes, a nice way to contribute is to add the table instructions to the reference: https://developer.mozilla.org/en-US/docs/WebAssembly/Referen...
I like PHP because it allows access to core system calls on any platform.
I see runtime interpreters as constraining when a system call is needed, but proscribed.
You don't want to use PHP (a server-sided language) to solve a client-side problem.
I know a person who wrote Linux X Desktop Environment using PHP. Worked for them. It is general purpose programming language.
> [PHP] is general purpose programming language.
To be charitable, yes — PHP has access to low-level system details like the file system, sockets, and processes.
> I know a person who wrote Linux X Desktop Environment using PHP. Worked for them.
However: (a) "Worked for them" is an anecdote, not evidence of comparative suitability; (b) Don't confuse possibility with empirical fitness for purpose. Virtually all decisions are relative to alternatives [1]; (c) Even PHP describes itself as only a "general purpose scripting programming language" [2].
Note that "scripting language" itself can hide important differences. PHP 8 introduced JIT compilation [3] which helps.
[1] In negotiation terms, your BATNA (Best Alternative to a Negotiated Agreement). When evaluating technologies, don't forget the human cost, so consider your BATSHIT: Best Alternative To Shackling Humans In Tedium (or whatever expansion you prefer).
PHP devs: "hold my beer."
'Member when a major crypto exchange, which had original been a market place for Magic the Gathering cards (so it was not a mountain named Gox), was hacked and everyone's crypto stolen because the owner had implemented his own SSH server in PHP?
> I like PHP because it allows access to core system calls on any platform.
Lots of people _love_ PHP precisely because of the size of its attack surface.
Do you have any examples of something you've built in PHP which benefitted from direct syscall access?
I keep hearing about this. I occasionally use PHP8 and so far I'm pretty happy with it. Is there any resource that teaches about security issues with modern PHP (version 8.x)?
This is a great resource for learning WASM. I bought a copy and I'm enjoying following along all the code.
Is it possible yet to write a high performance concurrent garbage collector in WebAssembly?
Until wasm-gc was standardized in wasm 3.0 all garbage collected languages compiled their own garbage collector to wasm, so in a sense you always could.
Unless by high performance you mean something more specific.
With wasm-gc you get the GC "for free" (from the host), you can still write your own if you want but you loose the interoperability that wasm-gc solves.
Is there a Github repo for code associated with this book?
Yes, it can all be found here: https://github.com/wasmgroundup/code
Over the course of the book, we also build up a small library for creating Wasm modules and emitting bytecode; that's available as an NPM package (https://www.npmjs.com/package/@wasmgroundup/emit) and the code is here: https://github.com/wasmgroundup/emit
nice job