CoWasm: An alternative to Emscripten, based on Zig (demo: Python in the browser)
cowasm.orgI started this project and can answer any questions about CoWasm.
That's just incredibly cool, my congratulations!
Foremost, my apologies if this is a nonsensical question. I haven't been soaking in the WASM ecosystem enough to know how much WASM is "just" JS versus ... something else.
Caveat aside, I saw one of the commits mention jython, which notoriously has ancient (and probably incredibly incomplete) python 2.x support; do you know if python-wasm would run on top of GraalJS (https://github.com/oracle/graaljs#nodejs-support)?
Separately, do you want issues related to zython.org in the cowasm issue tracker? It returns 405 (method not allowed) over and over on POST https://zython.org/python-wasm-sw/read-signal for me
This is really cool, William! A lifetime ago, we were pondering what it would take to get Sage running in the browser from a static server. It feels like you're nearly there. Is that in the plans?
Yes, porting https://SageMath.org to the browser is part of the plan, and this is a key foundation for that. I ported some of the components of Sage (e.g., https://libntl.org/) already as part of https://github.com/sagemathinc/jsage, and I don't see any fundamental obstruction to porting all of Sage, except time. It would be great if once CoWasm is more stable, I can get some help porting some components to WASM.
It feels really fast and responsive. Have you done any benchmarks to compare against other Pythons-in-the-browser?
Or 'normal' Python for that matter.
With regards to differences with 'normal' Python, depending on WASM also means that despite having the same sys.get_int_max_str_digits() = 4300, my local Python can print out (2222*5555 + 5555*2222) in its entirety, while the browser based one can't.
Values that can be set via maxdigits in sys.set_int_max_str_digits(maxdigits) are also limited to size of int in C.
It’s basically the same speed as Pyodide for cpu bound work. It is overall about twice as slow as native Python on the exact same machine. There is a benchmarks folder in the CoWasm git repo with a several dozen benchmarks.
no questions, but please keep working on that. I'm very impressed with how smoothly the current cowasm implementation is working.
For the future do your have a plans to support WASI and other modern WebAssembly stuff (like WebAssembly Component Model support f.e)?
CoWasm supports WASI right now via this library https://www.npmjs.com/package/wasi-js, which I actually develop as part of CoWasm . One unusually thing I did, which goes beyond what emscripten does, is I implemented a quite a bit of posix functionality, often by writing extension code to nodejs and calling it from Javascript, because there's a lot of POSIX that Node.js doesn't expose. This only works on Mac and Linux and is also available standalone in this library https://www.npmjs.com/package/posix-node, which is implemented in Zig. You can get a sense of the scope of POSIX functionality that goes beyond what WASI defines here: https://github.com/sagemathinc/cowasm/tree/main/packages/ker...
One motivation for doing this is to try to get the full Python test suite to pass, including all the functionality that involves subprocesses, posix calls, etc. I've only got to about 85% at this point. It can be a ton of tedious work, but at least Zig helps impose some discipline (e.g, it doesn't let you ignore handling errors until later), and makes it easy to test compilation for all supported targets on every change (due to excellent cross compilation support).
What pain points with emscriptem do you want to address? What are the trade-offs you’re considering?
Is this really an alternative to Emscripten? It the README makes it seem 100% Python focused. I want to switch from Emscripten to a more maintainable, modulare WASM build tool - how would I go about porting a simple C->WASM w/ Typescript library project to CoWasm? Does CoWasm support asyncify?
I am using the Python ecosystem (with full support for dynamic loading of C extension modules) as an initial motivating project. Also, the Python test suite is extremely useful to root out problems. I certainly hope that this can provide a more complete alternative to Emscripten to the community eventually. That said, Emscripten is huge, and the problems involved in creating a more maintainable modular WASM build tool are subtle. For example, when implementing a custom module loader for Python-wasm last week, I discovered several bugs in the memfs and unionfs Javascript libraries (https://github.com/streamich/memfs/pulls?q=is%3Apr+author%3A... and https://github.com/streamich/unionfs/pulls?q=is%3Apr+author%...). I had to learn the code sufficiently to fix all these bugs, submit PR's, etc. Emscripten has its own analogue of memfs, which is optimized specifically for WebAssembly in the browser, where memfs is a more general widely used library (with 10M+ downloads/week).
CoWasm has no support for asyncify. Where I've run into setjmp/longjmp so far, I've been rewriting the code instead. E.g., the dash shell uses setjmp/longjmp, and I'm rewriting that to use return error codes instead (see, e.g., https://github.com/sagemathinc/dash/commit/7117e1f6496728af0...).
> how would I go about porting a simple C->WASM w/ Typescript library project to CoWasm?
That's a great question, which I'm not sure how to quickly answer, so I've created a discussion item here https://github.com/sagemathinc/cowasm/discussions/40 I did a massive refactor of the codebase during the last month, and it makes doing what you want much easier. However, it will help a lot once there are some good examples. I'm probably boing to individually package some of the things I've ported for CoWasm as standalone examples.
In any case, CoWasm is extremely open source (BSD 3-clause), and I strongly encourage anybody who is interested to grab a copy, build it, poke around, make a PR, etc.
Thanks for all the effort you have put into this. It is pretty cool and enormously valuable to have an alternative to Emscripten
This is a slim alternative to Emscripten which focuses only on the C/C++ <=> JS interoperability part:
https://github.com/schellingb/wajic
I don't think there are actual Emscripten replacements though which give you all the features, but in a 'cleaner' package. Emscripten does a lot on top of clang/llvm. A lot of the 'Emscripten magic' has been moved into the Binaryen tools though, so it can be used by other toolchains (like wajic): https://github.com/WebAssembly/binaryen.
It's great that eg asyncify is upstreamed to binaryen/LLVM. My issue with Emscripten is that it's quite challenging to fix bugs in the wrapper Javascript code it generates. I spent a few nights trying to understand how to change the compiler output so I could import the Emscripten result as an ES6 module in both a browser and Node 16+. The way this works is unfortunately 1000s of lines of `if` statements that decide whether or not to concatenate this or that helper into a file, and all the little helper snippets may or may not call into each other. Some of the concatenation is in Python, some of it is in Javascript. I eventually gave up without making any upstreamable progress.
Relevant github issue: https://github.com/emscripten-core/emscripten/issues/11792
A quote from that thread:
> Btw, this issue shouldn't be tagged good-first-bug, because fixing it will require a substantial overhaul of how Emscripten sets up its imports and organises its startup code. There are several ways it can be accomplished, but it really needs direction from one of the project leads.
It doesn't seem like there's any progress here :(
Not yet, and it's not clear that's the goal, exactly. Up near the top, the README says CoWasm "goes far beyond just Python". Then later: "The goal of CoWasm is overall similar to all of emscripten, WebAssembly.sh, wapm.io, and Pyodide in various ways." But the "How does CoWasm compare to Emscripten and Pyodide?" section doesn't mention Emscripten.
Thanks. The docs could use some work! (https://github.com/sagemathinc/cowasm/issues/41)
Duplicate of this?
looks like a duplicate, but it has no discussion. I'm not OP of either of these but I'd love for this to get to the top to foster a discussion about their approach (which seems exciting)
Some of the interesting goals from the repo: https://github.com/sagemathinc/cowasm:
- There is no business unfriendly GPL code in CoWasm. CoWasm itself is extremely liberally licensed and business friendly.
- Unlike emscripten, we use modern Typescript, our code is more modular, and we make use of existing components when possible (e.g., the nodejs memfs project), instead of using our own.
- A core design constraint is to efficiently run on a wide range of platforms, not mainly in the browser like emscripten, and not mainly on servers like wasmer. CoWasm should run on servers, desktops (e.g., as an electron app), an iPad/iOS app, and in web browsers.
> There is no business unfriendly GPL code in CoWasm. CoWasm itself is extremely liberally licensed and business friendly.
This is one to save for posterity, when they eventually change the license because the likes of Amazon decided to make a product out of it.