Show HN: Clx – Compile Lua to Native Executables Through C++20
github.comHi HN,
clx is an ahead-of-time compiler for standard Lua that generates C++20 and produces standalone native executables through GCC, Clang or MSVC.
The project started as an experiment to see whether modern C++ could be used as a portable compiler backend instead of LLVM or direct machine code generation. The generated code is then compiled and optimized by the host toolchain.
The latest release replaces the previous NaN-tagged value representation with a new shadow-types implementation, adds full int64 support, improves native arithmetic code generation and adds ARM64 macOS coroutine support.
Performance is typically much faster than the Lua interpreter and can outperform LuaJIT on some computation-heavy workloads while remaining fully ahead-of-time compiled.
The repository also contains graphical examples written in Lua, including a Pong game and a Mandelbrot explorer using a Sokol binary module (using the clx C++ API)
I'd be very interested in feedback on clx :) Does the backend output c++20?
If so, why? I’m curious what specific features you guys are using from c++20. Have c++20 coroutines been useful at all? Yes, Clx generates C++20. The main motivation wasn't a particular C++20 feature, but using GCC, Clang and MSVC as a portable optimization and code generation backend instead of LLVM or custom machine code generation. As for coroutines, no. Clx doesn't use C++20 coroutines because Lua coroutines are stackful, so the runtime uses platform-specific context switching instead. My understanding of the original question is: what motivated you targeting C++20 instead of e.g. C++17 My bad ! There wasn't a strong reason to target C++20 specifically. I simply choose the latest standard available at the time as started experiments. In practice, the code generator doesn't rely heavily on C++20-only features, so targeting C++17 would likely be possible with some adjustments. FYI https://github.com/samyeyo/clx/releases/download/v0.2.0/clx-... Extracts to current directory instead of clx-linux-x86_64 Good catch, thanks. The release archive currently extracts directly into the working directory. I'll package future releases into a dedicated top-level directory to avoid cluttering the extraction location. Anyone know if there’s still a community around https://github.com/snabbco/snabb ? This seems like something they would appreciate. Besides that, first use case that comes to mind is that gamedev likes Lua but iOS does not like LuaJit in JIT mode. Yes, game development is one of the use cases I had in mind. Since Clx is fully ahead-of-time compiled, it avoids the JIT restrictions present on iOS and relies on the platform's normal native toolchain. But Clx currently only supports Linux, Windows and macOS, including Apple Silicon. I haven't tested it on iOS yet, so I can't claim iOS support at this stage. Interesting project! I have not looked to closely at the source so forgive me for asking - Are you doing a source to source or byte code to source transformation?
- How are you beating lua performance, since the naive implementation of a dynamic language would require tons of tables with pointer chasing Thanks! - clx compiles directly from Lua source code. It has its own parser and C++20 code generator; it does not use Lua bytecode - The goal isn't necessarily to beat Lua, but some workloads benefit from the optimizations performed by modern C++ compilers. As for tables, clx doesn't use a naive boxed-object model. Tables have a split array/hash layout: integer keys are stored in a contiguous array, while other keys use an open-addressed hash table. That keeps common accesses cache-friendly and avoids a lot of pointer chasing. Thanks for the explanation! This is cool, it could be a neat way to get Love2d games on the web with wasm That would be interesting! Since clx generates portable C++, targeting WebAssembly through Emscripten is theoretically possible. The challenge is that Love2D does not really expose its backend as a reusable library. A more natural fit could be a game library like SFML or raylib, used as the base of a game engine provided as a third-party clx module. I'm currently learning to make games with love2d. I'm using love.js to deploy for web. Would this have some advantage over love.js? Probably not today. Love.js is already a mature solution for running Love2D games in the browser through WebAssembly. Clx currently focuses on ahead-of-time compilation to native executables rather than web deployment. The idea I was referring to is more about native distribution: compiling Lua code into standalone executables without requiring a Lua runtime on the target machine. That said, if Clx eventually gains a WebAssembly backend, it could become an interesting alternative path for Lua-to-web deployment. Neat, did you take any inspiration from Shedskin? It is a Python to C++ compiler. The perf numbers look decent. What optimizations are you the most proud of? Do you have plans for eval? what would stop you from supporting eval? I wasn't familiar with Shedskin, I'll take a look ! The optimization I'm most proud of is native type specialization when possible, allowing many values to stay as int64_t or double instead of generic slow Lua values. As for eval, Clx is fully ahead-of-time compiled. Supporting it would require embedding an interpreter in the runtime and use an interface with Clx... making it significantly larger and going against the idea of compiling everything ahead of time. For dynamic code execution, the Lua interpreter is probably the better tool. As a Lua fanboy, I have to say this is really, really great! I have not dug into it deeply enough yet - saving a deep dive for the weekend - but meantime .. what are your thoughts about having objc_msgSend included in the runtime so that native MacOS (and eventually iOS) GUI's can be made using this technique: https://medium.com/@michael.mogenson/write-a-macos-app-with-... Thanks! That's an interesting idea, but I'd probably lean towards a third-party clx module rather than adding Objective-C runtime support directly to clx. Since clx already exposes a C++ API and is able to link clx modules, projects like metal-cpp are a natural fit and don't require any changes to the runtime, or any FFI tricks. That keeps the core runtime small while still allowing platform-specific integrations. Ah, that makes sense .. so I could write a clx-oriented app which chooses to load the clx-objcsend module (based on its assessment of which platform its running on of course) .. which I could then use to formulate a native GUI on MacOS/iOS. I have a few GUI libraries I've written in Lua for other projects, which depend basically on having surface-manipulation and event-management functions from the native platform available .. I might have a go at porting these once I dive a bit deeper into clx. I've used other Lua-based engines in the past to accomplish a 'one codebase/GUI runs on all platforms' target in the past, but those engines have faded away - mostly due to the dependency on LuaJIT and the subsequent brickwalling of JIT in iOS, where the apps were making their revenue - so having clx at hand is an exciting opportunity .. assuming I can get the GUI code ported in a way that it is functional on at least the major platforms .. will ping you on GitHub if I make some progress on this, I've been meaning to bring the GUI code from those older projects into the modern context, and clx may well be the way to do it .. That sounds like a very interesting use case! This is exactly the kind of scenario where I think third-party clx modules make sense: keep the core runtime portable and let platform-specific bindings live outside of it. A clx-objc (or metal-cpp or any gui backend) module would be an interesting experiment, and the same approach could apply to other platforms (Win32, GTK, etc.) Clx is still a relatively young project, so please be a bit indulgent with bugs or missing features, but I'd definitely be interested in seeing/helping what you build! I'll join you in Github once I've got a chance to catch up with clx so far .. with the work you've already done, I think the basics are there to support multi-platform GUI modules .. but I'll know more about the effort once I catch up with you. Great work! Thank you! I'd be very interested in your feedback once you've had a chance to explore the codebase. Multi-platform GUI support is definitely something I'd like to see emerge through Clx binary modules and the C++ API. Feel free to reach out on GitHub when you're ready — additional platform expertise would be greatly appreciated. Interesting, will be pairing this with my Funnel setup and see how it goes. Fennel the lisp on Lua or something else? Way neat So the cases where you loadfile a file just for it to have a separate sandboxed _ENV that the function that created it, is flat out unsupported, forever? loadfile() isn't implemented in clx, and neither are the other dynamic code-loading features. Supporting them would require runtime code interpretation, which doesn't fit clx's current AOT model. Could that change one day? Maybe, but it's not a priority right now. I looked at your goals and Load is clearly out of scope. I think the question here is could you load your AOT compiled and executable code like we do with Lua scripts only give the compiled code access to a safe Lua env. i.e. Not giving it access to all symbols. That's an interesting idea. Loading precompiled CLX modules is a different problem though. In theory, a native module could be loaded and executed with a restricted environment, similar to how Lua modules receive a specific `_ENV`. It's not something CLX supports today, but it's much closer to the project's goals than runtime compilation of Lua source code. One challenge is that standard Lua does not provide a way to change a function's `_ENV` after compilation (unlike the old `setfenv/getfenv` mechanism from Lua 5.1). I've been considering adding environment manipulation capabilities to the CLX C++ API (while keeping Lua compatibility intact). If that happens, loading CLX modules into custom environments could become relatively straightforward. so a kind of setfenv type thing, but only from c++. That could work Yes that's it