A brief introduction to BEAM
blog.erlang.orgThe JIT in OTP 24 mentioned at the top was discussed in https://news.ycombinator.com/item?id=24441841
I am looking forward to more blog posts about it.
A few interesting snippets from the PR comments from it:
---
[...] number of Estones as computed by the estone benchmark suite becomes about 50% larger, meaning about 50% more work can be done during the same time period. Individual benchmarks within the estone benchmark suite vary from a 170% increase (pattern matching) to no change at all (huge messages)
[...] If we run the JSON benchmarks found in the Poison or Jason, BeamAsm achieves anything from 30% to 130% increase (average at about 70%) in the number of iterations per second for all Erlang/Elixir implementations.
---
I’ve heard that errors in native code called via FFI in BEAM code is the primary place where the runtime can behave uncharacteristically. I haven’t seen anyone in the community describe it this way but it has the feel of undefined behavior to it.
There’s apparently some heightened interest in Rust for these purposes because its guarantees offer some additional robustness here.
But having JIT for the BEAM changes the calculus of whether it’s worth maintaining your code in two languages for a speed improvement that comes with additional liabilities. It will be interesting to see what libraries and tools become pure as the JIT covers more territory.
When you call into native code, you're explicitly opening the door to letting whatever happen happen. It's not undefined, it's a clearly defined escape hatch to do whatever. There's rules to follow, but it's the honor system, BEAM doesn't care if you do or not, but you'll have to live with the decisions you make.
Calling into native is by definition the "unsafe" code block of managed languages, even if you are calling into Rust, unless you explicitly validated the code of the dependency being loaded.
There are zero guarantees that the Rust code is correct (free of logical errors), or makes use of unsafe in ways that are properly correct, and given cargo, you also need to validate every dependency that is brought into the shared library.
The only guarantees are that it will abort/panic on bounds checking, numeric overflows and whatever unsafe code it has (regardless of its correctness), is explicitly marked in unsafe code blocks.
It's super exciting, but the JIT in its current form doesn't offer the kind of speedup that something like FFI into Rust or C provides. Also there are lots of great libraries in the Rust/C ecosystems that sometimes you need. Fortunately FFI is not hard, and with the relatively recent addition of dirty schedulers and with the Rustler library, it doesn't block the schedulers and it is safe, which were the two biggest issues.
There are two problems. Firstly, you don't want a crash in C code to take down your reliable Erlang VM. Secondly, external code has to play nice with the reduction counter and Rust doesn't offer benefits over other languages in this regard.
well, this is pretty timely. I have been peeking at some of the opcodes in the BEAM and some of this stuff I had gotten wrong; since there isn't AFAICT easy-to-find documentation on how this works. Does anyone know of other resources along these lines?
I'd highly recommend "The Beam Book" by Erik Stenman: https://blog.stenmans.org/theBeamBook/
A few years ago, I spent my free time building a Haskell BEAM codegen library, and that was by far the best resource I found. I noted some other projects that helped me understand things here: https://github.com/kofigumbs/codec-beam#acknowledgements
I'm familiar with the beam book but iirc it's incomplete, in particular in terms of what OP speaks of :|
I was surprised to read that the BEAM has no concept of processes.
I assume that the BEAM has some primitives to do with reduction counting and pre-emption that can be used to implement the full erlang process, but the BEAM book (https://blog.stenmans.org/theBeamBook/#_concurrency_parallel...) isn't very clear which bits are in the C code (the BEAM) or the Erlang code (the ERTS).
In the exception example, there is a line saying:
{'try',{y,0},{f,13}}.
Because 'try' is also a keyword in Erlang. The other symbols like that without the quotes are atoms (http://erlang.org/doc/reference_manual/data_types.html#atom).
So to have an atom named after a keyword we'd have to add single quotes around it.
Yes, got it - thank you for the response.
Has anyone seen benchmarks of the new JIT vs HiPE?
Everything I’ve read is comparing the JIT to the interpreter, but HiPE itself is faster than the interpreter. So is JIT faster than HiPE?
According to the recent Thinking Elixir podcast [0] the performance is comparable. They specifically cite the performance of dialyzer, which typically runs through HiPE, as being just as fast with the new JIT.
From what I recall HiPE isn't widely used and will be (may be?) removed.
[0]: https://thinkingelixir.com/podcast-episodes/017-jit-compiler...
I wish there was a python implementation that ran on BEAM.
Here’s a reasonably exhaustive list of languages that do:
https://gist.github.com/macintux/6349828#alternative-languag...
Luerl is probably the closest thing in terms of semantics, but Elixir might feel right in terms of Syntax. Actually running Python on the BEAM would probably be pretty challenging based on what I've read about the GraalVM python efforts.
There is efene (https://efene.org) which claims to embrace the ZEN of python.
go for elixir. the differences compared to python are minuscule.
Python and Elixir could not be further from each other. Just for starters, Python is OOP and Elixir is FP, which is a far bigger difference than you seem to imply. Elixir is compiled while Python is interpreted. Python has absolutely incredible community package support while Elixir is much more niche. It's hardly miniscule. It's a completely different way of programming.
I think the parent comment is referring to the philosophical similarities. Both languages focus on constrained and careful language design, good standard libraries, friendly human readable syntax, and saying no to requests that don't fit.
OOP and FP are really not all that different anymore. I joke it's become like risc/cisc: OOPLs (including python) these days have lambdas, map, and reduce... Elixir has inheritance (behaviours) and polymorphism (protocols).
What elixir doesn't have is pass-by-reference. Which, imho, is a huge win, unless you are going matrix math.
Polymorphism is not an OOP concept. Behaviors are like interfaces, not inheritance. Behaviors allow dynamic dispatch but let's not try to fit OOP where it doesn't belong.
When people talk about how similar are OOP and FP languages they miss the elephant in the room - immutability.
Maybe my PL history is wrong, but weren't interfaces made popular by java as a way to avoid C++ - style problems with multiple inheritance?
> When people talk about how similar are OOP and FP languages they miss the elephant in the room - immutability.
>> What elixir doesn't have is pass-by-reference
But immutability is not INHERENT to the FP style. In the strictest sense BEAM languages are not immutable, because message-passing is not state-safe, and there's also the Process library (and NIFs), but practically they are. Julia is also a very powerful FP language that isn't immutable by default, and Swift is an OO-ish? is it OO? language that is pass-by-value by default.
You're not going to run imperative code on BEAM (or ERTS?). It just wasn't built for it.
what's the recommended IDE for erlang dev? I've tried getting emacs setup but boy it's a struggle. I tried following this beast of a guide with questionable results,
https://www.lambdacat.com/post-modern-emacs-setup-for-erlang...
Vscode has two lsp plugins. I find pgourlains version to be better
There is an Erlang plugin for IntelliJ which is...ok.
What is the struggle part ?
I ask, because maybe I could/should write up something for /r/erlang,
What kind of static analysis is available for BEAM programs?
I'm working on one for elixir: https://github.com/ityonemo/selectrix
Take a look at Xref and Dialyzer