Elixir 1.9
elixir-lang.orgIf anyone is on the fence on learning Elixir / Phoenix (Elixir's most popular web framework library) please do yourself a favor and give it a shot. If you're on a time crunch and want the best bang for your buck on "why bother learning yet another language to write web apps in?" then watch this talk called "The Soul of Erlang and Elixir": https://www.youtube.com/watch?v=JvBT4XBdoUE
I'm not affiliated with the language but I just want to throw out that I've been around since the Geocities era as a web developer and I haven't seen a tech stack or platform this well thought out and so in tune with what makes developing (and testing!) modern web applications an absolute joy. There's really something special here. I see myself sticking with it for 10+ years with 0 regrets.
The best part about it is everything is ready today (and was before 1.9 too btw).
The Authn/Authz story of Phoenix is a joke though compared to Rails/Devise or Django. The framework authors refuse to ship a first party implementation and instead leave it to the community. Problem is, the ones available are mostly extremely low level (ueberauth, guardian etc.) and you have to spend ages customizing it. On Rails it's literally two commands to set up user registration, email confirmation, password reset. Despite its supposed Rails roots, Phoenix has a long way to go (the greater irony is that the company behind Devise, Platformatec, is also the company behind Elixir).
If you only need basic functionality and absolutely have to use Phoenix, Pow is a good choice:
I can't speak for the Phoenix dev team but the fact that some of the Devise authors are now heavily invested in Elixir / Phoenix and they didn't port Devise to Phoenix by now makes you wonder if they still think the idea of Devise is a good one.
Personally I haven't found auth to be a problem in Phoenix and I have lots of experience with Rails (and using Devise).
You can put together an iron clad user registration + auth system in about 150 lines of app-level code (not including lines of code for the template forms, but with Devise you would end up customizing your own forms anyways). This includes registering new users, authenticating by email + encrypted password, logging in, logging out, session management and having the idea of a current_user in your system, along with a way to restrict routes, controllers or actions to only logged in users. The essentials basically.
I would much rather manage ~150 lines of code in my own project that I fully know and can easily customize than pull in something like Devise which is 6,000+ lines of Ruby and have to configure / override a number of things.
Everyone has their own opinions but I would much rather spend my time developing features for my app than trying to figure out how to customize a massive third party dependency.
Personally I'm rolling with a magic link authentication system in my Phoenix app and it was also around 200 lines of code to implement everything, including configuring and sending the emails out. There's no external libs beyond Phoenix to get it all working (except for sending the emails out, in which case I use Bamboo which is a lib dedicated to sending emails).
As someone who used Devise multiple times and always ended up regretting said choice in the long term, I am actually glad there is no Devise for Phoenix.
In one day I can quickly get something up and running with Guardian and have full control over how my app does authentication. Password reset and confirmation are easily done with Phoenix.Token without a need to touch the database.
Also, authorisation is a breeze on Phoenix. Take a look at Canada: https://github.com/jarednorman/canada - and it is only 10 LOC.
> The Authn/Authz story of Phoenix is a joke though compared to Rails/Devise or Django. The framework authors refuse to ship a first party implementation and instead leave it to the community.
I agree, but it's not unique to Phoenix. In the PHP world the otherwise excellent Symfony framework is only starting to add generation of authentication/authorization, and email confirmation and password reset remains the developer's responsibility.
I see two reasons for this.
1. Frameworks see themselves as building blocks, not RAD systems. When the core framework developers are (mostly) working on APIs they don't see the need for approval/password reset. After all "you'll issue the user with a JWT so who cares about the rest (shrug)"
2. Flexibility to auth with anything. Apparently not bundling a system makes it more "flexible".
I believe (1) to (a) be shortsighted and (b) we need RAD frameworks, of higher-level flexible components that can be bolted together.
I believe (2) is wrong, and bundling standard auth flows does not reduce flexibility: you implement your own if you need something else.
That's partially true because you have FosUserBundle as kind of defacto standard for user facing auth (but yes it's a community package) and many built-in things for APi auth.
FosUserBundle is no longer recommended, and is superseded by Guard. Guard is considerably nicer to work with, and doesn't take long to build the extra bits (password reset etc) but (a) it's not built-in, and (b) if done as a bundle, it doesn't get the same level of support.
>> The framework authors refuse to ship a first party implementation and instead leave it to the community
Maybe for AuthN, but AuthZ is pretty far from the business of a web framework. The framework authors seem, for the most part, to be avoiding first-party endorsement of secondary tooling that doesn't deeply benefit from being first-party. I find it super refreshing coming from Rails where it is either the golden path or good luck.
A lot of the early AuthN libraries seem to target the controller layer, but I've found, having worked on a couple of sizeable Phoenix production codebases, that AuthN at the app domain layer works better. Regular use of channels as an alternate entrypoint to web request controllers means that controller based helper libraries are less valuable. Once you push this logic inside the app domain, it really isn't the business of Phoenix to know or care about it, since you're just working in plain Elixir, probably with some sort of Ecto struct to represent identity.
I would like to point out that it is not only the company but the person behind Devise (ie José Valim). It may means, possibly, that they are not really a fan of what was created with Devise and think the experiment should not be replicated. You know.
My preferred way of learning a new language/framework is watching someone walk through creating a real app in that language/framework so I can see all the folders/files I need to touch and the thought process moving step-by-step. Unfortunately I haven't found much in the way of that for Phoenix, the stuff on Udemy is pretty outdated last time I checked. Any recommendations?
The way I've learned so far was building my own app while skimming the docs and looking at successful open source projects like https://github.com/thechangelog/changelog.com for inspiration and guidance. Basically error driven development and looking up things as needed. Of course also taking advantage of IRC / Slack / Forums when I get really stuck.
But recently I read the Programming Phoenix 1.4 book (written by the authors of Elixir and Phoenix) and it really filled in a ton of gaps and made so many things click together. I can't speak highly enough about it. In fact, finishing that book today is what provoked me to write my comment here. The book might be exactly what you're looking for as like 5% of the book focuses on the super basics and a toy example, but 95% of the book focuses on building up a pretty real app and layering on features as you go. It's also up to date and feature complete even though the book is labeled as a beta release.
By the time you finish the book, you exercise a ton of interesting things along the way (including how to take advantage of Erlang's VM to build features that would be 100x more effort and probably 10x worse in other tech stacks). You definitely don't just build a simple todo app. I don't even want to say what we build in the end because it feels like a spoiler. I only say that because when I hit the last section of the book I had no idea what was coming and was blown away by what it covers. It literally made me smile for hours just thinking about how badass everything is when it all comes together.
Glowing recommendation, I'll have to check it out!
I concur. Programming Phoenix (I read the old version, 1.2 or 1.3, don't remember) and it is simply so well thought and helps you understand why Phoenix is not as magic as its appears in the start.
I also recommend you to try to play with Plug. Things make much more sense in Phoenix after you understand how Plug works.
That's exactly what I do at https://alchemist.camp. Most the videos and all the intro-level ones are free.
I've also put up about a hundred screencasts at http://youtube.com/alchemistcamp.
Looks like a cool site. I'll check it out. My Elixir is Good. My Phoenix is not.
I’m building a side project in Phoenix right now. For me the learning curve is wondering whether Phoenix has a built-in or “everybody does it this way” solution to the problem I’m facing or whether it’s something I should implement myself.
In Rails the answer is always “use this popular gem” and in Go the answer is always “roll your own”. Phoenix is somewhere in the middle.
The good thing about Phoenix is you are not forced to go in any particular convention for everything, you can craft them yourselves from scratch. The bright side of this is you will only every little thing in the implementation and you'll know how to optimize it.
The one time I really want to override something in a Phoenix app was deleting a response header bolted on by Cowboy (I think it was the server header). I couldn't find a work around, so I did something lame like intercept the formatted response and modify it. Some flexibility in that area would have been nice, but it was a pretty niche corner case on a not too important app, so a time crunch hack was my best option.
I'd say you're in good shape, then. Phoenix is really well-designed and doesn't have a lot of "magic" in it.
https://pragmaticstudio.com/elixir
https://pragmaticstudio.com/courses/unpacked-full-stack-grap...
These have been my go-to that seem to fit what you're describing. I hope this helps.
For event sourcing in Elixir this is a very good walk-through: https://blog.nootch.net/post/event-sourcing-with-elixir/
Another example is: https://github.com/AltTracker/alttracker
It's an implementation of fully functional site. The beauty of Elixir is that the code is very readable.
The thing is, nowadays, statically typed languages became so ergonomic, there is really no reason to go with dynamic languages.
I partially agree, but none have the preemptive green threads of the BEAM VM and its amazing tooling around supervisors and worker hierarchies.
Trust me, if somebody puts that in Go, Rust or OCaml -- I'd ditch Elixir tomorrow.
The BEAM VM is just too good. You can have thousands of smaller tasks running in parallel and nothing lags until you hit the physical limits of the hardware -- which 99% of the apps never do.
I think this is right on point. The BEAM is finely tuned for a specific problem and it solves it beautifully in a way that is extremely hard to replicate unless you actually are willing to invest in some kind of run time (which instantly makes you ineligible for consideration in a lot of peoples' minds). Every language that has tried, as far as I know, has done so as an afterthought and obviously does not end up replicating the important parts.
Knowing that, it's perhaps more pertinent to talk about what you can actually bring to that VM. Alpaca [0] could potentially be the solution. Interoperation with the rest of the VM languages and static typing on top of that. I have no doubt that most of the community would simply not use it, because they don't care about static typing, but I have zero reservations about saying it would be a better way to write code for the BEAM in the future.
Without something like that, it's unlikely that I will actually put any code that does more than something like routing messages to other services or the like on the BEAM in the future. Beyond a fairly low number of lines I simply don't trust anyone to write code that does exactly what they think it will do and no more in a dynamically typed language.
> The BEAM is finely tuned for a specific problem and it solves it beautifully...
Not disagreeing with you per se but that was true 10 or 20 years ago. Nowadays the BEAM is very solid general-purpose runtime with the best parallelism on the planet attached.
> Knowing that, it's perhaps more pertinent to talk about what you can actually bring to that VM. Alpaca [0] could potentially be the solution.
Realistically, I'd say that ship has sailed. Elixir has a lot of inertia for that to happen at this point. People focus really hard on improvements over Dialyzer (like Dialyxir + improving warning messages), linting, standard formatting, higher-level code generators and what-have-you.
At this point I'd think an OCaml-to-Elixir transpiler that enforces the compile-time guarantees of OCaml and translates them to Elixir code is the much likelier road to strong static typing on the BEAM. I might be wrong though, it's just a speculation.
Most people who point to Elixir's lack of static typing as a weakness, or believe their productivity would be adversely affected by it, do so because they haven't actually used the language.
Anyways, this subject occasionally comes up for discussion in the community. This post sums up my thoughts fairly well: https://elixirforum.com/t/static-vs-dynamic-typing/9824/3
It's a near certainty that I've written an order of magnitude more elixir code than you have and I can say without a shadow of a doubt that elixir's lack of static typing definitely makes it less useful.
Even if you disregard the modeling power that you can get from this there is an upper bound on any elixir project after which any work on it becomes less and less easy to do, as with all languages that lack a static type system. We write assertive elixir code as much as we can, but that doesn't mean that you can actually guarantee anything about a code path that is less traveled.
dialyzer is also not the answer. Oftentimes we'll find ourselves in a situation where dialyzer complains because someone who made a major library simply doesn't use dialyzer. You might wonder why they don't in that case, because it would pick this particular issue up very easily, but the next time you run into some garbage error that dialyzer spat out that you are instantly reminded that you can neither trust it to be correct or safe, so people turn it off.
Elixir has upsides: I think it's solidly the best language for creating servers of different kinds, but it's near useless after you pass a fairly short distance with it. I would absolutely never, in my personal endeavors (as opposed to my working contract) write an elixir server that actually tries to do anything meaningful itself other than just route messages to other servers.
What language would you use to write a server "that actually tries to do anything meaningful"?
Any language with the possibility to express variant types that all can have different structure and have that structure be checked at compile time, properly, will be a better choice for describing simple, medium and complex domains.
Whether or not this is based on nominal or structural typing is less important, because the feature itself only depends on differentiating between what is essentially different cases and having that be done safely and properly at compile time.
Elixir has no satisfying solution to this problem and it likely will never be able to. It is a huge penalty in domain modeling, which is why I personally don't want to use it for anything that actually has to deal with the flow of things.
To some extent I expected and wanted the VM to feel like it makes up for this by being so great that what it does, but there's no band-aid you can use to fix lack of modeling power.
> It's a near certainty that I've written an order of magnitude more elixir code than you have and I can say without a shadow of a doubt that elixir's lack of static typing definitely makes it less useful.
You should really read that post I linked, and do a bit more research on this topic. The consensus in the Elixir community (where many members have written an order of magnitude more Elixir than you) is that static typing would be "nice to have", but is not critical by any means. Your post seems to be an attempt to spread FUD.
I've done all the research I need to, thank you very much, and I was there when this thread was started and much earlier than that. The consensus in the elixir community is hardly relevant when most of the users aren't experienced enough with type systems to know what they're talking about.
If you asked a community of people who primarily have never used languages that don't have garbage collection, they'll likely not understand at all why it could be a bad thing.
Someone who has used a language without garbage collection and is comfortable with basic allocation strategies is very likely to remark that having control of memory allocation and deallocation is very often something that you end up wanting instead of having to re-architect your solution in indirect ways to influence the garbage collector. This comes from having a wider perspective and the elixir community at large does not have this.
Given that the elixir community is also very cultlike it's hardly a productive thing to take what they say as the objective truth.
(I'm not saying this is an outsider at all, I've been a part of the elixir community since 2016. It's not really despite that I am saying these things, it's because of that.)
Friend, you're making a lot of assumptions and claims about people you don't know (including me), and have a certain streak of arrogance that is difficult to get past. This makes it impossible to have a productive conversation, if having a productive conversation is your goal.
I advise some humility, as well as some effort to recognize that your opinions sound very dogmatic. You are of course free to like or dislike any language you want, but it is important to understand that going from one's own preferences to grandiose claims about a language's merits and usefulness is a rather big leap.
> Friend, you're making a lot of assumptions and claims about people you don't know (including me), and have a certain streak of arrogance that is difficult to get past.
This is rich, coming from someone who claimed that anyone who wants a type system in elixir hasn't actually used elixir.
> You are of course free to like or dislike any language you want, ...
This is the most common reaction within the elixir community, to assume that anyone who doesn't put the language on a pedestal dislikes it or hates it. You might want to consider what that actually says about the community and the atmosphere of near unconditional admiration it displays. It's an extremely distasteful attribute for a community to have.
Elixir is useful for shuffling data from point A to point B... That usefulness has its limits, that's it.
> This is rich, coming from someone who claimed that anyone who wants a type system in elixir hasn't actually used elixir.
You may want to read my original post more carefully. I said most people who complain about the lack of static typing haven't actually used it. Obviously there are some, like you, who have experience with the language and dislike the lack of static typing. And that is fine.
I mean, you only need to casually browse HN threads about new languages to find posts along the lines of "this looks interesting, but the lack of static typing means I probably won't use it." Instead of taking a holistic view of the language and its merits and researching its design choices and trade-offs, they adopt a dogmatic mindset and dismiss it outright.
>>This is the most common reaction within the elixir community, to assume that anyone who doesn't put the language on a pedestal dislikes it or hates it.
No, that's just me, and it was a figure of speech. Elixir community is very mature and welcoming, and most people in it come from other languages, and are able to objectively discuss the language, its merits and shortcomings. The post I linked in my OP is a great example.
>>Elixir is useful for shuffling data from point A to point B... That usefulness has its limits, that's it.
I mean, again, that's just your opinion, and it's a simplistic one. I obviously don't know the depth and breadth of your experience (and won't make assumptions about it, like you did about mine), but Elixir is useful for, and excels at, more things than simply "shuffling data". That is why there is so much excitement for it: the use cases it enables, and the elegance and simplicity with which it enables them, are nearly unparalleled. The lack of static typing does not change this fact, even if it can make certain things a bit more cumbersome towards the extreme end of the project size and complexity spectrum.
> This is the most common reaction within the elixir community, to assume that anyone who doesn't put the language on a pedestal dislikes it or hates it.
Not sure you and I read the same forum. My reaction to Elixir criticism usually is -- "well, if you dislike it for this or that reason, or simply don't find it useful, then don't use it, there are a lot of other languages out there". Many others do the same.
I haven't noticed cargo-culting on ElixirForum in a long time. You might simply be too cynical to differentiate people whose work life has been genuinely improved (and they express that sentiment on the forum) and people who overhype something you dislike (for valid reasons for your use-cases).
I agree. I've sincerely asked for explanations what is missing when pattern matching is so similar to type inference. Combined with guard clauses and "let it fail" philosophy, what is the problem other than boilerplate? Is it lack of mechanical refactoring? Totally honest question for type enthusiasts, my other posts have been met with silence.
Erlang’s Dialyzer (and the related Elixir project Dialyxer) provide compile-time type safety analysis. It’s optional and only at compile time, but immensely helpful in verifying correctness.
Worth noting that Elixir Language Server will also hook into your text editor/IDE and provide warnings every time you save a file (it compiles the code behind the scenes).
I also highly recommend the dialyzex [0] project as well. While it's not going to match every feature of some type systems, it does a much better job than one might expect.
It's always a little frustrating that folks pass on Erlang or Elixir as entirely dynamic. It's a lot closer to what you find in gradually typed languages if you take advantage of the tools available.
[0]: I know there are a few of these elixir projects, though I am not familiar with the parent's. Perhaps it's a typo?
dialyzer is terrible and doesn't actually work in practice, as is evidenced by the many dialyzer errors that slip through to current releases on hex of libraries even the size of Phoenix, StreamData, etc.... Even the major community members don't use it. It's not a substitute for a type system and especially not one that allows you to properly model things in it.
have you used it with elixir_ls? I find that in practice it catches about 80% of my errors before I send them to test.
Yeah, I managed to get our team members to use ElixirLS for this very reason, but in the end I found that it doesn't work when it matters. The good thing about ElixirLS is that at least you have to configure something to disable dialyzer, so by default to at least get some coverage…
In the end you need to some kind of compiler that does proper, rigorous and unapologetic type checking. If Alpaca could reach critical mass I could convince our project leader to use it, which would solve the problem entirely.
Sure, the ecosystem would likely remain the same in that the majority of people simply don't actually have any familiarity with strong, static typing, but at the very least you can now set up these guards yourself and you can have code that is much more likely to be safe and good.
Imagine typechecking a cluster of nodes all sending messages to each other.
Statically typed languages deal with data un/marshaling all the time. Maybe I’m missing something, but how would this be an issue?
One thing I love about Elixir is I had read the Erlang book from Joe Armstrong ages back, and a lot of ideas and concepts still apply. I just didn't get into Phoenix cause PGSQL is a PITA on Windows, so I have to go back and try with MySQL instead.
Use Vagrant! Then you don't have to install/run PostgreSQL natively at all, and can run it inside a Linux VM.
You could run PG in either WSL or a Docker container.
I currently use Windows (where I spend 99% of my time in WSL) and everything runs great.
Maybe somebody else knows where to wave the dead chicken to make postgres under WSL work well, but my last two tries (Feb 2018, Nov 2018) suggested that trying to do so invites lots of yak-shaving, pain, and poor performance. And that's just using the package manager to install, if you want to build from source, it may not work at all.
Stories to the contrary and hints how to make it work are welcome.
Might be worth retesting, we did some changes to PG to handle the unsupported syscalls more gracefully (warn once, and then not anymore) in a recent set of minor releases.
What issues are you having getting Postgres setup in Windows?
I've used the Windows installer[1] and have had no issues.
I was unable to figure out how to setup accounts, and I am not used to PostgreSQL, I couldnt figure out my root password, which I swear was just "developer" but that didn't seem to work. I'm used to other database engines, despite my love of the idea of using PGSQL.
That’s not a windows issue, it’s an unfamiliar software issue. Take a weekend to learn how Postgres handles accounts. It takes a little getting used to but it is very much worth it.
Ah ok. Maybe it's because my install was basic without additional accounts. I just have my phoenix apps setup to use the default postgres/postgres login.
For some reason I remember having this PGSQL issue a couple of times on Windows, so I may either go with MySQL or what others suggested with WSL / Vagrant. Also I always forget database passwords after a week if it's not saved somewhere.
Screw windows and move to Linux! Opens up a ton of possibilities including docker
I gave it a try, but for me after years of programming in C like languages the syntax is really hard.
I know it is very similar to Ruby.
I wish there was a ElixirC syntax that would also compile to Earlang.
After years of programming with C-like langauges, I actually came to really appreciate the syntax of Erlang. It's actually fairly simple and straightforward. I think the biggest issues people have with it are:
* Capitalized variables names
* use of comma, semi-colon, and period as clause delimiters
It's interesting, but I think that Prolog-style syntax is probably the easiest to read if you don't have syntax highlighting available (though I don't think there is as much of a difference with syntax highlighting).
IMO, the use of header files is a bit baroque and I find it makes it harder to reason about your code. Erlang structs are also no fun at all.
It seems similar to Ruby, but in the end it's more like Prolog
I think you might find one that suits your needs in this list
Fantastic talk. I have been meaning to learn more about Elixr for a while now, and this video was perfect. Thank you!
I sometimes joke that microservices are half of Erlang, badly implemented (I guess Greenspun’s Curse is broken?).
Elixir has been second or third on my todo list for a long time. With the scary news from Vue, it might have just jumped the queue.
Thanks Robert Virding
http://rvirding.blogspot.com/2008/01/virdings-first-rule-of-...
See also, Stigler's Law.
Or Poe's law, thus named in 2005, but based on a phenomenon that was well known by the time of Godwin's law.
> Scary news from Vue
Do you have a link?
He's most likely referring to: https://news.ycombinator.com/item?id=20237568
Which is a bunch of FUD (see top comment by EvanYou).
Thanks
That was exactly my thought when I went thru release notes.
Fantastic. Congratulations to the Elixir team!
Something striking from the release announcement:
> As mentioned earlier, releases was the last planned feature for Elixir. We don’t have any major user-facing feature in the works nor planned.
Part of me is a little alarmed—no one wants a language to stagnate. José makes it clear that this doesn't mean it's the last version:
> Of course, it does not mean that v1.9 is the last Elixir version. We will continue releasing shipping new releases every 6 months with enhancements, bug fixes and improvements.
Elixir is remarkably extensible. When I was first getting acquainted with the language finding macros with all the expressive ability of a LISP blew my mind. Making powerful macros a core part of the language meant that the language could grow. That in my mind is the most important part of any language's long-term viability.
Fantastic work all who contributed. Thank you for making a fast, functional, and fun language to work with!
"no one wants a language to stagnate"
In the sense of the language itself, I do. The language community should keep growing, libraries, frameworks, etc., but I'd like to see more languages qua languages declare themselves "done" sooner and more often, or failing a sudden cutoff, start seriously raising the bar on the next "new feature". There's a lot of good languages out there that were great in years 5-15 and then choked on all the features they kept adding to the core language.
"no one wants a language ecosystem to stagnate"
I used to be so happy when there was a new Java version every year, something new is good, something better is good, some hype is good. Now I am getting old I just want to use the same tools for the next 15 years. I mean it is not like most of the problem we are trying to solves were limited by the languages itself. ( Mostly the ecosystem around it )
A difference with Java is that new Java releases tend to come with changes to the JVM. The language's semantics and the abstract machine are intertwined in a way where the JVM might be changed to enable a new language feature, or vice-versa. And the JVM itself improving is pretty much always an unalloyed good, so people tend to look forward to releases of Java to see what JVM improvements it has in store.
Elixir doesn't hold the same position for its abstract machine. BEAM, the abstract machine, has its development intertwined with that of Erlang's language development, but not-so-much with Elixir's. Elixir can stabilize while BEAM+Erlang continue to evolve "underneath" it. Given that Erlang syntax is pretty much† always valid Elixir syntax, Elixir doesn't even need to change to expose new Erlang stuff!
† There were big changes when maps were introduced, but another change of that scale seems unlikely at this point. (Unless Erlang gets a mutable byte-array type or something.)
Unless you are using Android J++, then you get some Java features at the expense of de-sugaring.
Also earlier versions of Java were a pain. The flexibility of a well thought out functional and dynamic language means new syntax to reduce boiler plate or extend generics don't need to be added. Compare Clojure to Scala for extreme examples in the JVM land. Similarly I think modern OO languages seem to be adding new syntax to handle OO complexities, e.g. much of the newest syntax additions to Python3 aren't useful (walrus operator? Well everything is already an expression... Etc).
In the end, I agree that the idea of a language which could be stable for a decade or more sounds fantastic! Given macros could add almost any other required feature should a good idea arise. Plus the BEAM and elixir compiler can continue improving performance and tooling.
Thank you. That's what I should have said. :)
Programming languages are software products, when they are done, it is time to move on.
Honestly I'm happy to see that they're gonna slow down a little bit. You need a certain stability for the language to become more attractive. Think that the book "Elixir 1.6" was only released a few months ago, and today we're already getting Elixir 1.9.
Yes, of course 99% of the book is probably still relevant and all, but I believe it is important to brand a language as having reached maturity and stability.
By the metric above, which is release numbers, we won't slow down because we will continue with two releases per year (this is our frequency since v1.2/v1.3). :)
However, in terms of new features, we have already been slowing down over the last releases.
FWIW, the new books they are releasing for Elixir and Phoenix are both titled "Programming Phoenix >= 1.4" and "Programming Elixir >= 1.6"
That they are willing to put that in a primary book title is pretty bold.
I guess Elixir is "done" then? What could be the downsides to this?
Lack of language improvement. Sure, you can do anything with macros, but then when you open some random open source library it may be using different macros (or macros that are named the same way but act differently) than those that you use. Putting something in the language forces everybody to use the same standard. It can be good and it can be bad.
There's a middle-ground stance that's pretty common in the Elixir ecosystem: five or six libraries get written to solve a problem over the course of a year or two, people play around with them, and then a common set of primitives gets factored out of those libraries and made into an interchange library that all those libs support.
That interchange library can then be integrated into the Elixir stdlib—that's what happened with DateTimes in Elixir—but there's no reason it needs to be. It works just as well to just keep the interchange-type library as its own tiny little dependency that all these libraries pull in.
This is one of the awesome things about Plug, in the sense that if your library or what have you targets Plug it'll work reasonably well (if not entirely seamlessly) in a Phoenix app, a Sugar app, a pure-Plug app, a $SomeOtherFramework app, or what have you.
Plug happens to be somewhere in between the two outcomes you've presented, of course; its development happens very closely to that of Elixir (and it's arguably a sub-project thereof), but it ain't a core part of the language itself (last I checked; maybe this has changed in recent years).
This is great news! The deployment story has progressively improved since I started using Elixir in 2016, but the best way is to just have _one_ community blessed system.
Distillery has helped a lot for sure, so it's nice that it's basically been rolled into the base elixir distro.
There are some comments below about Distillery vs Elixir releases, so I will hijack the root comment to add some clarifications.
It was clear to both Paul (Distillery author) and the Elixir team that Elixir releases were going to be a subset of what Distillery provides. There is a good chunk of what Distillery does that is well established and that's what we streamlined and brought into Elixir. Some other areas were left out, such as a complete structure for setting up CLI hooks, and most notably, hot code upgrades.
The documentation explains some of the complexities behind hot code upgrades: https://hexdocs.pm/mix/Mix.Tasks.Release.html#module-hot-cod... - so I won't go over those. More importantly, during discussions with different teams and library authors, it was clear there is not a golden path for hot code upgrades. There are different approaches, with different gains and pitfalls, and it would be too premature to choose one as blessed.
So what is the path forward?
While Elixir releases don't do hot code upgrades by default, its whole structure supports it. We use the proper names and structure everywhere. But the front-end to start the upgrade is not there.
I have discussed with Paul the possibility of Distillery building on top of what Elixir releases provide with a more complete feature set. It doesn't have to be done by Distillery either. Maybe someone (or multiple people) will provide a smaller package that focuses on hot code upgrades. Then once we gather enough collective knowledge, we can choose if and how to proceed.
Still, I believe Elixir releases cover the majority of use cases out there. But we will know for sure over the next months. At least, by making releases part of core, we hope we are easing the learning curve for releases altogether, which we will naturally lead more people to explore hot code upgrades.
We've never actually done hot code upgrades yet before, although, like many Elixir/Erlang/OTP things, I feel confident that if I was required to do so, it would be easier than in most other ecosystems.
Similarly I suspect many users of elixir are doing the "boring" thing that we're doing – load balancer -> stateless web server(s) running phx -> DB server. And they'd probably be more than happy with this.
It seems like a pure win as long as there's still a zero downtime solution that doesn't rely on containers. I'm definitely looking forward to what gets built on top of 1.9 releases!
Based on what I've learned from the Elixir forum, these new releases have "no support for hot code updates / appups / relups".
This means that Distillery will remain the best choice for many of us, including myself.
I have a very simple deploy process with Edeliver and Gitlab CI/CD that basically just consists of pushing or merging a commit to master that passes the tests. There's zero down-time and it doesn't require containers on the server.
I'd also say that the "deployment story" wasn't any worse than Rails or other stacks in the past except in that people have a choice of having a stateful server.
This is how I'm doing it now: https://youtu.be/-mm44ADU3kc?t=172
Wait, what? I wrote an Elixir/Phoenix webapp around 2015 and one of the major selling points was the ability to hot-upgrade code that was inherited from Erlang. What changed?
I compare it to unwrapping a handfull of utility razors chucking them up in the air and saying "free razors!".
You CAN do hot upgrades, but many times the complexity of doing so far outweighs the benefits. For any non trivial app it makes updates/deploys to the app non trivial as well.
The problem with hot code updates is they can break your app if your structs change shape and are extremely hard to get right over a period of time. Best to do rolling deployments with Kubenetes or something...
You make it sound like setting up k8s is a walk in the park! I've used hot code upgrades in production for 3+ years, and we've had it blow up in our face perhaps twice. If you do canary deploys, it's not hard to control the blast radius.
Hot code updates can be challenging at times, but the decrease in deployment time and hassle for changes where it makes sense is definitely worth it, and using hot code loading for most updates doesn't prevent you from using a rolling restart for the updates where it's more appropriate.
If you're using distribution, there's a good chance you need to deal with messages from both updated and not updated nodes anyway, so handling different shapes from different versions is a skill you already need to have; it's just hot load exposes it at more layers than previously.
Since working in Erlang for several years, when I have to work in languages where hot loading is not easy or common, it's always frustrating. Throwing away connection attached state to update code is a hard choice that doesn't need to be.
> The problem with hot code updates is they can break your app if your structs change shape
I have no experience with Erlang/Elixir myself, so please excuse me if this is a silly question – but, why can't the language/platform actually detect the difference in struct shape, and refuse to deploy the new version in that case? Or, demand that you provide some sort of conversion function that maps the old struct to the new one? Is this a consequence of lack of static typing?
I have never actually used this feature but here goes:
The language standard library gives you hooks to relup the internal state of all of your processes: https://hexdocs.pm/elixir/GenServer.html#c:code_change/3
What it doesn't do is give you a hook to relup the messages that get passed between processes. If you do it right, you can pattern match against different versions of messages, and do the correct conversion function, but there are liable to be many, many, shapes of messages (including ones that you might not write yourself, e.g. coming from a library) and it might be difficult to catch them all.
To do it right, I would want to set up a lot of testing to make sure you can hot code reload safely. There is no specific guideline, and it probably hurts forward progress in developing guidelines, that generally, it's okay to have some downtime in any individual server node as erlang/elixir encourages you to think about failover anyways, and most elixir apps are relatively stateless webservers, so you've probably got robust load balancing and migration scheme in place in your cluster to begin with, making blue/green or rolling updates a "pretty much good enough" thing.
> Is this a consequence of lack of static typing
No.
It was talked about a lot by people who were fans of elixir/erlang but hadn't really done much work with it (these folks are increasingly common in tech, confusingly so sometimes but good).
Digging deeper the line from folks using erlang/elixir "in anger" was always that it was a supported feature but the reality is that most people shouldn't do it and wouldn't need to for the hassle it has.
Nothing changed; hot code reloading is still possible through the mechanisms that have always existed. What's being stated here is that the new Elixir support for "releases" does not include support for this, so you would have to use a separate mechanism to perform the hot code reload.
That "separate mechanism" could be as simple as a plug that tells Mix to recompile and reload. Example (designed for - and part of - the Sugar framework, but should theoretically work for any Plug-based app, including Phoenix-based ones): https://github.com/sugar-framework/plugs/blob/master/lib/sug...
Hot upgrades like this are not foolproof (which is likely the reason why the above example is gated to :dev environments); there are other concerns like database migrations and other internal and external variances that make this inappropriate for most production situations. That said, these same concerns often exist for other high-availability situations as well, so if you know that you want zero-downtime code upgrades, figuring out a way to do it cleanly within the application is likely valuable as a way to avoid the hell on Earth that is trying to do this with, say, a bunch of load-balanced Docker comtainers.
As a bit of a correction here to my own comment, it looks like Elixir releases don't ship with Mix, so the above example probably wouldn't work in that particular scenario (unless maybe you figure out some way to include Mix).
So the better option would be to figure out a way to compile a new release, get the modules in place where the currently-running release expects them, then write up a plug similar to the above to check for new module versions and load them (or just do it as part of whatever mechanism you'd use to get the updated modules into the running release in the first place).
OP mentions it's still entirely possible with other deployment libraries (Distillery).
Just the one being newly packaged into Elixir 1.9+ by default does not yet support hot-loading...
I would be curious to know what makes this feature a major selling point. Aren't rolling deploys easier to handle? I'm not saying it's not a great feature, but honestly, it probably stopped being useful since Erlang/OTP was originally designed.
Hot code upgrades are one of the Erlang features that helps me keep headcount down on our operations team. Appcues is three years into running Elixir in prod, and we still have not had to set up blue/green deploys because the built-in upgrade and rollback feature is so robust. For the first year and a half of that, we had a single platform developer (hi!) supporting dozens of servers and hundreds of customers. Hot upgrades are very, very useful.
yes, but they do come with a lot of cognitive load and sharp edges. For many apps they are simply not worth it.
The cognitive load is quite low when deploying changes which only involve your app code (i.e., no dependency upgrades). My app has a few tens of millions of open websockets at a time, and it's worth it to me to avoid mass reconnects. I'm not everyone, but my use case isn't totally unique either.
I am not sure about your specific use case but it owuld be hard to imagine that graceful reconnect on websocket + a more traditional deploy would not be much more effective resourcing vs that hurdles you have to go through to properly appup. Is it really an advantage to you or has it slowed your ability to get to market for something that seems like it "should" be an advantage.
Its not like you don't need graceful reconnect handling logic for the websockets anyways -- net-splits and endpoint failures are a thing.
It really depends on how much effort is involved in setting up the connections, and how much effort is involved in directing connections to a new host.
If I'm running 1M sessions per machine, and each machine can handle 10k new connections / second, rolling restart is really expensive. If I'm running 10k sessions per machine, and can handle 10k new connections / second, rolling restart isn't too bad. This generalizes to really anything that gather essentially ephemeral state, but that state is costly to gather (tcp flows in this case, data caches in others, etc).
BEAM excels at applications with huge numbers of sessions per machine, which is why some users really value hot reloading.
Editted to add -- the OTP application update sequence doesn't necessarily need to be used. Where I work, we certainly don't do that. Just a little bit of logic around code:soft_purge/1 and code:load_file/1
"Few tens of millions of open websockets" is more than Slack and Discord combined, not?
BTW: we were the first who did 1M long-living connections load test with Cowboy on a single EC2 instance back in 2011. And 3M long-polling HTTP requests on a single beefy physical server.
It was before WhatsApp upstreamed their optimizations and before Phoenix team made it easy.
I imagine both Slack and Discord have much higher numbers than this, as well as a very different workload (they're doing chat, we're not).
I don't want to give the impression that this is happening on a single server, either! I did the cowardly thing and threw more boxes at the problem. :)
It's a difference in run time purpose.
With a single code base capable of having millions of processes running as the norm, some handling direct client requests and others handling in-progress work, data storage, holding open connections for transfers, etc...you get the capability to deploy without disrupting ANY of that.
Most run times can't do anything close to that. Think about all of those X million websocket benchmarks...now think about being able to deploy without forcing all X million to try to reconnect at the same time.
And it can do this while all of the nodes are connected and communicating with each other as well as the outside world.
For standard issue client server, it's not that big of a deal. You just separate the web parts behind a load balancer.
For background workers, long lived connections, web sockets, video/audio streams, file transfers (CDN)...it's huge.
For all that to work you will need:
- to understand exactly what your app is doing
- to understand exactly what Erlang/Elixir releases are doin
- to understand exactly how cod upgrade works
- to understand exactly or very damn well how to make the system handle those 1 million connections
- to understand exactly how to handle all the things you wrote about
And then, and only then will you be able to "think about being able to deploy without forcing all X million to try to reconnect at the same time."
There are no magic bullets.
Eh...it’s basically a 1 line command with distillery. Another for the rollback capability.
It’s pretty magical. There’s a reason people love it.
Certainly, don’t use it if you don’t need it...it introduces extra complexity...but if you do it’s really hard to beat.
Having to support two different versions of the Elixir service during the rollout period is risky...
What if something goes wrong during the rollout? For example, if you change the database schema or upgraded your database engine or changed your back end authentication approach, it can break the old code. Then how will you know whether it's a problem with the old code or new code if many nodes are running both?
Those are things you have to account for in any zero downtime deploy situation though. It’s mostly minor changes in how you roll out schema changes.
I was mostly listing things you need for Erlang. Elixir, thankfully, hides a lot of things away in much friendlier packages.
That said, even with Distillery, if something goes awry, you'll need to know the warts behind the magic :)
> Distillery has helped a lot for sure, so it's nice that it's basically been rolled into the base elixir distro.
That would be nice if it were true. However, for better or worse, Distillery was Not Invented Here, so what Elixir shipped with today is a reimplementation of the simple half of Distillery.
Or, you know, the creators want one official blessed way so as not to fragment the ecosystem, so they integrated part of Distillery upstream in the language itself.
Which is fine for probably most of us using Elixir who do blue-green deployments behind a load balancer.
Although we should probably embrace this aspect of the ecosystem, (like with clustering of servers) I bet it's even easier than I can imagine.
One part of releasing Elixir applications I never found a definite answer was how closely the build environment has to match the final environment where the release is deployed. The Erlang runtime is bundled with a release, and this part is platform-dependent as far as I understand.
If I'm writing an application that I'll only deploy myself I can of course match the environments exactly. But what about if I wanted to distribute an Elixir application to end users that might install it on all kinds of different operating systems and distributions? Do I have to build a release for OS/architecture combinations like Linux/64bit, Windows/32bit and similar? Or do I have to match each Linux distribution and windows version exactly? That would make it rather impractical to use Elixir to write software you give to other people, and don't deploy yourself.
Of course when I looked at this Distillery was the standard way to deploy, but it doesn't look like this part is any different with Elixir Releases.
Currently releases require the OS and architecture to match. However, Erlang/OTP does support cross compilation, so at least a possibility is there, it needs to be further explored: http://erlang.org/doc/installation_guide/INSTALL-CROSS.html
The other concern is packages with NIFs (native code) and they would need to be changed if they don't support cross-compilation yet.
I believe Nerves, which is an Elixir framework for embedded, does a good amount of cross compilation too but they have a more controlled environment.
How does `mix release` handle NIFs currently? Same architecture is assumed?
Yes, basically it does not. It is up to the NIF packager to package a working way to handle that.
This sounds like a job for Docker to me. Build a container, deploy an image to Kubernetes. Someone else wants to use it, they can get it running wherever they want to run containers without worrying about the runtime environment other than just the container configuration.
We build it in Docker, but copy the release tarball out of the container and upload it directly to our servers.
It needs to be a matching libc. So a release that includes the Erlang runtime and is built on Debian (glibc) won't run on Alpine (musl).
Congratulations! Notably, they mention switch to Cirrus CI, because it supports FreeBSD also. I wish Travis[1], Appveyor[2], and Azure Pipelines supported it too.
I know nothing about Elixir/erlang besides following installs for apps. I have used it, installed via ASDF on my Raspberry pi since dependencies weren't up to date.
Can anyone provide a good place to start on where to get started with Elixir?
I tend to learn by working on stuff and not just reading etc, maybe a step by step in elixir? I am going into the literature now as well
Thanks in advance for any help!
I learned by converting a simple Rails API that I had running to do one or two things into a Phoenix API. In my case, it was very eye opening both because it was basic ETL and because it was something that benefited from high concurrency.
I especially recommend this approach because it gives you more or less the most complete cross-section of some basic data structures, some web stuff, and some database stuff via their db adapter of choice, Ecto, which is excellent.
Each part can be a pretty steep (but fairly quick) adjustment depending on your background.
This is awesome, Cross-section is what I am also looking for.
Appreciate the input, thank you!
Don't forget to learn how to write a GenServer from scratch. I lot of people I've met who hop straight into Phoenix forget to learn the basics of GenServers (and processes in general).
Here's a stupid example I whipped up:
https://gist.github.com/amorphid/3dec7028b05bd10f6ff3180d199...
^^^ you wouldn't ever code it that way, but you should (in my opinion) know what each line of code is trying to do.
This is great, I have been teaching myself everything and there are a lot of day 1 things I learned way too late, this is extremely valuable, thank you!
My pleasure, I'd you want another Elixir reference examples, feel free to email me. Email in profile.
Programming Elixir[1] has hands-on exercises throughout. The author also published a video course[2] with exercises.
[1]: https://pragprog.com/book/elixir16/programming-elixir-1-6
[2]: https://codestool.coding-gnome.com/courses/elixir-for-progra...
Excellent! This is great thank you!
I'd definitely recommend the official getting-started guide, it's quite well-written and thorough: https://elixir-lang.org/getting-started/introduction.html
I read Dave Thomas' book cover-to-cover like some kind of legitimate nerd :) It was very good.
For understanding genservers there's The Little Elixir and OTP Guidebook, as well.
In my case I experimented with it for at least a year before I decided to jump in, my "experiments" are here: https://github.com/pmarreck/elixir-snippets (I tend to use single-file dev for exploring simple ideas; this is not a general pattern used in the community or anything, it's just something I discovered I could get by doing)
The How I Start guide was just updated for 1.9 http://howistart.org/posts/elixir/1/index.html
Excellent, I saw this researching today as well, I will check it out.
Thank you!
Sorry if this is a dumb question, but how do releases fit into the deployment story with containers so prevalent these days? (As in why is this a benefit when you can just package code into a single container to ship it?) Is it that it works with hot code updates? Or it’s a more Erlang-sanctioned way of deploying code?
Fred Hebert was on the ElixirTalk podcast and mentioned speed of deploys as one advantage - no need to rebuild a bunch of VMs.
> From experience, we could deploy to a 70-node cluster in something like under 5 seconds by doing it in parallel. If you want to rotate your infrastructure... your deploy could be taking from 5 minutes to an hour depending on how fast the connection draining can be done.
https://soundcloud.com/elixirtalk/episode-145-feat-fred-hebe...
There's a good 'why releases' section of the article.
Even without the compilation and configuration stuff, it's easier to put the release bundle in something basic like an alpine image, rather than keep docker image versions and app in sync.
But the article says you have to run the release on the same OS/version that you built it on. So if you're running it on alpine, won't you need to build it on alpine, which suggests you'll need to configure your Dockerfile to compile it anyway?
You would have two images, one with all compile time dependencies, sources, etc, which you use to assemble the release, and another image with only the artifact (the release) and none of the rest. Hex (Elixir's package manager) is open source and uses this approach, which you can see here: https://github.com/hexpm/hexpm/blob/d015973e472af59644ee537f...
That is correct. But that can be multiple stages, and therefore separated concerns.
Also, releases can be very small, after all the superfluous parts/symbols are stripped. Combine that with a fresh alpine image and you have quick-to-deploy containers.
With Distillery you can specify a directory for the ERTS so the release can be built on one OS and run on another. You just have to make sure ERTS was built on the OS that it will eventually run on.
Building a container and building a release are quite similar in many respects, so if all you want to do to compile your Elixir app during a container build, a release might not give you very much. An Erlang/Elixir release has some goodies that you might be interested in, so I'd read up on those goodies to see if they are attractive to you.
As someone using Elixir full time for a couple years now, I'm very happy to see releases become first class parts of the language. Is there a migration guide for those of us using distillery currently? And if we're happy with our distillery setup, what are the advantages of moving to native 1.9 releases?
Not much of a guide but at least an example: http://blog.plataformatec.com.br/2019/05/updating-hex-pm-to-...
They're moving config from each individual umbrella application to the root of the umbrella. Gotta love how the team keeps things simple and opinionated. These little choices add up to quite the developer experience.
Are releases supposed to replace Distillery? If so, are there any particular reasons to switch?
Elixir's `mix release` is intended to replace (or remove the need for) third-party packages like Distillery. However, it's not there yet, and Distillery is strictly more powerful at the moment.
Notably, Elixir's release implementation does not support hot code upgrades. I use upgrades all the time, and won't be trying out Elixir's releases until this shortcoming is addressed.
What's the benefit of a hot code upgrade vs a blue/green deploy?
speed. A hot code upgrade can deploy a whole cluster in a few seconds. Plus it can keep states safe without rebalancing.
I guess so, I imagine that's up to the maintainer.
The maintainer of Distillery joined the Elixir team to develop releases IIRC.
This is incorrect. I'd feel a lot better if it were true!
Paul went to work for Dockyard with Chris McCord when they started working on this feature for Elixir (and still is).
If he isn't the one writing it, he's certainly been highly involved with the conversation.
I was initially hired at DockYard to work specifically on releases and their eventual integration into core, but I am not a member of the Elixir core team itself :). In any case, that in itself is not really a relevant detail, since José and I talked frequently about the core implementation, so being on the core team wouldn't have changed much there.
As someone who's new to web dev, is there any incentives in going for Elixir than using those established languages(i.e. Node, Ruby)? Any book/sources recommendations?
This is anecdotal but I have inadvertently created FAR fewer bugs in Elixir/Phoenix than I ever did in Ruby/Rails. A lot of that has to do with the immutability guarantees and the pattern-matching and the fact that everything is explicit without being wordy... everything in this language seems designed to make it easy to grok what the code is doing, and that means fewer bugs produced.
I used to work with rails before. While it is fun to build stuff with it, it comes with a set of bothersome headaches:
- too much magic
- slow and painful to scale
- too much metaprogramming
- no compilation, makes it easy to hide syntax errors
- concurrency is absent
- frameworks like rails are too bloated
All these are addressed on the elixir platform, and there are more pros:
- immutability
- mostly purely functional
- well defined modules with good documentation
quite new to elixir ( still haven't deployed anything to prod ).
interested to know, how the releases feture going to affect the way we deploy elixir/phonenix apps in production ?
You can simply "mix release" to build a release without using the Distillery library. There are some limitations, though, mentioned on the forums: https://elixirforum.com/t/elixir-v1-9-0-released/23366/4
Previously most people used distillery. Now they'll use Elixir releases. Because it is pretty much the same solution but now moved into the language tooling itself.
Besides speed, what are the advantages of Elixir/Phoenix that a lot of fintech startups use them instead of Ruby/Rails?
Failure handling. Load profile is slower but far smoother, it degrades far more nicely.
It can handle stateful things and distributed systems far nicer, with a programming model that is really adapted to it.
I'm still not a fan of tail regression, er, recursion.
Why not? The recursive call gets optimized away, so it's just as fast as a loop.
good news: For 99.9% of use cases, you can use the Enum module.
I would perhaps agree that having to structure the code in such a way that TCO is "triggered" can be a hassle, but once you realize the pattern and learn it, it's not hard at all to refactor your code to trigger TCO. And it allows you to enjoy the semantics of an infinitely-deep stack without actually needing an infinitely-deep stack :)
Why not? I've always enjoyed it in languages that support it.