PulseAudio and Systemd Creator, Lennart Poettering, Reportedly Leaves Red Hat
phoronix.comMy god the comments in this thread are terrible. How can grown ass adults be so pathetically mean to a single person because he wrote (free & gratis!) software they don't like. Seriously, it's mystifying.
When his employer uses politics to try to coerce others into using that "free & gratis" software it can leave a bit of a bitter taste in your mouth.
It's not that something like pulseaudio is bad so we can just ignore it, it's that pulseaudio is bad because because redhat funds a lot of gnome stuff gnome won't really work unless you use pulseaudio, and since you use gnome you now have to use systemd. If pulseaudio just existed in a vacuum I don't think anyone would have an issue with it. If it wasn't bundled into a giant ecosystem it wouldn't be so bad, and if the redhat contributors involved cared more about non-redhat stakeholders I don't think people would have nearly as much ire.
But honestly writing "free & gratis" software that kind of sucks, and that forces you to use other sucky software, well I'm honestly not sure it's better than nothing. If it didn't exist I think some other solution would eventually come into being, and it might very well be a better solution. Sure it might not happen as quickly, but personally I'm alright with that.
It's not charity, it's not nobility, it's just redhat building a moat around their core business.
Desktop Linux is not Red Hat's core business and it hasn't been for a long, long time. Way before pulse or systemd.
This is, fundamentally, a gross misunderstanding or misrepresentation of the way that open source works in general, and of the way Red Hat-employed developers in particular interacted with the ecosystem.
I was in the engineering side in various positions for nearly a decade. The policy was upstream first. Open source first. If you think that the engineers at Red Hat didn't care about open source, or that anyone WANTED to invest god knows how many man hours into trying to create a sane experience from scratch, you're badly mistaken.
Pulse was optional in gnome-settings-dameon for a long time, and the primary reason it's deeply integrated now is because people LIKED IT (the horror!), since it finally solved a bunch of problems with Linux audio, and the dbus backend made stuff like "media keys work out of the box" seamless.
Similarly, GNOME has a dominant place in the market because... people liked it. It's market competition. GNOME had/has a place in Red Hat as the de-facto user experience for Fedora, and Fedora's principal role is user experience testing/feedback for the next version of RHEL, at all levels of the system, but the number of users who are gonna run Fedora without a shell is low.
Principally, a lot of Red Hat's engineers ran (and probably run) Fedora as a development workstation, though there was a fair amount of Gentoo, Arch, Slack, and others.
The only reason why there's a perception that "non-Red Hat stakeholders didn't matter" is because the vast majority of community users weren't submitting patches, and the other big vendors (SuSE, Canonical) were working on other DEs (KDE, Unity) which only shared parts. EndlessOS and other projects were GNOME because... it worked.
Pulse and systemd came into being because better solutions did not, despite a bunch of effort. I've used Linux since the 90s. ALSA was better than OSS, but it had enough thorny edges that JACK still had to be a thing (and still is, for audio engineering). Pipewire is an iterative improvement on Pulse. Audio will never be done.
OpenRC, Upstart, sysvinit, runit, and others were all terrible in various ways. The sheer existence of stuff like supervisord speaks to the existence of that. If I'm going to bet, it's always safe that whatever "booo systemd" person I'm speaking with has never actually had to maintain, troubleshoot, add features to, or otherwise work with pre-systemd init systems in any meaningful way.
Red Hat's business is consulting (or at least that's what "platform" is, discounting OpenShift, but "platform" had been stagnant in revenue for a bit, which precipitated the buyout). It is not selling Linux. It is getting paid to help companies make Linux work for them so they can save money in other places.
It is in writing features which make Linux better because some customer needs/wants it and has a bag of money, but not the in-house expertise to write it.
It is not desktop Linux, and it hasn't been since RHEL3 (or maybe before that). Any argument predicated on that is built on quicksand. Don't make a Texas sharpshooter fallacy.
You have (had) a bunch of very smart people who were very passionate about open source working in Linux every day, at a company with an "upstream-first" policy. They were actively working with/against desktop Linux user experience at airports, at home, on newer laptops, trying to get on Bluejeans calls, etc. It should not be surprising that they also saw problems and wrote solutions, which the gratis marketplace liked enough that they have widespread adoption.
You can personally dislike those technologies, but condemning their existence is just a different way of saying "I don't like open source when ideas I don't like succeed".
The "upstream first" way that Red Hat-employed developers worked mostly seemed to involve pushing all the buggy bleeding-edge stuff upstream so that they didn't have to deal with rebasing it, whilst keeping the work of cherry picking the parts that were actually functional and resulted in a working system as Red Hat's proprietary edge. I've had to pull important PulseAudio bugfixes out of Fedora packages before now because they literally did not exist in the upstream version, and their stable Linux kernel patches - an area they actually make money off, unlike desktop Linux - are effectively proprietary, with customers not allowed to reveal what's in their patchset and co-operation with the upstream stable releases that everyone else uses basically non-existent.
This is the opposite of how it worked. Again, I left a couple of years ago, but the methodology was "get a bug/RFE downstream, reproduce upstream if it's still there or implement the feature, get it through code review and merged into the codebase, cherry pick it into RHEL, making whatever changes are necessary".
Fedora is approximately as "vanilla" as Arch. If any patch was in a Fedora package which you had to pull out, it's because upstream did not accept the patch for some reason, but the maintainer decided it was critical enough to diverge. This isn't/wasn't SOP.
The Linux kernel patches are, as above, not really where they make money, which is consulting. But there is no such thing as an "upstream stable release" which Red Hat contributes to in any meaningful way.
Kernel features which are going to go into Fedora go into mainline. Not the LTS release. RHEL's support policy has a somewhat strict SLA on kernel ABI/API, and their support cycle is much longer than upstream "LTS" which "everyone else" (whomever that is, because Canonical and SuSE have policies similar to Redhat) uses.
Patches are cherry picked from mainline not to the "upstream stable" release, but to whatever version of the kernel RHEL-whatever shipped with. 2.6.18 for RHEL5, 2.6.32 for RHEL6. Forever. It has been this way essentially forever is not a new change, has nothing to do with excluding the rest of the community and everything to do with the fact that it was the only way that nvidia/emulex/whatever_hardware_vendor would agree to support Linux AT ALL. They were not going to rewrite their driver for RHEL3.5. Just RHEL3, now and forever so the ABI had to stay the same.
The patchset is publicly available as part of the SRPM, and as a raw .patch file. It is not individual patches anymore because Oracle was cherry picking patches from their cherry picks to repackage Redhat's source and poach customers.
It was moved to a single, enormous patch file so Oracle's kernel engineers actually had to work to integrate their ksplice/RAC/whatever patches, but that's politics.
Forgot to mention the best part - as far as I could tell, the package maintainer basically was upstream at the time. (I think both might have been maintained by Poettering himself, but it was a long time ago.) So in order for patches not to make it upstream the same person had to decide that it was important enough to diverge from the upstream release but not important enough to fix for everyone else. Oh, and one of the times I ran into this involved a seemingly really easy to reproduce crash (at startup, if I recall correctly) in a released upstream version...
You can whitewash history all you want, but kernel maintainers hated working with the pulseaudio team. This went so far that even Linus gave that team an Nvidia-caliber piece of his mind.
> project maintainers and project contributors don't agree. News at 11
This is just open source. "kernel maintainers hated working with the pulseaudio team" is not news any more than "Theo de Raadt/Ulrich Drepper dislike some ideas of others". It is part and parcel of open source development. It doesn't need 'whitewashing'. This is how quality software is built.
Conversely, even referencing that the LKML was full of angry messages (again, LKML having drama is like the sky being blue) about pulse (or kdbus from the systemd team, which is also a thing) is just evidence that Redhat did and does work with upstream rather than having some walled garden which they jealously guard from the rest of the community.
> Desktop Linux is not Red Hat's core business and it hasn't been for a long,
>It is getting paid to help companies make Linux work for them
There's an obvious joke there :p
But I think it's reasonably clear that I wasn't saying they were trying to build a moat around desktop linux, but around those enterprise offerings. One part of that is making their init system the defacto standard.
>The only reason why there's a perception that "non-Red Hat stakeholders didn't matter" is because the vast majority of community users weren't submitting patches
I don't think I can get into it without being rude, but I think that hasn't born out history. If an outsider does submit patches to scratch their own itch they can expect to at best be held to a much higher standard and at worst be treated quite rudely. I recognize that that's going to be subjective but there's definitely some bad blood between gnome and many third parties. Look at lxqt/lxde, appindicators, etc.
> There's an obvious joke there :p
> But I think it's reasonably clear that I wasn't saying they were trying to build a moat around desktop linux, but around those enterprise offerings. One part of that is making their init system the defacto standard.
Frankly, no, it was not, or I wouldn't have written that. Those "enterprise offerings" have zero interaction with Pulse outside of some edge cases in SPICE for VDI.
The only "moat" that "they" were building (and by "they", you mean "individual contributors who happened to have a @redhat.com email address") was in making desktop Linux usable enough for their engineers to work on it day-to-day.
As I said elsewhere, RHEL6 used upstart. "They" made "their" init system the defacto standard by using upstart, finding a metric ton of things wrong with it (and upstart was used in the first place because maintaining complex sysvinit scripts and day 2 operations in horizontally scaling applications was such a nightmare that it was worth throwing away all of the accumulated man hours in those scripts), and writing something better.
Nobody forced the Debian steering committee, Arch, Gentoo, Slack, or anybody else to default to systemd.
> I don't think I can get into it without being rude, but I think that hasn't born out history. If an outsider does submit patches to scratch their own itch they can expect to at best be held to a much higher standard and at worst be treated quite rudely. I recognize that that's going to be subjective but there's definitely some bad blood between gnome and many third parties. Look at lxqt/lxde, appindicators, etc.
GNOME is what it is, and they had that reputation before Pulse.
We shouldn't conflate Pulse and systemd just because they were both Lennart any more than you conflate selinux and podman because they're both Dan Walsh.
This perception also grossly misrepresents or misunderstands open source. If an outsider does submit patches where? Which one of the 600 different projects are they submitting to? How many of them do you presume are controlled by black-robed @redhat maintainers?
OpenSSL, Apache/httpd, coreutils, glibc, and a bunch of other core projects have so few maintainers/contributors that we're lucky they move at all. There is not some "Linux@redhat" repo which it all goes to where patches from outsiders get gated. Open source is a reputation-based community. If a @redhat.com email happens to be a maintainer of a widely-used project, it is, broadly, because they earned it through contributions like anybody else.
Maintainers, once they get there, do tend to get less rigorous code review. That's true. But nobody hands you maintainer rights along with your badge, and the perception that things work this way is untrue both historically and today.
I don't think we're going to reach and real agreements here, a lot of these things are going to be subjective. I think there's a certain amount of "if everyone around you is an asshole maybe you're actually an asshole" but I doubt I'm going to be changing anyone's minds today.
That being said there is a bit of hard data, this post nicely shows gnome contributions over time, as less and less non-redhat developer contribute.
https://hpjansson.org/blag/2020/12/16/on-the-graying-of-gnom...
Now I'm sure you have a dozen explanations lined up for why that might be the case, most of them involving how good redhat is and how they're a bastion of light all that, which sure I have no way to definitively prove that there's any malfeasance going on.
Still, I think it's clear that at the very least gnome has a bit of an image problem going on. Where we're not debating any kind of concrete facts as just shouting our subjective opinions of who's the asshole at each other, well I don't expect us to come to any sort of an understanding.
Frankly, there is no agreement to have here. You are claiming something without any sort of experience/knowledge, which is false, contributing nothing to the topic. This happens all around by similar people under every systemd, gnome and wayland thread. I just don’t understand why are you (plural) pushing a conteo where some evil, open-source hidden government wants to take over the word by.. giving away free software, killing other free software?
> I don't think we're going to reach and real agreements here, a lot of these things are going to be subjective. I think there's a certain amount of "if everyone around you is an asshole maybe you're actually an asshole" but I doubt I'm going to be changing anyone's minds today.
I don't think we'll reach any real agreements either, but I also don't think it's all that subjective. We could say my perspective is skewed from being "behind the curtain" for a decade, but it's equally true that I'm dramatically more informed about the economics and realities of open source contributions due to it.
Redhat was (and likely still is) a collection of engineers who are passionate about open source, and get paid to work on it. For a long, long time Redhat paid below the market average. It was not a place you went right out of college. It was a place where people who really gave a damn about open source went because the money was enough, and they got to work on something they felt was actually making software better.
It wasn't some corporate machine, it wasn't particularly profit-driven, and it definitely wasn't somewhere where the engineers wanted to push some corporate vision of how Linux should be. As of maybe 2017 or 2018, we started hiring more "new grads" and pivoting more to cloud workloads (outside of what Openstack already had), but the fundamentals didn't really change.
The perception of what Redhat is/was and what they actually were/are? is very different. It wasn't a perfect place, and the best ideas didn't always win, but at least you always felt like you could voice them, and you never felt like you were railroading a project to build a "moat" around some business model. Good projects got a lot of developer/admin/ops mindshare, and those dominated the market, so you should write good software which people will like/use. It wasn't complex.
One of the things which I miss the most (outside of being surrounded by people who were passionate about software ethics) is QA -- Redhat's QA is, in no small part, what has kept software quality high.
> That being said there is a bit of hard data, this post nicely shows gnome contributions over time, as less and less non-redhat developer contribute.
> https://hpjansson.org/blag/2020/12/16/on-the-graying-of-gnom...
> Now I'm sure you have a dozen explanations lined up for why that might be the case, most of them involving how good redhat is and how they're a bastion of light all that, which sure I have no way to definitively prove that there's any malfeasance going on.
Try not to turn people into strawmen.
The likely explanation is that GNOME has gotten much more complex and harder to contribute to, and the divide for new contributors to meaningfully fix bugs or add features has gotten wider as GNOME3 has stabilized (and bloated a bit). There's a clear pike just before 3.0, and it kinda declines after that, but some of the other major contributors (Ximian, Collabora, arguably Sun) aren't players anymore.
The other missing piece of data here is that, outside of the kernel (which gets a ton of contributions from hardware vendors submitting code to support their own stuff), you'd find the contribution ratio to be shockingly similar. glibc, openssl, and a number of other core projects are also more Redhat than not.
That doesn't mean they're trying to control the ecosystem. It means that, yes, it contributes to the company's bottom line, but you shouldn't assume that the contributions have a motive any more sinister than any other contributor.
GNOME's image problem is the same as GNOME's image problem always was, but we really haven't been talking about that. I'm not trying to defend Redhat or call you an asshole, but the amount of commits Redhat makes to open source projects all over is not subjective, that they get those commits upstreamed into projects they do not control is not subjective, etc.
Don't let your personal opinion about Lennart color what Redhat engineers do for open source or to paint all of them with the same "corporate peon building a 'moat' brush" (which also doesn't describe Lennart).
You don't agree with some of the decisions the communities made. That's fine. That's open source, too. It doesn't mean that you're right and Redhat is some Benedict Arnold scheming for control.
>Try not to turn people into strawmen.
As an aside, part of my frustration with talking to you is that specific problem. You're very early to dismiss complaints with things like
> (me) If an outsider does submit patches to scratch their own itch they can expect to at best be held to a much higher standard and at worst be treated quite rudely.
>> (you) This perception also grossly misrepresents or misunderstands open source.
Now I can point to at least a few instances of that happening, but I know from experience that the goalposts will likely move again. I can point to a well respected SDL developer saying "this architecture is not how anything else works and will cause significant issues for the SDL project, and therefor many games on linux" and a gnome developer saying "Fix the applications and libraries that claim the support Wayland, but don't do it properly.". Now even if they were right on a technical level (they weren't) that's no way to talk to your fellow open-source developers who were being very polite and explaining why the proposed architecture wouldn't work.
For that particular issue at some point gnome developers did step back and implement CSD.
But regardless, "this perception also grossly misrepresents or misunderstands open source" is completely out of line and where I decided you weren't worth talking to. I can point to multiple specific instances where gnome developers were rude, unhelpful, arrogant, etc, but I'd rather not drag into that. Like I say it gets insulting very quickly. The simple fact is that their reputation for being assholes is, on the whole, earned. I'm sure there are specific cases where it doesn't happen but assholery seems to be endemic to the entire organization.
If you want specific documentation on times systemd has caused significant issues (not counting the pwnies that are well documented) I've started documenting my experiences here (unpublished, and something that I'm unlikely to ever publish): https://traverseda.github.io/scratchpad/systemd.md.html
Or see this article here: https://igurublog.wordpress.com/2012/11/05/gnome-et-al-rotti...
Or this rather infamous interaction: https://trac.transmissionbt.com/ticket/3685
At some point I suppose I'll get around to documenting the times gnome developers were publicly assholes to other open source contributors, but honestly I'd rather let it go and just ignore the whole project as much as I'm able to.
>I can point to multiple specific instances where gnome developers were rude, unhelpful, arrogant, etc
I too can point to multiple specific instances where Linux developers were rude, or where gcc developers were rude, or where SDL developers were rude, or where a lot of other developers were rude. I bet someone could dig through your comment history and find instances where you were rude and unhelpful too, probably even more if you shared any comments from when you were a young teenager. I hope you can see that it is completely ridiculous (and somewhat creepy) for someone to do that to you. Open source programmers just disagree sometimes, and sometimes they have a bad day and aren't totally professional about it, and in larger projects it is a lot more visible than on smaller ones. All you can ask of them is for an apology and then move on. It is unreasonable to expect anything else or to keep harping on this years after the fact.
Also it is completely ridiculous that this thread is getting derailed again into complaints about missing features in GNOME. Lennart isn't a GNOME contributor, I don't think he has written any significant code for GNOME at all. The fact that nonsense conflation is being used should be a "snap back to reality" moment. Just don't do it. This always seems to happen and it's always from the same suspects trying to revive the same old >=10 year old forum threads. Please give it a rest and try to do something positive instead, don't let the lwn trolls get inside your head. You are generalizing an entire group of people based on some cherry picked interactions that both of us likely lack the context to fully understand, unless you were "behind the curtain" like the parent comment actually was. If you weren't, then it is just another tone policing comment from the peanut gallery, the kind you probably would find to be bad if they were made from random users on your projects.
>At some point I suppose I'll get around to documenting the times gnome developers were publicly assholes to other open source contributors, but honestly I'd rather let it go and just ignore the whole project as much as I'm able to.
Letting it go is probably the right choice. If you want to revisit it, I have a suggestion: why don't you go through and publicly document all the times they did something right or the times they were nice to people, and then praise them for that and encourage them to do more of that. Surely you understand that it is not useful to keep beating a horse that died long ago. Just focus on the positive, if the worst thing they did is send some slightly unprofessional email 5-10 years ago (as if most open source developers in the 1990s-2000s didn't do that at some point, I think I lost count of the number of flamewars I read from that period) then it really should not be hard to find some examples of good.
>This always seems to happen and it's always from the same suspects trying to revive the same old >=10 year old forum threads.
I think that has something to do with redhat defenders/employees needing to nitpick every little issue that anyone has in a public forum. The inability to just say "oh well that's bad but we're working on it" and instead try to mitigate or distract from any legitimate issues. Any discussion about this issues always turns into the same sprawling discussion where I'm asked to prove a bunch of subjective things. I try to illustrate that third parties have issues with redhat engineers and submitting patches. I back that up by showing that the developers of lxde, transmission, and SDL, have all had significant issues with the gnome development team (mostly redhat), and I'm told it's creepy to point that out. But those are some of the cases that gave redhat this reputation.
You can't have a negative opinion about redhat/gnome without a bunch of people crawling out of the woodwork to nitpick and force you to justify every negative opinion to absurd amounts. Most of those people aren't just regular users, they are or were directly involved with redhad or gnome. It's impossible to have a straightforward debate, it will get derailed into a bunch of minutia.
My main complaint is that redhat encourages tighter coupling between their components that I think is reasonable, that's what I said in the root comment and I stand by it. The fact that I'm not debating a bunch of minutia with a bunch of redhat fandboys is frankly endemic to the problem.
Tell me, have you been paid to work on systemd/gnome/redhat in the past?
>I'm told it's creepy to point that out. But those are some of the cases that gave redhat this reputation.
Yes, I think it is creepy. Don't you think it would be creepy if someone was in a completely different thread in a conversation with someone else who had nothing to do you or with your decisions, started going through your 10 year old comments looking for disagreeable things, and using them as an excuse to discredit everything that you might have been slightly involved with? This person isn't a GNOME developer and he doesn't even work for Red Hat anymore.
>It's impossible to have a straightforward debate, it will get derailed into a bunch of minutia.
Please don't try to make this into a debate. This is not a constructive way to approach anything. If you look at this conversation as something you need to "win" then that would be the core of the problem. It is perfectly fine to express a negative opinion, but what is not okay is when you start saying things along the lines of "I am going to prove all the developers of XYZ are permanent irredeemable assholes". That is just furthering the flamewar, it's not moving past it into the actual issues.
>Tell me, have you been paid to work on systemd/gnome/redhat in the past?
No. This also is a somewhat trollish question that is unnecessarily making things personal and leans on a stereotype, in the future you shouldn't lead a conversation with questions like this. In particular, people can change their mind on all those things despite being paid in the past, or could be paid and still have a bad opinion of them, or could not be paid but still have a good opinion, or any number of other things that are outside the preconceived notions. A much better and friendlier way to phrase this would be something like "What is your experience with these projects/developers/companies?" Then the other person can divulge their employment status at those companies at their own accord, if they have a positive view of the employment then they probably will.
Putting this in a separate comment thread to avoid filling up the other one. I have some extra time so I read your blog post and wrote some comments. There is very little in there that is any kind of actionable issue.
>Sure, you can pipe journalctl into grep, but what if you want to use inotify, or tail a log, or expose logs over a network share? Well you have to learn the systemd-specific solution, instead of just using the tools that work with every other file.
This is absolutely not different from any other tools to manage structured data, like any database. I don't see anyone complaining that postgresql doesn't follow the unix philosophy. Logs are actually structured data, any adherence to a philosophy cannot change that fact. Actually a lot of those "standard" unix tools specifically exist to parse unstructured data into structured data, so from that perspective, having the format already in a structured data format is only there to save you some typing.
>I could complain about service files being spread all over the file tree, instead of in one central obvious place, but as I understand it systemd has some reason why that's better for them and that's fine.
The reason is straightforward. /run/ is for temporary services. /lib/ is for immutable service files shipped by the OS or by programs. /etc/ is for service files defined by admin configuration. /home/ is for service files defined by user configuration. This is all standard filesystem stuff that systemd is following. Actually, it is the old-style inits that break convention by putting everything in /etc, causing bugs in the process. The package manager should not install files to this folder that are not intended to be modified, doing so is sure to cause things to break whenever you upgrade a package. What seems like a "simple" solution in this case is actually too simple to the point of being broken.
>Unfortunatly the built-in OS image was not running systemd, and the kernal was several revisions out of date. While I had no problem getting a debian chroot running, all of the services were designed to run under systemd, this made the whole project much more of a pain in the ass than it should have been.
Well I don't see what this has to do with systemd, this would happen if you want to use any program or driver that depends on a new kernel version. Systemd cannot really do anything to solve the fragmentation caused by embedded devices running ancient kernel versions, that is very much a Linux-as-a-whole problem.
>I took the boot media out for trouble shooting, but when I systemd-nspawned into the host to try to address the problem, I discovered that journalctl would segault. Thankfully /var/log still had all the entries I needed to fix the problem. This appeared to be a general issue with running journalctl using qemu-static and binfmt.
A segfault sounds like an actual unintended bug that should be reported.
>By default systemd will kill long-running processes when I log out. Processes like screen or tmux.
Nit pick: this is not systemd doing this, but logind.
>The intentional and willful breaking of screen and tmux was to fix a GNOME bug of GNOME not closing up as it should when the user logs out
No, this is extremely wrong. GNOME cannot actually do anything about this, if a random nohup processes decides it is going to try to keep itself open.
>There's now no way to, by default, keep a program running in the background without a live ssh session.
The default is a flag in the config file that distros get to decide. There is nothing else they can do to accommodate you here, besides do what they already have done.
>For some reason my mother's computer can no longer resolve DNS. Ripping out systemd-resolved seemes to have fixed it, but note before I lost a few more hours.
>I used to think the systemd hate was silly... until I tried to get a VPN running and realized that all my DNS requests were going through a mysterious local DNS server
Unclear what is going on here without additional info. Copying more random comments that say vague things like "it doesn't work" is not really useful or meaningful to anyone. You could fill an entire book of comments like that made about everything in Linux over the years. It might be humorous, or interesting for historical purposes, but it won't help get any outstanding issues fixed.
>the graying of gnome
They broke their own user -> developer pipeline through years of deliberately alienating power users.
Never saw anything wrong with upstart. I used it for a few years and was pissed when it was unceremoniously dumped because of weird debian politics, effectively killing the project.
I'd have had more respect for systemd if it truly were crowned because people liked it rather than because of a ~7 person vote.
RHEL6 used upstart also. I had to write/maintain upstart files, too. systemd picked up steam inside Red Hat because of the number of edge cases upstart DID NOT handle.
It didn't do socket based activation at all. It didn't handle day 2 operations well (if you changed the configuration of some parent service and the children needed to be restarted, this was just as manual as sysvinit but with a clunkier format). overrides were terrible. Failing out on a depchain was problematic.
The debian politics never would have gotten there if Lennart (and others) didn't decide that it was easier/better to start from a clean slate instead of upstart's weird middleground of a "script" verb to kinda sort make transitioning from sysvinit scripts easier, but not a wholesale reconception of what an init system needed to be in 2015 or whatever year that was.
systemd was, in a sense, crowned because Redhat also tried upstart and had to suck lemons for an entire release/support cycle, wrote something better, and Debian followed on. Not because Redhat/others didn't try to make it work, or because of 7 people.
>It didn't do socket based activation at all.
Pretty sure it did actually.
>upstart's weird middleground of a "script" verb
What middleground? Upstart files werent scripts like sysvvinit they were declarative just like systemd.
I can see why red hat dumped it - they were clearly keen on "owning" PID 1 and would prefer not to depend upon ubuntus project.
It's the 7/8 person debian vote (split 50/50 and swung by one vote) that I found mystifying. It not only crowned a worse designed project burdened with a massive case of feature creep, that was weirdly unresponsive to bugs it killed off its competition.
Systemd was voted on multiple times by debian maintainers in one of the most democratic votes possible and always won hands-down. It was also independently chosen by several other distros including Arch.
Things like systemd suffer from the xkcd traffic engineer syndrome:
Same thing with cars that were designed in the 1960s before we had to worry about things like emissions and crash safety.
Sysvinit is great up until you have a use case that it winds up sucking for. Most people don't hit that problem, or don't realize that their high level problem (correctly handling something being hotswapped probably) requires better integration than that.
They're also pissing and moaning without any solution that addresses all the requirements, and just use "Desktop" and "Enterprise" like they're slurs rather than what should be tablestakes.
And I wrote some truly awful code 10 years ago dealing with an abstraction layer built over the dozen different service managers than existed back then (upstart, runit, etc, etc). I'm quite happy that my code turned out to all be ultimately quite useless overdesign in retrospect and systemd just took over everything. It was a mess back then.
OpenRC is pretty good and I think it addressed most of the issues systemd was originally intended to solve. Let's all just use it.
Convince Debian and Ubuntu to adopt it.
If systemd is just evil garbage pushed by RedHat and OpenRC solves all the same issues, they should jump at it.
Gnome depends on it, so that would be challenging. These days I just use alpine for servers where I can.
No, that is very wrong. GNOME can be used on it if you use elogind.
Truthfully, they will not adopt OpenRC because OpenRC is not at all comparable to systemd.
gnome works on FreeBSD, so that can't be true. It probably uses the logind fork though.
> Let's all just use it.
How about you use the software you want to use, and I'll use the software I want to use?
>How can grown ass adults be so pathetically mean to a single person because he wrote (free & gratis!) software they don't like. Seriously, it's mystifying.
Because he and his orbiters were extraordinarily arrogant and condescending about their software. And let's not forget that while it's pretty good now, in the early days systemd was not very good at all. Because things like GNOME 3 decided to make systemd a hard dependency, pretty soon it became nearly impossible to avoid using the entire systemd stack even though it was nominally modular.
It's not just that he wrote $thing that someone didn't like. It's that $thing got shoved down everyone's throats at a time when $thing didn't work very well, and then when people rightfully objected, they were belittled by Poettering and his fan base.
But it's not Poettering, it's the Gnome folks in general. These controversies have been around since the early Gnome 2.x days. Nothing much has changed.
What we really need is for people to propose alternatives to systemD that address its purported use cases while removing some of the needless complexity it introduces, and restoring the primacy of general mechanisms over hacked-together policy. This has happened with some components, e.g. elogind but not really as a comprehensive effort.
>But it's not Poettering, it's the Gnome folks in general.
It might also be the Gnome folks but it's definitely also Poettering. Let's not pretend he's some blameless martyr. The guy's a jerk.
OpenRC and alpine are nice.
OpenRC is the only Linux init I've used that I actually enjoyed using, not just could-tolerate.
What was wrong with Slackware's bag full of shell scripts?
Maybe I'm misunderstanding what people want an init system for, but I'd expect to need little more than "run these 10 tasks. Potentially, wait for A to finish before starting B". Its job should be over when you see the login prompt.
It feels like modern inits are more like a watchdog/messaging system-- which seems like something which would be an optional add-in atop a real init system.
I want to run a custom utility at power on. It's not a weird expectation. This should take no more than one line of code to achieve, adding "/usr/local/bin/toolname &" to some sort of rc.local file.
sysvinit did nothing wrong.
this is the way.
Also because "mean" is the default state of far too much of humanity these days…
I agree that the comments in this thread are uselessly snarky and uncivil, but Lennart Poettering is not just a single person who wrote software people don't like. PulseAudio has, deservedly or otherwise, been the face of audio issues in desktop Linux for coming on two decades now, and the tumultuous changeover to systemd as very nearly the mandatory init system for Linux left a sour taste for many long-time users.
I don't think anyone would care (probably most of us here wouldn't even know his name) if these projects had been written and integrated such that it was trivial to avoid them in favor of alternatives. No-one who hated Upstart, say, is pissed off at the people who made it. Between Red Hat's maneuvering and Poettering's choices, though, it's not trivial to avoid them.
It's very trivial to bypass those: just delete the packages off your system. Of course this means your system now won't boot, and you must expend effort to deploy a replacement. That's the actual non-trivial part. There's nothing the systemd developers can do to make that any easier for you, unless you expect them to suddenly start developing two or more init systems for some rather unclear reasons.
If the problem is that your employer has required you to only use packages supported by some vendor, that's a different problem and it extends to a lot of other things besides systemd. For some perspective you might want to go talk to some people who still have to use COBOL at work...
OK so in some hypothetical sense it's fine, but in practice it's a pain in the ass to avoid. So let's go with I dislike systemd in practice.
I still don't know what point you're trying to make though. To me this is like if you said "in a hypothetical sense planes are fine because you can buy a private jet, but in practice it's a pain because you know I have to fly in a crowded coach seat". Like, ok, sure, that's technically true, but still no one is going to donate you a private jet. If you had another hypothetical replacement and you could afford to spend all this time switching the company servers over and training everyone else to use it then maybe you could do something, but you don't, and the systemd developers could not help you with that even if they wanted to. What do you want anyone to do about it?
To quote myself:
> Between Red Hat's maneuvering and Poettering's choices, though, it's not trivial to avoid them.
I'm just explaining why some people dislike Poettering's work, while typically having no strong opinions about the careers of every single other author of init and sound daemons, whether or not they like the software. It's a combo of 1) disagreeing with their choices and methods, and 2) the fact that those choices and methods have made his stuff much harder to avoid than the alternatives. That is why people don't like him (and, indeed, probably the main reason so many people know his name at all)
>It's a combo of 1) disagreeing with their choices and methods, and 2) the fact that those choices and methods have made his stuff much harder to avoid than the alternatives.
That doesn't follow unless you consider "developing a software that becomes popular" to be a method or a choice. Again, I don't know what you expect them to do or what other choice you expect them to make. Should they have suddenly quit once it was clear that it was going to become popular? What good would something like that do?
>That is why people don't like him (and, indeed, probably the main reason so many people know his name at all)
This also doesn't follow, it doesn't make sense and is extremely unprofessional. It isn't good for anyone's mental health to hold grudges like this. In order to attempt to have an unbiased view, you have to be able to separate the work from the person. Though I realize asking for this is probably an unwinnable battle in open source. Now that he doesn't work for Red Hat anymore it will be interesting to see people keep trying to blame both him and Red Hat for this.
> That doesn't follow unless you consider "developing a software that becomes popular" to be a method or a choice.
THIS doesn't make sense unless you consider that to be the only thing that happened.
If it had been, this story would never have made HN in the first place.
I think that is the terminal reason for your complaint, if it was unpopular we probably also wouldn't even be talking about this.
OK, this has to be intentional at this point. Bye, HAND.
I don't know what you mean intentional. The problem is that you feel pressured to use it, no? Well it seems that pressure comes mostly from its popularity, or you would be using something else.
But these things are inherent to the problem domain.
Sure, noone will be as angry at a Paint-clone’s maintainer because, well just use something else. But it is not trivial in case of non-trivial problems like booting and audio, both requiring some standard IPC communication besides the inherent complexity of the program itself.
> PulseAudio has, deservedly or otherwise, been the face of audio issues in desktop Linux for coming on two decades now
It is actually very hard to say in reality whether some other product would have been better.
The more users software has, more issues there will be. If there is ”de facto” way to play audio, all conversation focues on this ”de facto” way. It has been also kinda first of its kind in this scale. PipeWire might work well now, but they had excellent example what to not do or how to do otherwise.
The biggest problem with systemd is that it broke with so many "established practices" and philosophies of a lot of distributions and iirc it took a while for generators to appear that automagically integrate old-school sysvinit scripts into systemd, meaning an awful lot of work for distributions to recreate unit files with sometimes decades of history buried in the old init scripts. Additionally, its overhaul of logging broke established file-based workflows.
Personally I'm happy with systemd these days - it has a steep learning curve but once you get how to write them and how dependency ordering works, it's just a breeze.
If there is one thing the Linux ecoysystem desperately needs if it ever wants to be a serious challenger on the desktop, it's standardization for software vendors... and slowly but surely, life gets easier, in no small part thanks to systemd.
I gather that there is a spectrum of views regarding LP, but I'm in the "thanks for making this mess relatively uniform" camp.
The combination of unbridled arrogance along with the dubious technical merits of your all-or-nothing design decisions, can lead to that; so, hardly mystifying.
Why does systemd need to mess with user processes and home directories? It was billed as a replacement for init.
Then don't use it. No one is forcing you, there exist alternatives. Attacking someone for making an optional thing seems kinda like the problem lies in you.
Might be worth pondering what Poettering and Red Hat did that make people hate their work so much, and to make Poettering uniquely infamous in the open source world. No-one's upset with the creators of dozens of other init systems and audio daemons, including ones that are pretty bad, and even ones that were promoted by major distro vendors.
From my point of view as an old guy who remembers the hell of init scripts, what they did was make an alternative to remembering arcane details of how each os and distro chose to set up their scripts. This:
* made debugging a random system a lot more straight forward
* made putting up software for packaging a lot easier
* made customizing the start-up of a daemon a lot easier (this ties in with the first point)
* made it so I didn't have to go to the guru who's only job seemed to be managing our obtuse set of init scripts that were extra baroque because order of startup really mattered.
That guy I had to go to - he was an asshole an lorded his knowledge of arcana over anyone who brought him a question. I'm 90% sure he made it intentionally complex since he didn't have much skill beyond memorizing those silly facts (this is the same guy who got an entire project reassigned to me because he couldn't understand the concept of functions when I removed some extreme repetition from a python script he wrote). Most of the systemd haters remind me of this guy.
I guess what you're implying is that he helped automate away dead weight and that made the leeches mad?
These "simple" use cases break when you need to do anything even slightly more complex than whatever the systemD designers envisioned for the user. And if you try to submit an issue upstream, the typical response is "you're holding it wrong" or equivalent. This lack of a consistent design might be excusable when first trying to support a bunch of entirely novel use cases, but let's not pretend that it doesn't come with serious drawbacks.
These "simple" use cases break when you need to do anything even slightly more complex than whatever the sysvinit scripts you copied envisioned.
FTFY
systemd might be sad but makes the simple case simple. sysvinit makes everything complex and error prone.
sysvinit could be debugged, at least. I tried to debug a couple of systemd edge cases with Pottering, communicating over Github Issues, but he lost interest very quickly.
To his, credit he is reactive until he decides the issue is not worth his time.
To his discredit, he made himself into a giant bottleneck. Random systemd users can not easily understand it.
I have yet to encounter something that systemd doesn't do OK or well. What is your example of something that systemd can't handle that was better handled by init scripts?
I’m sure then you can list a use-case which doesn’t work under systemd. Please do so, otherwise you are not contributing much.
How many different servers of non-systemd and systemd types, have you set up and managed?
Where you had ultimate responsibility for the correct operation of the box and any VMs or services that it ran?
The reason I ask, is that despite running many more non-systemd than systemd boxes over the past 22 years, I have had far more time taken up by systemd issues than issues with other init script systems. And that includes Solaris both sysV and SMF based.
This, and it seems like it needs repeating every time somebody brings up systemd.
The SysV init scripts weren't good, they were crufty and slow, and messy when you wanted to customize something, and -- this doesn't get said enough -- used no Linux-specific features at all. Systemd was an upgrade that was sorely needed, and brought features to the init system that just didn't exist before.
I don't see any real evidence people hate their work. Some people do, but it's not like, an overflowing toilet or wet socks that almost everyone hates.
I personally wouldn't touch a non-systemd system if I didn't have to.
The thing they did to piss people off was breaking with UNIX tradition and not prioritizing simplicity. This makes them hated by enthusiasts, tinkerers, people who don't trust tech, etc, and loved or just passively ignored by corporate sysadmins, non-hobbyist everyday desktop users, general devs just trying to make modern software, etc.
It's kind of an unsolvable conflict. People who want simplicity are looking for something very different from people who want everything to just work and never require looking under the hood.
I'm actually strongly in favor of someone coming in and replacing a bunch of the bike-shedding horseshit in Linux with some "no, it's this way, period, live with it" if only to make distributing software for Linux less crazy-making.
I just think Poettering's (and, more generally, Red Hat's) taste is really bad, so I wish it weren't him(/them). Possibly when they finally achieve Systemd/Linux and the entire stack atop the kernel is stuff Red Hat likes & directs it'll all be worth it, I dunno.
FWIW I also haven't much cared for a lot of Ubuntu's attempts to steer the direction of Linux—they've just lost so badly at every attempt, that it hardly matters.
Ubuntu is wonderful as long as they don't do a anything new, and you're using a derivative not raw Ubuntu.
I'm actually not even fully sure on the details what they even do that isn't in Debian aside from maybe some driver stuff and assorted annoyances like Snaps. I just know to stick with Ubuntu derivatives because almost all large popular apps seem to be well tested and designed to work on it, until the community gets tired of it and probably Fedora or something takes over.
It's the perfect example of "I'd rather have a mediocre standard than something excellent but obscure".
I'm not a fan of Pulse, because of its lack of low latency, although It still offers a lot over raw ALSA, and PipeWire solves basically all of it's issues.
But systemd is a lot more comprehensive, it doesn't seem to leave any use cases out, aside from ultralight embedded and old school "Control everything yourself with imperative scripting" style setups.
you are kidding, right? It is like saying you can't critique US laws because you can move to Canada (technically true but it misses the point)
Overract much? It's not at all like that. First and foremost because using a different distro doesn't come with all sorts of application process to use the new one.
systemd-homed is not systemd.
This whole Lennart Poettering thing of naming and bundling otherwise completely independent pieces of software "systemd-${thing}" even when they are at best just sightly related to "things you might want managed for you in your computer" doesn't make a lot of sense, and doesn't help in his defense that "systemd is supposed to be modular".
My guess is that if PulseAudio had came later, it would be called "systemd-audio" or something like that.
Things which talk to the systemd dbus endpoints to add/stop/create/manipulate units are systemd-blahblah. This isn't that confusing.
If it dynamically creates systemd units or sets their properties (systemd-homed, systemd-networkd, systemd-resolved) using the dbus endpoints, it's systemd-something.
If it uses systemd libraries and lives in the systemd repo because it shares baseline docde, it's systemd-something.
systemd is modular in the sense that those components are build-time flags. For homed, for example: https://github.com/systemd/systemd/blob/main/meson_options.t...
Again, not confusing. Literally looking at the systemd repo for longer than 30 seconds would show you both why this it's considered modular, and why it's systemd-whatever
Pulse doesn't.
> Things which talk to the systemd dbus endpoints to add/stop/create/manipulate units are systemd-blahblah. This isn't that confusing.
It is because systemd confuses what should be split concepts into one big overarching thing.
'org.freedesktop.systemd1' (the actual systemd init D-Bus interface) somehow is still implied that it is vital to the entire systemd-something ecosystem while it's not really.
A example being systemd-hostnamed and hostnamectl complaining that 'System has not been booted with systemd as init system (PID 1). Can't operate.' [1] while in fact they use 'org.freedesktop.hostname1.${something}' [2] and that is supposed to not be bolted with the entire rest of systemd besides using sd_bus.
[1] https://github.com/systemd/systemd/blob/93258c7d72fae23c9f81... [2] https://github.com/systemd/systemd/blob/93258c7d72fae23c9f81...
This looks like a good first contribution.
hostnamectl DOES talk to sd_dbus, which DOES require systemd, which DOES require being PID1, and this message only pops up if you're in a container without the full subsystem running, access to the right cgroups, etc.
The naming is simply part of how dbus structures things, not to say that it's not part of it. I'm sure that, in retrospect, they wish they would have used "org.systemd...", but hindsight is 20/20.
If there's a scenario other than container without the right mounts/etc where this can come up, you should file a bug. If there isn't, you could see if you can submit a patch to improve the message.
It's not an indication that systemd confuses concepts. Its can indication that a) dbus is kinda shit, and b) I doubt if the systemd mapped out hypothetical dbus names 5 years in the future.
That looks like a bug, if you really have a need to do that you should report it.
It just shows the usual knowledge on the topic of its critics, not even knowing the target program.
sounds like you never read his actual writings on his actual blog. As far back as 2014, he was talking about building a base layer for linux.
People dislike his work for various reasons but are still forced to use it thanks, in part, to his employer promoting it aggressively (it's not like he did all this for free). That's a recipe for saltiness. Kinda like Bill Gates back in the day (remember that "Kill Bill" game, and other similar ones?)
It was XBill, and there was a fork called XLennart.
Funny how his detractors spend more time on jokes than on shipping a better init daemon.
I'm like 90% sure that at some point in that era there was one called "Kill Bill" on Windows, though I do remember XBill.
> Funny how his detractors spend more time on jokes than on shipping a better init daemon.
Part of this might be because a high percentage of his detractors already have multiple init daemons they prefer to his.
No one is forced to use it - you can always switch distro.
I also use hyperbole as a rhetoric instrument but in text seldom works.
At work you may not have that choice, especially since all the most-used (so, best supported) distros are deep into systemd now, or if you have a distro that you otherwise like but that has switched to systemd. It's possible to prefer something over alternatives but still think it's worse than it used to be, or could be better, or to very much dislike one part of it.
Again, much like Windows and other MS products. People effectively had to use them ("well technically they could quit their jobs instead, so they didn't really have to"—LOL), which sets up circumstances for some serious hate. To repeat what I've written elsewhere, no-one is upset with any other init system or audio daemon creators, even if they don't like the software. Something different happened with these projects.
> well technically they could quit their jobs instead, so they didn't really have to
There are jobs which force you to use Linux, and SystemD in particular? Where do I sign up?
> At work you may not have that choice,
If you cannot switch the distro, then you deal with it. That's the tyranny of not using Linux-From-Scratch. But that's always been the case; the distro maintainer make the decision what to package and what not. It's not different with the service manager systemd.
> since all the most-used (so, best supported) distros are deep into systemd
Ever wondered why? Because it is vastly more simple an implementation target than writing bash scripts. It's more easily tested in isolation and the documentation is outstanding.
> no-one is upset with any other init system
Oh, I remember the heat and hate that Ubuntu got when they switched to Upstart. The reason that systemd gets all the rage is because it is the largest target with the most innovation. People always get mad when things change. People *hate* change.
> or audio daemon creators
Pulseaudio had to plow the way and thus got all the heat for exposing tons of bugs in the alsa kernel drivers. And what else is there besides pulseaudio? PipeWire? This is the successor to pulseaudio, the next iteration.
> Something different happened with these projects.
Which is...?
At time of writing I don't see a single comment attacking him personally outside of the replies to you complaining about it. I was actually somewhat surprised to see the comments only complaining about his software (and even those are currently a minority of the comments) rather than going the usual route of calling him an awful person.
> because he wrote (free & gratis!) software they don't like
I don't think you understand what the story was.
That's the story; he proposed new software and his employer liked the idea and paid him to develop that idea.
End of story.
People getting mad because they are "forced" to use a free software system? That's the fan-fiction from the peanut gallery.
> That's the story; he proposed new software and his employer liked the idea and paid him to develop that idea. End of story.
If this was the story the comments in this thread would have been very different.
Nah. The comments in this thread are just fan-fiction of people that are unhappy with the actual story.
So why's no-one mad at any other init or audio daemon authors? Poettering just had bad luck?
Because there is no other* that actually works properly to even get half as popular?
* there is now pipewire which will hopefully overtake pulseaudio though
There were popular audio daemons before PulseAudio took over, though. It's not like it entered a completely empty "market", if you will. Nobody was mad at the authors of those, even though in fact they did also suck. Nobody seems upset over Pipewire.
So I don't think you've found the reason.
I don't know anything about Systemd or PulseAudio, but this Lennart Poettering must be quite a trailblazer based on the number of arrow shot into his back
Here's documentation about the old SysVInit "system" of cobbled together hacks: https://wiki.archlinux.org/title/SysVinit#Writing_rc.d_scrip... Note the missing features, which I would call dealbreakers: - How do I know if a daemon is running? What if you several "copies" of the same daemon, like multiple OpenVPN connections? - How do you test if your cron script runs with the correct environment variables when it runs at 4AM? How can you test it with the same environment that will be set at 4AM, if you can only run it only on the interactive bash shell? - Say you want to change an environment variable from a daemon shipped by your distro. Are you sure you included all " and ' on your arbitrary shell program pretending to be configuration? Can you lint your change or do you just pray, reboot and get an emergency shell? - Freaking dependencies. Parallel startup? Making your webserver automatically wait for the NFS mounts? Make you sshd not wait for NFS mounts? Make sure your sshd starts even if NFS fails, so that you can actually reboot the machine remotely?
Here's how this can be done with SystemD, it's all supported and it works: https://wiki.archlinux.org/title/Systemd#Writing_unit_files
I'm not talking about spherical cows, just basic server administration problems. Don't get me started on laptops...
Ugh, this thread! For years I pleaded with people to understand that there was value to being able to start up in a deterministic manner, but some shell scripts that don't actually work are supposed to be enough? Then someone finally solves the problem and it is written off as some kind of weird corporate control move. And people could still come up with some kind of alternative, but the open source movement that solved problems for people has been replaced by a hostile cargo cult.
At least I get to use systemd not just for reliable start up but also controlling the services that so many applications are made of nowadays. But it feels like some terrible Balkan conflict that just won't simmer down.
He can easily hop ship wherever he wants given the major impact he's landed in PulseAudio, Systemd and elsewhere. His design decisions have certainly been controversial but his impact is certainly agreed upon.
I've been using linux for decades and by far -- by a HUGE margin -- PulseAudio has been the most annoying and broken thing I had to use. I'm still baffled by how something as basic as audio output can be eternally fragile in linux! And before you say you might have quirky hardware: my daily laptop is a Lenovo X1 Carbon 7th gen.
Now (since 2021) I use PipeWire and tbh do not know to what extent pulseaudio codebase is still utilized in my system but it all just works now.
I'm also using and happy with Pipewire but I can't remember the last time I had a PulseAudio issue, across many systems.
My recollections of Linux audio issues are more from the days when one thing might want to use OSS and another ALSA and you might have to fiddle around and enable software mixing in some ALSA configuration files to get non-exclusive sound to work. And then some issues right around when Pulse first started to be a thing. But really it's been a "Just Works" for me for years and years.
Yeah, same; I can't help but think there is a vocal minority in these threads that thinks Pulse is broken.
I use Pulse for some pretty complicated stuff, too; I'll pipe individual streams to my speakers, if I want to show something to my SO (e.g., a reddit video) but want to keep, like, my music or my backgrounded game on the headset. I can do that with PA, and honestly pretty easily. AFAIK, no other system supports that, though admittedly I haven't used Windows in some time.
Far more complicated, I also have TTS bot I use in Discord sometimes, when I can't speak. PA allows me to push that into Discord; this is another thing I don't even begin to know how to set up on other systems. (The bot emits sound, so it's like a program playing sound, not a recording device. Discord thus doesn't recognize it as such. So I end up creating a fake input device, of sorts, and wiring the bot's output to that fake output (so it goes to Discord) but also to my own headset, so I can also hear the bot, as the TTS is far from perfect and will sometimes just make hilarious garbage of words.)
I also use Bluetooth, and that usually works. It certainly works more reliably with PA on Linux than it does w/ macOS, though it is my no means flawless. But that seems to be how bluetooth is: it sucks on every OS. There will be days it won't connect to Linux, and there will be days it won't connect to macOS.
I will say that Pipewire looks like it more solidly hits the nail: the thing I don't like is that PA doesn't really model audio as a graph of nodes. (PA's model is equivalent to a graph of nodes, but that it isn't just a graph of nodes makes it a lot harder to deal with and intuit.) For that reason I'm excited about Pipewire; I need to try it one of these days. PA would also be served by a better API; it is difficult to script against.
Also, systemd units are hands-down a more solidly engineered thing than the mess of shell scripts they replaced. I will take systemd+journald any. day. of. the. week. People that … want shell back — I honestly can't comprehend this. I've literally not had to deal for years now with a service wedging itself in because some daemon orphaned itself somehow between some unreliable mess of shell, pidfiles, and other hacks. Is systemd/journald perfect? No, I've spent time fighting it too. But it's a lot more productive than what came before, and the design is engineered to solve the problem at hand… which the shell mess was not.
For me Pulse was brutally broken for the longest time. Bit by bit most of the various glitches did get worked out, but the "pain points" of that process did leave me a bit bitter over how Pulse was really kinda "forced" on us long before it was "ready for prime time". Same with systemd, and same with the KDE3⇒KDE4 transition. All pushed too hard for widespread adoption too early in their development, while many of the worst bugs were still commonplace occurrences in daily usage.
Ever used a soundcard with more than two inputs and more than two outputs? Or better: that soundcard while also outputing audio via HDMI?
PA is ok if you use consumer software for consumer tasks. As soon as you go into the production side of things (or even do it live and/or for money) things can fall appart pretty quickly.
The vocal PA critics I know don't even complain about the way pulseaudio does things. They complain about the persistent problems this widely deployed piece of software still has after years in the field. And some of those problems are of conceptual nature.
I don't Pöttering really deserves the heat he gets (no dev would), but I sometimes wish that type of (usually male) programmer would just sometimes stay with the trouble and not ride of into the sunshine when things aren't a beautiful fresh sheet of white paper anymore, but a tangled mess.
Yes to both. My laptop has HDMI & TRS out. AFAIK, the soundcard is not capable of emitting to both HDMI & the TRS port simultaneously, but it's also not something I've ever wanted to do, so I'll admit that the limit of my use case there.
(I'll use a bluetooth headphone, if I want audio while pushing something to HDMI. Though I could see using TRS headphones being a nice to have, and I honestly don't know how to configure that, or if it is even possible, since it's all represented as one card in PA.)
While I'm no fan of Pöttering, I can certainly understand wanting to maybe "ride off into the sunshine" when the vast majority of humanity these days is more willing to tear down their fellow humans than they are to suggest or help with any sort of actual solution to any given problem. Makes even trying feel more than a little pointless.
> I'm still baffled by how something as basic as audio output can be eternally fragile in linux
Yeah, broken drivers.
You see, before pulseaudio came along, audio drivers were barely used and needed to be tuned for each use case. Most drivers barely implemented ALSA, almost none correctly. If you wanted fancy features (say; software input mixing, so that more than just one application can actually output sound) you needed to invest a lot of time.
By making all these fancy features (and more) more easily available, pulseaudio exposed all these bugs in the drivers.
And of course it got flak for that because the first software layer the user is confronted with is blamed if something's not working.
But actually - just as systemd - pulseaudio is a marvelous piece of code. That today we can get to an even higher level with PipeWire is to large degrees only possible because of pulseaudio.
Lennart Poettering is one of the most creative and influential system-space developers that Linux has. It's a shame that he's so maltreated by the peanut gallery.
Enabling software audio mixing with ALSA required copying and pasting a small handful of lines into your ALSA config file and just worked. Most of PulseAudio's hardware problems were caused by it having weird and unnecessary expectations, like wanting to know the exact sample being played by the hardware or expecting analog volume controls to have perfectly accurate gain and glitch-free volume changes.
PulseAudio also had pretty awful code quality in general. For example, I ran into a crash where the resampling code was trampling past the end of the buffer and it turned out that every stage of the audio conversion process - resampling, channel conversion, and so on - made assumptions about where it was in the process and what sample rate, buffer size, channel count etc it should have on its input and output based on members of a shared struct that were used by multiple stages in different ways. At some point the developers had accepted a patch which changed this order without fixing half the resamplers - and which said it was incomplete in the commit message - and this had somehow made it into the release I was using. One of the unfixed resamplers was the one used for VU meters in mixer apps, because that was implemented as a resampler for some reason. Also, all of this code had basically zero comments. Oh, and reverting the patch didn't even fix the crashes for me so presumably there was some other problem somewhere. Oh, and on top of that there were no stable bugfix-only releases of PulseAudio, so even if it did get fixed upstream it'd come with a bunch of new non-bugfix changes with the same level of quality and scrutiny.
> expecting analog volume controls to have perfectly accurate gain and glitch-free volume changes.
Why would it be weird or unnecessary to expect that?
Besides; that's a nice anecdote! Thanks for sharing!
The irony is that during that same time, FreeBSD somehow offered sound working out of the box, complete with input mixing, and without anything like ALSA.
This is factually false in my case since ALSA alone has always been fine, no issues. And since I switched (with the same hardware) pipewire also works just fine. But pulseaudio (pa+alsa ofc) is just simply broken on my archlinux install.
EDIT: also when pulseaudio breaks you just do `pulseaudio --kill ; pulseaudio --start` Does this really reset the hardware/firmware? I doubt it; it's just configured poorly for my hardware why is that hard to believe?
> Does this really reset the hardware/firmware?
No, but it can release resources and that in turn can cause the driver to reset in part.
It is probably the driver that leads to have pulse audio use 10% of the CPU of a quadcore when doing nothing / not playing any media?
Also, probably the driver that regularly confuse it's inputs and outputs settings...
Driver issues, right.. is that why my archived shell histories have hundreds if not thousands of entries for `pulseaudio --kill ; puslseaudio --start` ? Does restarting the userland sound daemon reset the drivers operating in kernel space? How would that work?
Driver issues can sometimes manifest as kernel exposed API interfaces blocking/using too much CPU in stupid ways/freezing when they should not; killing the "offending" userland program using those APIs will often end up releasing the resources acquired by invoking them.
Linux might be a monolithic kernel, but it's not like the entire kernel with all it's features/modules need to be loaded all at once.
Have never used those APIs, but I would assume it'd happen if drivers fail to implement basic API promises and PulseAudio doesn't code defensively against those kind of bugs.
If drivers fail that disastrously to implement the APIs they say they implement, how did they end up in distributors' kernel trees?
They were deemed "good enough" when all you were using was basic, simple OSS. Not when you're attempting to use all the capabilities of ALSA.
> PulseAudio doesn't code defensively against those kind of bugs.
That's a reoccurring pattern across Lennart software. He gets an idea of the way things should work, maybe from the spec or maybe just from his own subjective biases, and has his software expect things to work that way and fail when it doesn't. Every time, he'll say it's the other guys fault. Sometimes his software even fails unsafe and he still blames everybody else.
The '0day' username bug in systemd exemplifies this. Lennart decided that usernames beginning with numbers weren't valid, even though Linux permits it. Systemd was found to run service files with User=0day specified as root instead, not as the user 0day. To Lennart this was not a bug because in his subjective opinion the rest of the system was in error for permitting a username beginning with a number. Even if he were right that usernames beginning with numbers shouldn't be valid, the mere fact that they can occur is reason enough to code defensively around it. At the very least systemd could fail safe when it encounters such an 'invalid' username. But no, he thought there was nothing wrong with failing-to-root. It was everybody else's fault, of course.
https://lwn.net/Articles/727490/
https://github.com/systemd/systemd/issues/6237#issuecomment-...
Ubuntu's adoption of it was what pushed me away from Desktop Linux, after about a decade of being mainly a Linux user. Suddenly the one thing that had actually Just Worked for almost that entire decade, became unstable, unpredictable, and, to add insult to injury, ate tons of memory and processor cycles for unclear and likely unjustifiable reasons.
I’m sorry but I really can’t take any such take seriously. If you mean alsa, it may have worked (though let’s be honest, there was a painstakingly high amount of sound hardware that simply wasn’t supported well for decades, which was fixed over a similar timeline. “Linux sound bad” memes have been around way before PA became a thing), but the thing is, it is not adequate in itself for a typical desktop usecase, simple as that.
PipeWire has been increasingly beneficial to my usage of audio on Linux as time passes as well. Each update seems to fade it a little more into the background so I simply need not worry about it. Audio just does the expected thing for me these days, without fiddling or "hackery" required to make it so. It's also nice no longer having to worry about whether a particular bit of audio software is for Pulse, ALSA, or Jack. They all just work.
The only way Pulseaudio is palatable is by using the Pulseaudio sink in Jack.
I settled on alsa and dmix as the ultimate solutuon.
That works as long as you are not too concerned about latency, but it is a bit easier to set up than Jack. Though Ubuntu Studio takes most of the sting out of it.
I think that Redhat gave him a privileged position to make that impact from, and I'm not sure that he'll make a similar impact without similar political backing, but I suppose time will tell.
It's a fact of life that people get old and thus become less effective at making great changes. I cannot think of many counter-examples from our field.
Sickboy's Philosophy of Life. We all get old and cannae hack it anymore.
Eh, Pulseaudio. One of those things that on the rare occasion it's worked out of the box it's then broken with one step of the beaten path.
Evidently it seems to work for many people, the trouble it's caused has far outweighed any benefit (plus I know quite a few people dropped firefox for chrome when it went pulseaudio only - is that still the case?).
I don't know what kind of audio configurations you've lived with.
I do remember when PulseAudio seemed to be the source of many problems, but this was over a decade ago.
One of the big problems is that the Linux audio stack is built out of components that rely on assumptions about the next lower level of the stack that are often untrue. You might assume that your audio hardware has a volume control, assume it can perform mixing, assume it doesn't change configuration, assume there's only one user, assume (or want to assume) it has a specific sample rate, etc. PulseAudio papers over this stuff and deals with permissions, so your applications can actually work.
If you have sufficiently boring PC hardware and sufficiently boring use cases, the audio will just kind of work anyway, without PulseAudio.
Linux audio has more than its fair share of problems in the stack, but in my experience, PulseAudio is more of a solver of problems and less of a source of problems.
Stock Asus motherboards with on-board (Realtek, IIRC) audio (I tried aftermarket cards, though mostly after giving up on PulseAudio).
Separately the company I worked for ditched it as well due to the issues, to be fair though in tnat case since they only needed a single - reliable - audio output so there was no case of PA anyway.
Yea that is because anything Realtek is an utterly brain-dead piece of crap and "quirks", and that on Windows. On Linux it's even more pain, mostly because OEMs don't bother to release the specific quirks their specialized Windows driver fork had applied. Don't even want to know how many Windows systems ship with outdated, possibly insecure Realtek drivers because the OEM long since stopped backporting fixes.
I appreciate that Realtek can be an issue, but on a few PCs they works solidly with Alsa so seems unlikely it's the kernel driver (restarting pulseaudio would bring it back, at least for a while). So it seems to me the issues are very much with Pulseaudio (rather than with the kernel).
Alternatively, Pulseaudio could be exercising different code paths in Realtek's firmware that Alsa doesn't trigger. With Realtek, all bets are open...
> If you have sufficiently boring PC hardware and sufficiently boring use cases, the audio will just kind of work anyway, without PulseAudio.
Introducing breakage, complexity and general fragility to the common case, which was previously working fine, will obviously make a whole lot of people upset.
Audio in Linux before PA certainly wasn't working fine. Turning off your mp3 player and restarting your browser, because you want to play youtube via flash plugin? Sorry, but that's not fine.
I had my share of hardware, and PA had problem with only one piece (a nettop with Nvidia Ion2, audio via hdmi). It turned out that the hardware couldn't really play 44,1 kHz audio, it's clock was too unreliable and resulted in drop outs. PA couldn't fix the hardware, but could resample everything to 48 kHz on CPU, and with that there was no playback problem.
So yes, it exposed problems, and it forced to fix them. We are all better off today for that.
By far the best sound experience I've ever had on *nix was FreeBSD's fork of OSS. Dead simple API.. the /dev/dsp device had software multiplexing so multiple apps was no problem.
Alsa has it via dmix although not as the defaut. Probably because Lennaert.
Computer users are continually upset about a lot of things. That's just the way it is. If you work in the industry then that should make you happy, it means you will always be employed.
Linux audio was never working fine. It was a mess of broken drivers and incompatible implementations.
I cannot fathom how people are able to repress the memory of how bad Linux audio was before pulseaudio.
I never experienced any trouble with audio in linux before pulseaudio. There is no memory repression here; only gaslighting from you, telling me I experienced something I didn't.
> only gaslighting from you, telling me I experienced something I didn't.
Oh dear.. I'm not going to comment on that. I don't think it's productive.
> I never experienced any trouble with audio
I refer to stuff alsa drivers reporting wrong values[0] or cards being locked to a specific rate [1]
Before pulseaudio only one application could access the sound card. And alsa-mixer was broken half of the time. The list of defects goes on and on. Configuration relied on different settings of also that needed to be tuned for each driver separately. Getting sound to work on Linux was not a fun place to be.[2]
Pulseaudio exposed all these problems using a unified interface and forced "us" to fix these bugs.
[0] https://bugs.launchpad.net/ubuntu/+source/linux/+bug/410887
[1] https://lore.kernel.org/all/1034325263.474.37.camel@diesel/T...
[2] https://www.techrepublic.com/article/configuring-linux-sound...
> Before pulseaudio only one application could access the sound card.
That's only true if the sound card has no hardware mixing support (or very limited number of streams, where it would suddenly break after a certain number of applications) _and_ the alsa setup didn't include dmix (software mixing).
Not sure what problems you had with alsamixer, for me it always worked (but maybe I never tried anything sufficiently fancy for it to break).
But I am with the previous poster on this. I have had way more problems with pulseaudio than without. Also it's not only alsa drivers that can be at fault here, but pulseaudio also consults other hardware databases which alsa does not use, which can contain mistakes that can make pulseaudio unusable (had that issue not even a month ago with an external usb sound card).
> Oh dear.. I'm not going to comment on that. I don't think it's productive.
It's simple: Don't tell other people what they did or didn't experience. If you don't understand that, then you aren't worth talking to.
Have you by chance checked the bug tracker, or reported this bug, or attempted to do any debugging to find the root cause?
"Why didn't you just become a pulseaudio dev if you hate pulseaudio so much"
Get fucking real, I have better things to do with my life than waste time ingratiating myself with the likes of pusleaudio devs. "Audio stopped working" describes hundreds if not thousands of pulseaudio bugs, do you really expect frustrated end users to drop everything else going on in their life and spend weeks learning the ropes of this shit? Get real.
>do you really expect frustrated end users to drop everything else going on in their life and spend weeks learning the ropes of this shit
Are you sure you're responding to the correct post? I didn't say anything remotely like this. I asked if you would report the bug or try to do a little debugging, like spending a few minutes up to an hour. You don't have to learn any ropes, the internals should be very familiar to anyone who knows C and has some knowledge of audio. If this is important to your job, the company should be willing to let you do this on paid time. This stuff is open source, you either have a support vendor you pay to fix these bugs, or your company assigns developers to fix them yourself. There is no other way it happens. It's not a waste of time if you actually fix the bug.
And just to stress this, there is no other way the bugs get fixed. This is it. This is the whole thing. Some person somewhere does actually need to drop everything and go and start spending time fixing the bugs. No, it doesn't have to be for weeks. If you think a thousand bugs is a lot, well, yeah, it is. That's why you need to go take some initiative to give some information, to make your bug known and make it possible for anyone else to help. Otherwise, it blends in with all the other low-information bugs and it falls back on you to go fix it yourself. A bug report that just says "it doesn't work" with no additional information is not actionable, you need to provide more than that for anyone to be able to help you. If you become hostile after being asked to spend a few minutes reporting bugs, then it is definitely going to be impossible for anyone to help you.
That would required the "bug" to exist in the first place.
What kind of breakage, complexity, and general fragility are you seeing with PulseAudio? Can you elaborate?
One big problem here is that the "good path" for audio without PulseAudio applies to a shrinking segment of Linux user base.
Sound stops and doesn't work until I restart pulseaudio. You know, the usual. I think some people restart their whole computer and blame their drivers, but restarting pulseaudio alone is what fixes it nine times out of ten.
Interesting. I'd still rather have that problem than the massive, massive problems I tend to have without PulseAudio. Only one app can make sound at a time, can't adjust volume at all, that sort of thing.
Pulseaudio is one of the absolute best parts of Linux.
Have fun routing audio from one stream to one device and another stream to a different device on Windows or macos!
Jack is better. PipeWire, much better. Routing audio in Pulseaudio is confusing, at best, and _not fun_ most of the time.
On Mac, at least, this is easy with tools from Rouge Amoeba. Audio Hijack and Loopback being the main tools I've used to do this.
> Have fun routing audio from one stream to one device and another stream to a different device on Windows or macos!
I can do that with pulseaudio, but it's not fun.
I use to do that quite often with Pulse, and on Plasma it's just a matter of drag'n'drop in audio systray applet.
Okay, I have two computers on my LAN right now, one running OpenSUSE with KDE and Pulseaudio, the other with Debian and Pulseaudio. I want to send my audio from the OpenSUSE computer to the Debian computer. What do I drag, and to where? There's nothing dragable in my systray that even mentions the Debian computer.
I'm quite sure this process starts with me writing some config files on both of these computers. That's how it worked every other time I've tried these sort of things with Pulseaudio. These sort of tricks are possible for techies to do with Pulseaudio, but for the proverbial grandmother user they may as well not exist because the system as a whole certainly isn't going to make it easy.
> I'm quite sure this process starts with me writing some config files on both of these computers.
So don't be so sure ;)
You just tick a checkbox in paprefs to enable networking (which is perfectly reasonable, as in most cases you don't want to have it enabled by default), and that's it. It finds the other PC via zeroconf and it shows up as a regular output device.
Depending on distro you use, you may need to install PulseAudio's zeroconf module and make sure Avahi is running, but a properly configured desktop distro will have all that either by default or a single package installation away these days.
I'm not sure about SystemD, but PulseAudio is definitely a piece of software I wouldn't brag about. If you work on an existing codebase and do what your employer says, that's one thing, but when you have the rare freedom to design a system from the ground up, I expect it to do at least one thing: to run.
>He can easily hop ship wherever he wants given the major impact he's landed
I don't know about that. Talent alone doesn't make someone a good fit for a team. Would you want to work alongside Poettering? Or de Raadt?
I wouldn't, and that means I wouldn't hire them either.
> His design decisions have certainly been controversial but his impact is certainly agreed upon.
I can think of many historical figures that fit that description.
Considering the... magnitude of his contributions I'm surprised that there isn't some kind of press release floating around somewhere.
Maybe he'll go on to become an Executive Vice President at Microsoft.
I was thinking about Rick Belluzzo's time at SGI when I read your comment.... https://en.m.wikipedia.org/wiki/Richard_Belluzzo
cough Miguel de Icaza cough
Miguel criticized MS recently so this path might be closed for him.
Miguel de Icaza left Microsoft in March, 2022.
Ok, so they hired Lennaert as replacement, finally!
not so surprised pikachu face https://www.phoronix.com/scan.php?page=news_item&px=Systemd-...
Good job Lennart destroying Linux from inside.
This is an interesting development, to say the least. If they give him enough power, he can create havoc in Microsoft. Could be fun to watch.
Called it!
Actually, according to TFA, he's already left RedHat some time ago.
Does anyone know if Poettering has discovered NixOS yet? I'm reminded of it when I read some of his blog posts like this one [1]. It's hard to describe why, but I feel like he might be able to do a lot with what NixOS offers.
He’s definitely aware of it. I’ve seen him comment on issues related to NixOs before.
Most recently I think I saw him comment on an issue around LoadCredential not being available in ExecPreStart
Interested to see what he chooses to work on now
Maybe he's turning to the graphic stack (eg a grand unified graphics driver architecture), or redoes large parts of the kernel in Rust ;) Or something like that; I've got a hard time thinking what could be challenging enough seeing as, though controversial his creations are, he sure is able to deliver.
At least we will know to steer clear of it until he stops winning Pwnies.
Judging from the email in the article, systemd is still his main priority (which makes sense). With system-homed/etc he's been trying out some pretty interesting new ideas, so I wonder if RedHat has just been a too conservative for him now that systemd's off the ground and embed into distros.
There was always a mixed ecosystem. At one point, we were using Chef, Ansible, Puppet, AND Salt both internally and in shipping projects.
systemd-homed, systemd-nspawn as isolation, and the rest of his ideas aren't being held back because they're controversial, it's because OStree/Atomic "lost" more or less the instant Red Hat purchased CoreOS and they didn't need to build their own de-novo k8s compute distro (parts of OStree lived on in CoreOS and are there as a testbed for immutable images for trusted computing in kiosks/appliances/VDI).
systemd-homed, using systemd-nspawn for management, and the rest go directly against the investments which have been made in [fedora-]toolbox, rootless podman, and flatpak. At the time that I left (~2 years ago), "Openshift is the new platform" was the phrase of the day, which means podman "won".
Red Hat/IBM (via the container toolkit, rootless podman, etc) is/was primarily invested in a growth sector, which desktop/server Linux is/was not, and the Container Development Kit, rootless podman, and the rest are direct mechanisms to get containers sold on k8s and openshift.
Lennart's passion projects are great for Linux, and maybe better ideas than Flatpak and Snap. They don't have a business case inside Redhat, though, and there's only so long that even a principal consulting engineer can essentially spend his time on research projects.
I would guess that he wanted to put more effort into an overhaul of CoreOS/OSTree and try to "unify" it with flatpak somehow to get back to "one Linux" instead of split distros for different cases, didn't get traction, and decided to go somewhere he could work on a passion project.
Ah, thanks for the context! > systemd-homed, using systemd-nspawn for management, and the rest go directly against the investments which have been made in [fedora-]toolbox, rootless podman, and flatpak. At the time that I left (~2 years ago), "Openshift is the new platform" was the phrase of the day, which means podman "won
This is definitely closer to what I meant then what I said haha. I've noticed for a while now that RedHat hadn't been picking up a lot of the newer features that systemd-* has been adding, though saying they're "too conservative" for it was probably a bad word choice.
I personally think Lennart is a cool dude. I wonder where can a person like he go next, I hope it's not Microsoft
> Systemd Creator Lands At Microsoft
https://www.phoronix.com/scan.php?page=news_item&px=Systemd-...
Was happy seeing him comment on a systemd bug I opened a while ago. Should answer as well...
Is he retiring? Is it time for champagne?
That depends very much on where he lands. Rumors have it that Linus is considering transferring kernel maintainer duties so let's not put the flag out just yet.
Ok, just kidding ;)
I reckon he’s a lot less employable than he thinks he is.
Even assuming a non-controversial person, it's very hard to find a corporation willing to employ a dedicated, fully commited to upstream, foss, Linux Desktop experience developer.
I can’t imagine how to face an interview panel knowing you’ve pissed at least one of them off already before you even arrived.