Ask HN: Best Alternative to Homebrew in 2021?
Yesterday I ran `brew install virtualenv` and homebrew decided to upgrade postgres, ruby, youtube-dl, vim, rust, wget and 50 other unrelated packages. The single command took 45min and broke all of my local postgres databases (and who knows what else).
At this point I should know better than to install postgres via homebrew, but unwanted/unasked-for upgrades is not a feature I'm looking for in a package manager.
What's your experience with other package managers for macOS? Currently looking into macports & nix and not sure which one to choose... Homebrew project leader here: I hope you're able to find a package manager that better fits your needs and I'm sorry that Homebrew is not currently doing so. --- Homebrew upgrades dependencies and dependents of those dependencies (which, admittedly, can feel like unrelated) on installation and upgrade. As mentioned in other comments, you can customise this behaviour with `HOMEBREW_NO_INSTALL_UPGRADE` or `HOMEBREW_NO_AUTO_UPDATE`. Homebrew does this because the alternative is sometimes breaking things. An example: - you want to install `virtualenv` that depends on `python@3.10` - the binary package for `virtualenv` you want requires the newest `python@3.10` - this upgrades `python@3.10` on installation Now, Homebrew has a choice. Either we upgrade _everything_ that depends on `python@3.10` that you have installed or we knowingly break some of the things you have installed that depend on `python@3.10`. We choose the safer option by default. The more time left between updating/upgrading, the more likely to have more dependencies updated which requires more dependents to be updated. --- Regardless, I appreciate this is a problem and we're still figuring out potential solutions for this problem. A reminder that we're a volunteer run project so it's not always as easy as we'd like it to be to get these changes out quickly. Mike - just wanted to say thanks for all you (and the other Homebrew volunteers) do! My mac dev experience is 100x better with Homebrew - it's literally the first thing I install when I get a new machine. Perhaps in situations like this Homebrew could alert the user "Hey, this package hasn't been updated in a while and 40 other packages depend on it so this is going to take a while. Press enter to confirm or cancel and use an alternative method (HOMEBREW_NO_INSTALL_UPGRADE) to just upgrade the one package. Caveat - this may break things." +1 I love homebrew. I just set up my new M1 Max machine and half of the setup work was just brew commands to install cli stuff and casks <3. It's such a joy to not need to go to a bazillion websites to download dmgs what a flex on your new M1 MAX ;) But yeah, homebrew should ship as default with any OS X at this point... did brew leaves from your old machine not hey you a transferrable list of packages? ‘brew bundle dump’ does an even nicer job of this. Check it out! Every time I setup OS X I like to go through my osxsetup.sh bit by bit to see if there’s anything I didn’t use, change or update. Takes a little longer but it’s nice to take that time every couple years I go through it It's not as simple as you think: consider headless setups. Generally programs can figure out whether they should run interactively or not. OP here, thanks for clarifying this - I can imagine it's a super complex problem and I truly appreciate all the work that you (and other contributors) have put into this project. My only point (and a source of confusion) here is that I never anticipated the cascading effect that installing a single package can have on seemingly unrelated parts of the system. I've been using homebrew for many years and as far as I remember, the only time anything broke like this was when I explicitly ran update/upgrade. I'm going to try HOMEBREW_NO_INSTALL_UPGRADE/HOMEBREW_NO_AUTO_UPDATE — it does seem like a more intuitive default, but again, I don't know anything about how homebrew works under the hood. Thanks again! There's more than one way to skin a cat. Personally, I have moved to downloading large things (Python, Postgres by your example) as standalone apps, and only use Homebrew for small libraries and CLI tools. I think that makes more sense. You can download all of the Mac Python versions you want from python.org and you can get Postgres from postgresapp.com which provides a nice toolbar interface to start and shutdown servers of various versions. On Debian-based systems, I get a preview of what's going to be installed/updated when I do something with `apt`. With homebrew, I never know how many packages will be updated and how long it will take, so sometimes when I just need a small utility, suddenly it'll update a bunch of packages and it takes 45 minutes. It's a UI problem. An apt-style 'hey here's what you going to do, confirm/deny' would be _great_ in homebrew Not sure how to add that prompt by default without breaking existing scripts. Making it non-default seems like it wouldn’t help (given comments here). The prompt should appear only in interactive mode, so it shouldn't break existing scripts Fair enough. Like the sibling says, checking the moral equivalent of [ -t 0 ] probably distinguishes enough interactive vs non-interactive use to add an interactive-only default. I'd add that script-y usage of brew already can block on user input (e.g. if a cask asks for sudo), so adding a go/no-go check probably only breaks the most determined automators, while the benefits accrue to a wide swath of users (both new and fat-fingered). Regardless, thanks for all the work you (and the community!) do on Homebrew! I wouldn't like macOS half as much (or have bought half as many) without it. Yeh, that’s the interactive check we’ve used in the past and unfortunately it still has given false positives and false negatives enough I’m hesitant to make a change this big due to it. Did you use brew outdated after performing brew update? brew outdated will list outdated app/dependencies that need to be updated. If you want to know the app's dependencies, use the info argument. For example brew info ffmpeg will list the information about FFmpeg and list the dependencies that it uses along with status indication if the dependencies is installed or not. Am I missing something here? I am not sure if it is a UI problem since Homebrew have those functions with a few arguments here and there. Look like you are using a utility that have so much dependencies that it needs or you have habit of using brew upgrade which will prompt to upgrade everything expect pinned apps/dependencies. > Am I missing something here? Yes. The principle of least astonishment. I appreciate all your hard work and I like Homebrew, but I think updating high-level packages when a descendent dependency is updated is the wrong behavior - at least in this example. If Postgres has an unversioned dependency to PackageX, the only time I might expect Postgres to be updated automatically is if I update PackageX across a major version boundary, risking breaking changes. Even then, I'm not sure I'd want that behavior by default - I'd at least like to be prompted - since often this still won't break anything, or it will likely cause less damage than updating every package with a dependency to PackageX. And if Postgres breaks after I update PackageX I can easily reason about what just happened. The alternative can be incredibly hard to troubleshoot because so many changes have occurred. Overall, Homebrew feels far too aggressive here, and I think the result, for people who have some understanding of Homebrew behavior, is that they're afraid to touch anything. > If Postgres has an unversioned dependency to PackageX, the only time I might expect Postgres to be updated automatically is if I update PackageX across a major version boundary, risking breaking changes. Homebrew doesn't meaningfully have versioned or unversioned dependencies, because it doesn't incorporate any notion of version into its dependency solver. When you declare a dependency on `python@3.9`, you're not telling `brew` that ‘I need a copy of the package `python` which satisfies the constraint “version is 3.9.x”.’ Homebrew just treats `python@3.9` as the name of a package which is a whole copy of python, and it can exist alongside other, similar packages like `python@3.8` and `python@3.10`. Homebrew only incorporates this kind of `@` syntax for select packages of which maintainers have elected to maintain multiple versions. By my count, there are 81 such packages in homebrew-core, and the following is a complete list of all of those which appear in a `depends_on` declaration: This is a feature it shares with some source-based package management systems, like Nix and Guix. Those package managers solve this problem by basically vendorizing everything, so installing a new package never upgrades anything else you have installed. Other source-based package management systems, like Portage and Pkgsrc, have extended their model to include some handling of version constraints, but they also sometimes declare dependencies without specifying a version. That seems dubious to me, since in reality there pretty much always are version constraints that ought to be specified there, but maybe those two projects are mature enough that they've hammered out the problematic cases by now. — https://github.com/Homebrew/homebrew-core/blob/master/Formul... Thank you for all your work. Homebrew is one of the most useful tools in macos development ecosystem, at least for me. You're doing an awesome job. That said, I did notice recently the same effects as the OP. I.e. I try to install something small and it depends on something like python, which triggers "upgrade the whole universe" loop, which breaks a ton of things (e.g. python upgrade doesn't carry over installed modules, but does change current version - which breaks many local python scripts since their dependency modules are now in the old version). Auto-cleanup tends to break a lot of things too - especially bad for commonly-used things like openssl or ICU which commonly are linked as shared libs. I think there are settings for changing all of this, but you only find out once you've already been bit. Maybe it's worth considering making it somehow explicit - e.g. ask the user "do you really want to upgrade the whole universe" before going in? In many cases, I'm completely fine with using python 3.7 even if python 3.10 is already out. And most of the tools that depend on python would be fine with it too. It would be nicer if low-friction mode would be the default. I didn't know before this post that Homebrew's infamous ‘auto-update’ only automatically updated the dependency closure of whatever you were installing/upgrading, which sounds more or less correct to me. > Now, Homebrew has a choice. Either we upgrade _everything_ that depends on `python@3.10` that you have installed or we knowingly break some of the things you have installed that depend on `python@3.10`. We choose the safer option by default. A look at the formulae in homebrew-core shows that Homebrew basically doesn't consider versions in its dependency management at all, so formula are just using the ‘@x.y.z’ in place of all manner of version constraints. So you get wild stuff like this line here[1]: Of course the recursive update policy is going to be weirdly painful for some users! Homebrew doesn't even attempt to encode some very basic aspects of dependency, so now (a) you can't know what you're going to break if you do upgrade a common dependency and (b) you can't know when you can afford not to upgrade a dependency shared by a package you're about to install and the already-installed system. The choice you outline above is one that is not faced by most package managers, because they don't make this mistake. The naive wheel reinvention with Homebrew is so eternally disappointing, and it explains a lot of the pain users experience with it. > Homebrew upgrades dependencies and dependents of those dependencies (which, admittedly, can feel like unrelated) One relatively non-disruptive thing you might be able to do to make this behavior less surprising to users is (offer a way to?) print the dependency tree for package installations/upgrades that pull in upgrades of their ‘siblings’. You'd probably want to just do it in a textual way, but this project seems to model the kind of logic you'd want for printing dependency trees with Homebrew as it exists.[2] — 1: https://github.com/Homebrew/homebrew-core/blob/master/Formul... 2: https://github.com/martido/homebrew-graph/blob/master/cmd/br... That “wat” is because we use the system ‘curl’ when we can here. The comment is stating why we can’t in this case. Not ideal, I’ll admit. First, wanted to say thanks for heading a wonderful project. I personally love Homebrew! What’s the difference in what Homebrew is doing from what normally happens with something like Pacman or Apt? My impression is that “core” binaries, like Python might be, don’t get updated as often as they would on Homebrew, and perhaps those dependency chains are less frequent. Is it because (e.g.) python versions change rarely on linux package managers? Or something about how the binaries are compiled? `brew` lives across (at least) one fundamental divide from `apt`. Homebrew is a source-based package manager, where packages are built from a single shared source of build recipes, and distribution of software happens by distributing the whole collection of recipes. Support for a binary cache is added through the identification of build recipes with the hashes of archives of binaries. APT is what's called a binary package management system. While build recipes can be stored in a central repository, they don't have to be, as binary package management systems typically identify two kinds of packages: source packages and binary packages, which can be distributed separately. (Some repositories may not even include source packages.) Source packages contain metadata like build recipes, but things like archives of the upstream source code, patches, and sometimes helper scripts that might be usd in building. Binary packages include the actual built artifacts; they're installable packages. In the source-based package management world, if packages can be added to the main collection through external sources, this is typically done by pretending that those packages were embedded in (some particular version of) the main repository of build recipes. Collections of packages that are ‘injected’ into the main package collection like this are typically called ‘overlays’. In binary package management systems, an external package is not expected to have access to some copy of a complete collection of build recipes from some ‘master’ repository. Instead, source packages are distributed individually using the same mechanisms as binary packages. This means that the global namespace of terms which refer to packages can't be navigated implicitly at build time like on source-based systems, the identification of actual source code and build instructions with some package name happens both at build time and at install time. And in both cases, explicit metadata (package names, package versions, equivalency classes, lists of conflicting packages, etc.) is used to determine what versions of dependencies get pulled in. As a result, binary package management systems always include dependency solving at install time, and usually include some explicit version constraint for every single dependency declared in a package's build recipe. This means they have the resources to look at what's going to be installed, what's already installed, and say ‘oh, the new installee requires Python <= 3.10.4, and since your installed packages together only need Python >= 3.10.2 and you already have python-3.10.3 installed, we don't have to touch Python to install this new package unless you ask us to’. Pacman is somewhat atypical. The build system is essentially source-based and monolithic (there's no source package format) and the AUR is essentially an overlay, but the high-level CLI tool acts like a binary packager and uses the language of repositories to describe dealing with multiple package sources. Pacman's dependency resolution behavior is also unusually unreliable, and probably closer to what `brew` does than what `apt` does. Pacman has the resources to describe and test version constraints, both in its build system and in its CLI tools, but they're underutilized in a way that's more typical of source-based package management systems than binary ones. Homebrew is similar in that such constraints are radically underutilized (they're totally absent). The main difference between Homebrew's behavior and Pacman's in this instance is that by default, just `pacman -S` doesn't update pacman's collection of package definitions (pacman -Sy does) and Homebrew always tries to install from the latest build recipes. I imagine that Pacman's capacity for handling version constraints means that the story with `pacman` will usually be a bit better than with `brew`, but that otherwise `brew install` will be similar to `pacman -Sy` and `HOMEBREW_NO_INSTALL_UPGRADE=1 brew install` will be similar to `pacman -S`. (Pacman also sometimes makes the opposite tradeoff that Howell describes for Homebrew sometimes; it's not unheard of for installing new software or updating software using Pacman to break installed software.) yeah really unfair to blame a package manager for being unexpected when the context is installing/updating a python package. Don't care what package manager you use, python is always going to be a mess > python is always going to be a mess The irony is that poster was trying to install a solution to this very problem (virtualenv). The broader problem is that, at this point, python is effectively its own OS. Coupled with macOS being schizophrenic between "We want to provide a posix environment for users" and "No user should ever look under the hood, so no need to tell them about all the duct taped bits." Which brew attempts to bridge. Admirably, but never perfectly. Obligatory: https://xkcd.com/1987/ No, the problem is that Python's packaging system (like Homebrew's!) was created without adequate reference to prior art, has undergone much more evolution than design, and has suffered in every attempt to improve it from the fact of its adoption by a large userbase which democratically manages it so that changes must be conservative. As a result, Python packaging has an utterly insane design that it can't evolve its way out of. Problems that have proven more or less intractable for resolution by evolutionary change: This is why programmatically generating Python packages for general-purpose package managers like Homebrew ranges (depending on the resources made available by the package manager's design) ranges from impossible to stupidly complex and computationally expensive to kinda works but has no hope of reliability/correctness/completeness. > No, the problem is that Python's packaging system (like Homebrew's!) was created without adequate reference to prior art What's a good example of prior art that Python might have considered in its design? This whole discussion is super interesting to me and I'd like to read more on better examples of dependency management. > I'd like to read more on better examples of dependency management. I'll answer this before we do some Python history. Since there are a range of paradigms, if you really want a good sense of the main ways things are done and the tradeoffs involved in mature systems, take a look at: dnf/RPM as it's done on current Fedora, Gentoo Portage, Guix, Cargo from Rust, Flatpak, macOS' .apps, and Distri. Each of those represents a different approach. (You could get away with just looking at dnf and Portage, which also represent modern iterations of the paradigms Python could have looked to in the early days of its own software distribution tooling, if not the earliest days of the language itself.) SOME PYTHON HISTORY That's a great question, in part because Python has been around so long! It's easy to collapse the past in a way that's uncharitable to projects like Python, so let me expand on this a little and cross-reference some dates to get it into perspective for both of us. The best examples to consider today for dependency management are probably Linux distro package managers, *BSD ports systems, and Rust's `cargo`. But Python is older than Linux or any of the major surviving free Unices! So one legitimate difficulty Python has faced here is its age— what Python folks could have looked to for inspiration in 1991 was, compared to today, extremely limited and much less publicly available. They were not positioned, as the designers of Rust were, to really think through how code in the language would be distributed on day zero, before the language would see any real adoption. There have, however, been opportunities for the Python community to rethink things in light of the work and experience of projects (like Linux distros or ports systems) where dependency management is a very central problem. There have also been occasions on which some breakage was already expected or permitted, and such a change might have been feasible. But I think the focus has been elsewhere, and the Python community and its leadership have had to face the very real risk that too many breaking changes in their language's tooling could result in shrinking or dividing the community in a way that Rust hasn't yet. So the issue isn't some unkind nonsense like ‘Pythonistas are idiots’ or whatever. That said, by the time distutils rolled out as part of Python's standard library in 2000, apt, Debian's second-gen high-level package management tool was out and had undergone a couple years of battle testing. The *BSD ports systems had also been around for years by then, and Gentoo would hit the scene leveraging a similar approach to build a whole Linux distro that same year. I'm not sure what Linux package management tools were like, but that would certainly have been enough to tell already that static metadata for source packages was a must (and so something like setup.py is not appropriate). Similarly with the creation of setuptools in 2004— by then there were lots of examples of mature package managers in two major paradigms: source-based (like Homebrew), and binary (like most Linux distro package managers). Similarly again with the rollout of Python 3.0 in 2008. When pipenv came out in 2018, the answer to questions like ‘why does pipenv take so long to freeze dependencies into a lock file’ (‘it has to download every package and pretend to try to build them’) revealed that the Python packaging world had failed to meaningfully incorporate lessons on package management from the outside world in the intervening 10 years. You can watch/listen to a fairly non-technical overview of the history of Python packaging here: https://www.youtube.com/watch?v=AQsZsgJ30AE Homebrew was a much later arrival, so its case is a little more baffling. But some of its choices that now seem naive or questionable should also be credited with its widespread adoption. For example, the choice to depend thoroughly on the underlying macOS system instead of isolating dependencies makes Homebrew pretty brittle in the face of upgrades between macOS releases, but it also made it much more convenient for getting things installed quickly from a blank slate state than other source-based package managers (i.e., MacPorts) for macOS at the time, since it meant less compiling from source. Similarly, while the simplicity of Homebrew's build recipe DSL is a current source of problems, that same simplicity (paired with embedding it in a popular, interpreted language) has made Homebrew feel approachable and helped to draw in the huge number of contributors that have worked on Homebrew's formulae. Thanks for taking the time to write this out! I think of those sorts of things as paradoxical people requirements. To wit, only people interested in new languages spend lots of effort building new languages (and their associated ecosystems). People who have been using an existing language for years, long enough to internalize the design decisions and flaws, have little incentive to stop using something they're an expert at, to build something new. Ergo, the person you really want designing a language and ecosystem is someone who's used existing art enough to be an expert, then decides to give it all up, start from scratch, and do something harder. There aren't many of those people. Yep. This is definitely a factor. The insight that the technical is not entirely separable from the social is an important one, imo. The bigger irony is that Python has had its own venv built in since 3.3. I haven’t needed virtualenv for years. Can't you keep both python 3.10 and python 3.09, for older apps that rely on it? Sure, not optimal, but hard disk space is cheap these days .. Ideally, each app would list hashes of packages it depends on, so that everyone is happy. What about applications that just reference “python” (or any other dependency), without specifying a particular version? AFAIK homebrew cannot automatically modify $PATH before starting a program such that $PATH points to the correct dependencies for that program. If you have multiple versions of some dependency installed, it is up to you to run `brew switch` before running your program to manually set the correct versions for each dependency. I don't use homebrew too much (I'm usually on Linux, and when I am on macos, my development environment tends to be pretty lightweight), but based on my experience with Linux package managers, I wonder if this is mostly a pain point due to the performance issues brew sometimes has. On Linux, I tend check for upgrades fairly often (at least once a day, and always right before trying to install something), so I don't run into this issue much there. However, with my Linux package manager, checking for updates is always really fast (maybe a second or two at most unless I'm on very spotty wifi), whereas sometimes trying to do the same with brew takes significantly longer (and that's before even applying the updates). It seems plausible that users who are frustrated with slow updates start updating less often, which then causes updates to take even longer the next time they do, until eventually they have an experience like OP where they try to install one thing and it takes 45 minutes. I imagine there isn't an easy solution if this is in fact the core issue, but I think it's worth considering that the core issue isn't actually the update strategy you describe, as that's fairly similar to what I've seen with Linux package managers, and I can't remember ever seeing any complaints like OP's about any of them. this is why I love HN! Thanks for all the work you put in homebrew, I really appreciate it. Just upgraded to Monte Rey a few days ago and I was kind of fearful that some of the things that I needed where not going to work... nop, every singe package that I've needed so far was installed successfully. Thanks! Just wanted to echo the appreciation for everything you and other volunteers do on this project. Using a Brewfile has completely changed my setup process for a new machine, taking me from weeks of tinkering and going to find some tool I'd forgotten I use once every month or two down to a couple hours of downloading packages automatically. There may be a few rough edges here and there, but homebrew is one of the few truly essential tools for development on a Mac today IMHO. > HOMEBREW_NO_AUTO_UPDATE My life became much better after setting this. I wouldn't be averse to a HOMEBREW_AUTO_UPDATE_INTERVAL=180d either, or HOMEBREW_AUTO_UPDATE_ASK=y . Been using homebrew for many years, thanks. Although, I think the permissions model is broken. I wish Homebrew supported multiple accounts. On my macbook m1, all accounts point to /opt/homebrew and it is a giant ball of mess. Modifying the prefix to ~/homebrew leads to other packages that hard code the path such as pyenv. I think we need a new package manager since its not the fault of homebrew that many packages have hardcoded stuff in there. For the record I love Homebrew. I even use Homebrew to install regular applications whenever possible because I love how easy it is to update them. I appreciate the complexities of dependency management and yes, it sucked a bit when I installed postgresql. That isn't a problem with Homebrew though, that's just the nature of the beast. Anyway, I just wanted to say you guys are doing a great job! I've opened https://github.com/Homebrew/brew/pull/12413 to improve the discoverability and documentation for all this stuff. FWIW: Very happy Homebrew camper here, thanks a lot for all effort you’re putting in. At least for my tastes, Homebrew is a very nice and convenient package manager. homebrew is the BEST! I just reached the same breaking point -- and normally I am pretty sympathetic to software jank, but the maintainer of homebrew is so actively hostile and rude that my degree of empathy is significantly less -- and switched to Nix, which is quite good, conditional on three gotchas: 1. Nix has very limited support for GUI applications of the kind that you'd find in `brew cask`. Not no support, some of them are there, but limited support compared to brew. If you use brew as an out-of-the-box install-everything setup like me, you'll not be able to replicate that easily with Nix. 2. Nix can't currently build Swift applications because of some Xcode something I didn't fully understand. 3. I found a small number of packages that are broken on Nix on my aarch64 / M1 MacBook Air (and marked broken, so when you try to install it complains) and a small number that are broken without being marked broken. In specific, I think R was one that surprised me -- this is a major programming language and although the build requirements are onerous (it's a mix of C and Fortran), I would think it would work. That being said the actual process of installing, updating, and upgrading seems much faster and dramatically less shitty than homebrew, and so I recommend migrating stuff off of brew anyway. I find Mike McQuaid pretty reasonable and forthright whenever I have ever messaged him. Also, he is hardly responsible for every formula out there. For anyone who remembers the year 2000 and having to compile everything from scratch, Homebrew is brilliant - but there are complexities and quirks. For Python I’d go with pyenv. Postgres - well, watch your step and maybe use Docker instead. For PostgreSQL on macOS I use and suggest https://postgresapp.com/ which is free and able to manage multiple Postgres major versions. It's also self-contained in a folder in /Applications. Also, version 14 of Postgres in the app has m1 native support. For “infrastructure” apps like databases, caches, etc, I’ve been using Docker containers. Very easy to setup and tear down in a repeatable way. Unless I’m modifying the app itself, of course. For me it’s the only sane way to manage multiple small projects. A few lines in a docker compose file and I’m done, no matter the Postgres/Python/Ruby/Lib or framework version, everything is isolated and reproductible. Downside is gigabytes of RAM wasted. Exactly. I have a few build tools on my machine but any service installed locally has been long gone. Even the apps I work on - I just have the code locally and compile/package it up for testing within a container, which resembles the actual production landing spot. No more "it works on my machine". When I choose tools I like to ask "Who has the hardest version of this problem? What are they using?" I'm reaching the tipping point to adopt Nix, after seeing Nix as an alternative for supporting project after project. The unstated subtext always seems to be "Nix is what the smart kids are using, but it's harder to learn." Haskell? Same religion. But the joke is that one needs a PhD in category theory to use Haskell at all, and package management has long been a Haskell sore point, yet Nix hasn't taken over. The Lean Theorem Prover? https://leanprover.github.io/lean4/doc/make/nix.html A century ago, Hilbert's program to formalize and automate mathematics was stopped in its tracks by developments in logic such as Gödel's incompleteness theorems. There is a new push involving many leading mathematicians, using the Lean theorem prover. While some imagine computers replacing people, a better model is how a chess grandmaster checks their analysis using computers. Even as the experience of observing computer Go games leads to moments of amazement, as if observing an alien tournament, the role of a human to appreciate and generalize is preserved. Primates have long used tools; our future is to coevolve with machines. Dependency hell and Gödel's incompleteness are cut of the same fabric. This is where I first read about Nix flakes. I also tried Nix (as well as NixOS) but it's obviously difficult to use and has a very steep learning curve. But actually it might be the most flexible and stable package manager out there. So probably the problems mentioned can be fixed by custom configuration but that might require deep-diving into Nix and using Functional Programming for package configs... For some time I also used to just ./configure && make && make install (sometimes combined with stow) to my home dir, that worked also well but makes installing and upgrading tedious. That said, I'm sticking to Homebrew. Not really sure about this auto-upgrading, maybe it can be deactivated, on the other hand probably it's a good way to keep packages updated. I think the problem with Nix is that it is actually not a package manager but a build tool (that provides some package management utilities). People try Nix with the assumption that it should be as easy as apt-get (for example), but found tutorials with a block of code in a weird language... Sure, you will eventually need to use the Nix language if you want to have complex and reusable configurations, but this is actually not needed for beginners! To install packages, you need the Nix package manager (with channels already setup) and use `nix-env` to install for the current user. You can just search for the package in https://search.nixos.org/packages and use the command directly. If you want to have user configurations, use home-manager, just modify the example to add other options, you seldom need complicated Nix expressions unless you need to build a package by yourself. When you really need to do so, you may want to have a look at other's build script, and make sure you understand everything in the build process, including compile dependencies and runtime dependencies, environment variables, etc. You may want to read Nix Pills at some point, but maybe you can get away with reading just the relevant section (incomplete knowledge, but usually enough to get the work done). > I think the problem with Nix is that it is actually not a package manager but a build tool (that provides some package management utilities). People try Nix with the assumption that it should be as easy as apt-get (for example), but found tutorials with a block of code in a weird language... Indeed, I've been using Nix for many years (on Ubuntu, NixOS and macOS), and I now find the apt-get style 'considered harmful'. Instead, I define a single Nix derivation ("package") which provides everything I want in its 'bin/' directory, e.g. Just wondering, what is the benefit of doing this over nix-env for system-wide applications? For project specific environments, I usually use `shell.nix` (maybe I should switch to flakes now) together with direnv that runs nix-shell automatically when you enter the directory. For beginners to Nix, I think they should do the transition gradually, there are so many concepts in Nix that may be unfamiliar to them. I think getting a system to work quickly (maybe in a not so idiomatic way) is more encouraging than asking them to read Nix Pills and understand how packaging works in Nix :) `nix-env` is an imperative command, whilst writing a .nix file is declarative. In particular, the latter can be managed using git, e.g. here's mine: https://github.com/chriswarbo/dotfiles/blob/41c5c4643845f437... That's actually over-complicated, since I used to use nix-darwin and haven't bothered to un-pick it yet. You could use nix-env to install the file with `nix-env --install --file default.nix` rather than using nix-build + symlinking the result. What are the advantages of just nix-build, and adding the result/bin/ to path? Arguably, `nix-env --install` modifies the underlying profiles which manage symlinks. (nix-build + add to PATH is imperative invocation; but `nix-env --install` is and imperative invocation with a stateful effect). e.g. of "how could that be bad?". I'd managed to get myself into a confusing state with `nix-env --install`. When setting up NixOS, I'd installed firefox as the root user. Months later, in my normal user account, I was confused why such an old firefox was installed (since my profile didn't install firefox). I can't remember exactly, but this also somehow caused problems with fontconfig. -- Probably easily avoidable; but if you don't know what you were doing, you might also try all sorts of install commands and forget. `which -a` and `realpath` are your best friend when you're dealing with a multiuser package manager! Together, they make it easy to figure out if a package comes from your user profile, the default imperative profile, the system profile, or a transient shell profile I also use Nix, and agree with these caveats. My personal experience is that it can be hard to initially set up Nix, but once you do, it's rock solid. It's particularly great for sharing config across machines. Here's config for my personal dev environment for instance: https://github.com/mjhoy/dotfiles/blob/main/nix/nixpkgs/conf... I tried Nix while ago when NixOS made news in HN. You are right, it is difficult to set it up. Eventually I give up on it and tried to remove Nix from my macOS, let say it is difficult to remove Nix as well. I tried their command line to remove Nix and it didn't remove it even with sudo. Nix been sitting on my macOS for a month since it couldn't remove it completely. I realized I just have to go to Disk Utility to dismount/remove the Nix VHD and then ran the command line again which finally got it removed from my OS. Hm, that's a good point. Removing Nix is easier on operating systems where root is root (normal Unix-likes without a lot of filesystem-related security policies enabled). Nix only uses a separate APFS volume on macOS because Apple disabled writing to /. The way that affects the uninstall process should be documented. I spotted some people who've recently worked on Nix's install scripts and uninstall procedure elsewhere on the thread. You can file an issue if you think they'll miss your comment :) I've just gone through the process of configuring a new MBP with nix-darwin and home-manager, and I'm sold. Keeping my MacOS and my NixOS environments in sync has been a nice bonus. I still run Homebrew for a couple of services and casks, but I'm down to less than 20 packages still installed that way. There have been a few points of friction (which I'm used to as a polyglot with lots of active projects), but the community has been very helpful with ironing those out. You can also run nixos-rebuild targeting your NixOS machine from your Mac which is a neat trick - and it’s how I update my home server. Nix-Darwin is great! I wish the Nix installer on macOS were (a) always multi-user and (b) came with Nix-Darwin, so it could be a more NixOS-like experience out of the box. Regarding GUI applications: you're probably talking about the Nixpkgs repository, rather than the Nix tool itself. I use the following Nix function to install GUI applications https://github.com/cmacrae/.nixpkgs/blob/d4b51eb414b0edaffae... It doesn't work for everything, but I use it at the moment for Amethyst, DBeaver, DockerDesktop, Emacs, Firefox, iTerm2, Postman, Slack and VNCViewer. This is neat! I wonder if there would be value in having a repo for only GUI applications, equivalent to casks. I suppose it works best when the application doesn’t also auto-update, which many do. > I suppose it works best when the application doesn’t also auto-update, which many do. Yes, the update prompts get annoying (auto-updaters don't work since everything's read-only). I use niv to update things, e.g. - Update firefox's version field in nix/sources.json - Update its url field based on its template (for Firefox that's "https://ftp.mozilla.org/pub/firefox/releases/<version>/mac-E...") - Fetch that file to the Nix store - Update firefox's sha256 field to match that file's Running nix-build will create a new system package, and the nix/sources.json changes can be committed to git. I had R working with Nix on my work Macbook, but this was on a version of Nixpkgs from several months ago. Of course, one of the upsides with Nix is that I can look up the exact version of Nixpkgs that was working for me because it's in a git repo with the rest of my dotfiles :). Thanks for the summary, it is very helpful! I love Haskell and its philosophy and I'm intrigued to find that there can be some of it in a package manager as well. One question: How do you properly install things like R on the M1 Macs, then? Do you use a 'backup' package manager? Yes, agreed. Someone asked if there can be a progress bar on Homebrew. Instantly shutdown the thread. Homebrew is old and it needs to be replaced. It has served its time. > Homebrew is old and it needs to be replaced. It has served its time. You are more than welcome to build your own alternative. Homebrew is open source and volunteer run. I'm sure there is a very valid reason requests like this get shot down. No wonder open source maintainers hate dealing with people like you. I don't hate it, I just think there is room to improve. Also, no need to accuse me of not understanding open source development. I am thankful to you and others, at the same time it is I think OK to say it needs to improve. Although, I believe like a lot of technology pieces that sometimes it is not worth improving - just start from scratch. There is a lot of baggage with Homebrew. It seems this is the issue in question: https://github.com/Homebrew/brew/issues/9186 The maintainer did close they issue but they didn't lock the thread. If there was a very valid reason the request got shot down, it went unstated afaict. (Fair enough if they just don't allow feature requests on the issue tracker, I guess) > Homebrew is old and it needs to be replaced. It has served its time. Counterpoint: I've spent serious time (years) with, oh, six or seven package managers, in a couple decades of using package managers. More if you count Flatpack and stuff like that. Brew is my favorite, all-around. Package selection is incredible. Stability is excellent. It's a little slow and I think defaulting to updating the package list damn near every time one invokes the command is a really weird choice, but that's configurable. My second favorite? Docker :-) Third would probably be portage (from Gentoo), I guess. Then apt/dpkg. Worst I've used is easily Macports, and it's not even close—however, I switched years ago because it broke constantly under normal usage, on my machine, so for all I know it's great now—and I've never forgiven RPM-based distros for all the time I lost to RPM hell over the years, and for going so long with worse CLI tools than apt. Void's seems very nice, but I've not spent enough time with it to feel the rough edges. > I've never forgiven RPM-based distros for all the time I lost to RPM hell over the years, and for going so long with worse CLI tools than apt. apt and dpkg are way behind dnf and rpm these days, both in terms of the structure of packages and the interfaces of the high-level CLI tools What do you like best about Homebrew? > What do you like best about Homebrew? Some of its advantages are, in a sense, unfair, because it doesn't manage my OS or its most vital pieces, which is part of why it's never, in nearly a decade, broken my system the way any others have. Homebrew's never made my system unbootable. Homebrew's never broken my GUI. Homebrew's never fucked up some driver that had been working totally fine. In another sense, that's totally fair if you consider the platform as a whole. It's really hard for Linux overall to separate user packages from system packages, mostly because it's all mixed up together in a way that it's not on MacOS or Windows—but a distro could take that approach, if they wanted. So, that'd be one thing. #1: Separation of a very complete and capable base GUI OS from the rest of my packages, with the base OS upgrading in lock-step and very reliably, and everything else totally disconnected from that. #2 would be the package selection. The number of times I've found some lowish-star-count project on Github, blindly done "brew install [guess at package name]" and had it work, is crazy. Non-free stuff? It's there too. AFAIK Gentoo and Arch are the only ones that even come close—but then you're stuck either hoping rolling-everything doesn't break your system, or trying to maintain a jenga tower of pinned packages and still risking breaking your system when you eventually do update them. I'm actually not sure there's a single piece of software on any of my Mac hardware, for work or play, that's not either, 1) from Apple, or 2) installed via Homebrew. And I've never added any kind of 3rd-party repo to it. (a minor point: it's also nice that I can guess the correct package name more often with Homebrew than on Debian and Ubuntu) #3 is that it has never, ever broken for me, under normal operation. An "install" or "update" or "upgrade" or "remove" has never broken my package manager itself, or its package database, or left its installed dependency tree so deeply fucked that I lose an hour or five un-fucking it (in fact, I've never seen that happen at all with Homebrew). OS upgrades have occasionally made me run a one-time fix to its directory permissions. Maybe that's happened twice in a decade of use? AFAI can recall, that's it. Macports borked itself badly under normal operation every 3-4 months while I was using it, while having a smaller package selection and compiling (=taking way longer) more often—this was, IIRC, like 2011, though, so again, it may be better now. Every other package manager I've used for any serious length of time, I've repeatedly seen get in weird states that required substantial fiddling before I could use them for anything again, after what should have been boring and normal usage. (see point #1 for why it would also be a lot more tolerable if it did have that trouble, than it is when that happens on, say, Linux, and indeed, that's the only reason I was able to struggle along with Macports for a little over a year before switching—I could just rm its directory and start over, after the second time from a script that did it for me, without causing any harm whatsoever to my OS itself) > Some of its advantages are, in a sense, unfair, because it doesn't manage my OS or its most vital pieces[.] [...] [T]hat's totally fair if you consider the platform as a whole. It's really hard for Linux overall to separate user packages from system packages, mostly because it's all mixed up together in a way that it's not on MacOS or Windows—but a distro could take that approach, if they wanted. Yeah, this is a valid insight! It's essentially the approach that contemporary free BSDs take, and it's a direction popular distros are gradually trying to move in by shifting end-user applications to containerized package managers like Flatpak, AppImage, and Snappy. Fedora Silverblue is an example of an extreme case, where the base system is read-only. > #1: Separation of a very complete and capable base GUI OS […] I suppose the BSDs fall down here, since their base system doesn't include a GUI. > Every other package manager I've used for any serious length of time, I've repeatedly seen get in weird states that required substantial fiddling before I could use them for anything again, after what should have been boring and normal usage. I've only really seen this happen when using distro package managers to perform several years worth of release upgrades or convert between distributions. I like the uniformity of Linux's package management tools (i.e., the ability to use the same tools to manage OS components as end-user software), but I can see how using one thing for OS components, especially if it's reliable and mostly hands-off, but then using something else for end-user software can be appealing. Yeah, I think these days I'd find FreeBSD a better fit than Linux. I'm... increasingly kinda over Linux. Red Hat's continuing political success in pushing projects it (more or less) steers toward de facto standard status, while having (in my opinion) god-awful taste in project architecture and goals, has the whole ecosystem going a direction I don't like. Nonetheless, it'd be nice to have the GUI included in the "this doesn't break" part of the OS. It'd be pretty unlikely that Homebrew could screw up an entire day for me, because the scope of things it's likely to even be able to mess up is limited. A package upgrade breaks X or Wayland, though? Ugh, there goes the day. Increasingly, I think stronger guarantees that the GUI layer does not break, or become unstable (god, I've seen so many X crashes in my life) except under very unusual circumstances, would be table-stakes in my switching full time to another workstation OS. Which leaves me options of... MacOS or Windows. Win10/11's telemetry and adware crap is unacceptable, and I like unixy tools, so that leaves me one option. :-( Then again, ZFS with snapshotting can paper over a lot of packaging & system-update problems, which is part of why I'm thinking there may be a lot more FreeBSD in my future (yeah, yeah, I know, Linux has it too, but I've developed—like scar tissue—a healthy mistrust of Linux distros' ability to deal with "non-standard" file systems, when it comes to the root filesystem) Homebrew is still here to stay. If Apple provide a package manager, then it likely that people will swarm to that but again it depends on how much capabilities and feature parity it have over other package managers. Microsoft is working on this with their Winget. Homebrew is great for casual and daily users since we don't have to keep hunting for software updates. And Homebrew does have a progress bar for each app. The bar is for the downloading but other than that, I don't think it need a progress bar for installtion since they are quick to install than using the GUI installation. Winget isn't really a package manager so much as an installer wrangler and frontend for the Microsoft App store. Of the nearly 30,000 manifest files inside the winget repo, fewer than 150 of them define a single dependency. The rest run executable installers that can do basically anything, or just grab an app from the Microsoft Store. It doesn't do anything to eliminate the chaos of every program doing whatever the fuck it wants to get itself installed or create a common ecosystem of integrated packages, which are the main points of package management. Winget also lacks most of the core functionality of a package manager, including knowing how to uninstall anything at all. But uninstallation will never work reliably on winget anyway, as long as winget an installer wrangler, because installer wranglers inherit unreliable and bespoke per-package installation/uninstallation procedures from existing Windows apps. After a year and a half in public, Winget is still just Chocolatey but faster and much worse in every other respect. There's basically nothing on offer on Windows if you want a package manager instead of just some automation wrapping executable installers.
(Scoop is the closest thing, and it's sort of just now swinging back from the verge of death after months in limbo.) The fact that Microsoft announced a project and called it a package manager has done more or less nothing to change that. What’s an actual concrete example of a problem you have with homebrew? Here's my previously posted actual concrete example: https://news.ycombinator.com/item?id=27903499 I know that there are usually workarounds to these problems, but they tend to be obscure, and change frequently, and are not simple to look up. Pacman, apt, portage, etc have much more stable interfaces. But the workarounds for my edge cases in Homebrew from a year ago don't work anymore. Even knowing what is an edgecase or not is a problem. Most of these issues would be fine if the interface changed as slowly and consistently as Linux package managers. I posted a detailed comment of why Homebrew is a problem for me[1] if you would like to see. It’d be nice if it asked if it’s ok to remove old versions of dependencies when installing a new package. It’d also be nice for it to not upgrade, say, wireshark and nginx when installing, say, ffmpeg, without asking first. You might be interested in `HOMEBREW_NO_INSTALL_CLEANUP` and `HOMEBREW_NO_CLEANUP_FORMULAE`. See `man brew` for details. Yeah, I’m going to put those in my rc file. I’m surprised they aren’t the default options but that’s not a big deal. `HOMEBREW_NO_INSTALL_CLEANUP` used to be the default. Many users complained about how much space Homebrew used, and were shocked by how much they saved by doing `brew cleanup`, so now it’s done by default. Brew's package selection is unmatched and it works great for a particular purpose. Keep using it, but stop using your dev machine's system package manager to manage project dependencies. There are a bunch of good reasons not to do that, anyway. For server stuff, probably use Docker. Brew is for your tools you personally use, that you mostly want to be at latest. If a project needs, say, a compiler and library at a specific version, define that in the project and use something that will manage that for you, which will typically be language-specific—damn near every language has an "Xvm" now, where X is the first letter of the language or platform, or else something equivalent, right? That's not to cater to Mac with Brew, it's because that's a very good idea on any platform. You shouldn't use the system package manager on Ubuntu or Red Hat or whatever the way you're using Brew, either, unless you're exactly matched to your deployment target—same distro, same version, and you don't upgrade locally unless the servers are getting upgraded—and are scripting your local package installations and upgrades with the same script you use to deploy on the server. Brew used to be much more useful to me. Using brew years ago, it was way less opinionated and more flexible, which was perfect for OSX. Homebrew's response to criticism seems to be to continually narrow the field of users and the scope of their project, and tell users to use something else. People are only frustrated because they found it to be so helpful. But I suppose that it had to mature, or something like that. I've personally had very good experiences switching back from Homebrew to MacPorts. Wouldn't go back to Homebrew. I find it much faster now, which is quite funny to me because back when Homebrew was new (and a lot less opinionated) it was much faster than MacPorts, especially with the bottles (binary packages) feature. Today, MacPorts has binary packages, doesn't randomly force incredibly slow updates when I just want to quickly install something or run a package search, has entirely opt-in telemetry through installing a package (mpstats) and still lets me customise packages (through the variants system, which is more powerful than the with --with-feature system Homebrew seems to have abandoned). The only pain point I've found is upgrading to new macOS releases, though I've often found just reinstalling MacPorts itself will get me going again in a pinch without having to take inventory of my installed ports and reinstall everything from scratch. > I've personally had very good experiences switching back from Homebrew to MacPorts. Wouldn't go back to Homebrew. I have been much happier with Macports than with Homebrew (which I try every other year to see if it’s improved). > The only pain point I've found is upgrading to new macOS releases, though I've often found just reinstalling MacPorts itself will get me going again in a pinch without having to take inventory of my installed ports and reinstall everything from scratch. I used to use the commands suggested on the Macports website (save the list of installed packages to a file; update OS; install Macports; re-install ports). But since a couple of upgrades I have decided to use this as an opportunity to get rid of ports I don’t actually need. Macports made my life hard three times I upgraded OSes from having to migrate Macports versions, just leaving broken files in /usr/local/*. The worst instance was Mavericks, when it suddenly said i had no ports, after 10 years on the same image. It did something similar during the Monterey update. I'm now 100% brew but still have broken artefacts from MacPorts littering my system. I'm also still using the same image from 2012, moved to new machines with TimeMachine, so that could be it, but I'm damned stubborn. Perhaps it would be great opportunity for you to clean install macOS to start anew and start fresh from there without worrying about lingering old files in the system. I know, I know. I tried to do a clean install on my 2012 MBP to get a feel for what would break. Unfortunately the 2012 is out of service and Big Sur kept powering down randomly. It runs Ubuntu 20.04 just fine tho. I think OSes shouldn't atrophy. So I'm super-testing TimeMachine. Homebrew packages break on major macOS upgrades, too. It's not up to the package managers what Apple does or doesn't rip out on a given release I also switched back after one too many issues with brew file ownership and permissions. I think, you don't need to choose any other package manager because homebrew is the most mature thing on macos. Just disable autoupdate when installing something:
HOMEBREW_NO_AUTO_UPDATE=1 brew install <formula> Homebrew stinks of over-optimization and deletionism. In contrast to how it used to be, Homebrew now seems designed for people that only want popular bleeding edge packages, and nobody else. If I just wanted to install google-chrome, sublime, and skype, I would use the app store. I used to use Homebrew to install more obscure packages that were useful to me, like Arduino, Processing, and some specialized bluetooth tools. It is especially frustrating because there are good reasons to not want bleeding edge. OSX ages pretty well, and new versions of macOS often remove important features. As long as you have security updates, I think it is reasonable if you run an older version of macOS. For some of us, Catalina and Big Sur were pretty painful, especially if they are a liability for how you make money. Losing total-terminal and total-spaces were a big blow to my workflow, and not at all simple to work around, as well as certain accessibility software. Homebrew being so aggressive about updating itself compounds many other issues it has. For one thing, major changes are made frequently, and it is very aggressive about pruning packages and only supporting recent version of macOS. So, I have lost count of the number of times Homebrew has broken itself. Homebrew will go out of its way to search through its history and tell you a package has been removed, but gives you no easy way to install them. Homebrew could make it fairly easy to revert updates, since so much of it is in a git repo, but it doesn't. I wanted to install an SNES emulator, and I was using an install of macOS from maybe two years earlier, and Homebrew installed a version that was incompatible with my OS. Since it was a cask, I don't think anything depended on it. I had to actually search through the Homebrew repo history and check out that specific formula, since nobody had a tap for it. Unless you are near the center of the bell curve, you don't matter. And it is a shame, because I really think that it wouldn't impose a significant cost to support more of us, and substantially improve its UX. Homebrew used to be a powerful repository of most macOS software. Some of its less used packages could be unreliable, sure, but that was the case regardless of where I got them. Now it is morphing into an App Store. This is part of why I gave up on macOS two years ago and went to Linux full time. I can't rely on Homebrew. You may be interested in MacPorts, which literally supports back to Tiger. (Some of the stuff they do to make this happen is actually nuts, e.g. https://github.com/macports/macports-legacy-support.) Awesome, I will check it out. In my case, it was in 2017, and I remember clearly what happened. Homebrew alerted me about the most recent compatible formula, and what commit it was removed in a week earlier. I was then forced to unshallow the Homebrew repo, revert that commit, and then I could install it. I remember I had a good reason for not updating my OS, and I'm sure it was less than two years old. It was incredibly frustrating, and took a lot more effort than it was worth. Homebrew already had the ability to find the last good version, it would not have been hard to automate it. There were no taps at the time, I had no choice but to do it manually. I feel your pain, but can't help blaming Apple rather than Homebrew. They made an OS that users can't bear to upgrade because of removed features, homebrew is merely following their lead and focusing on bleeding edge users. Maybe the logic is: if you don't want constant churn you wouldn't be on a mac. But Homebrew didn't used to be that way when I started using it. And with some fairly small changes it could go a really long way to supporting more users. TBH if I had the time, I would roll up my sleeves and do it myself. The whole reason that I used Homebrew to begin with in 2014 was to help mitigate these problems. I think there has been a cultural shift away from being more useful for a larger base of users, to being optimal and "clean". The least it could do is confirm before it makes irreversible and sometimes breaking changes to your system. I don't see any good reason to delete your distfiles after potentially installing an incompatible package. And compared to most Linux based package managers, Homebrew makes big changes to its interface frequently, which makes it a headache to keep up with. Just look at how much casks have changed in the last two years. Anyway, when I felt I could no longer rely on Homebrew, that was the last straw for me dumping macOS. > They made an OS that users can't bear to upgrade because of removed features I thought the story was that Apple had a problem with GNU licensing, and that was the reason everything got locked to ~2007 software, forcing everyone who wants to use bash and GNU utils to resort to a package manager. Yeah… googling it, people have long been saying GPL v3 is what Apple has a problem with: What you're talking about has nothing to do with what GP said. Apple cheerily breaks functionality for end users when they decide to remove or replace APIs on every major release of macOS. This tends to hit power users especially hard, as they often use tools that extend Apple's narrow facilities for things like handling keyboard input, managing windows, or controlling applications' audio volume through third-party tools that rely on internals of macOS that are not supposed to be public-facing, and those get broken more often. If your favorite tool relies on an API that gets pulled and doesn't (or can't) get rewritten when the new macOS release is in beta, you're SOL. Homebrew breaking on macOS upgrade is also due to one thing that's fundamental to Homebrew's design, which is that it tries to rely as much as possible on that same external base system which Apple provides but destructively revises whenever it sees fit. (This design choice is apparently the author of Homebrew made because relative independence from the base system meant too much recompiling for him on Homebrew's competitors at the time.) Package managers are used to install a hell of a lot more than bash. The idea that they wouldn't be necessary on macOS if only no popular tools were licensed under the GPLv3 is laughable. If you have any kind complex toolchains, then a major update can be a real burden. Just for me, using a customized desktop, iOS app development, and being a zfs user means that I had to set aside a lot of time for updates. And beyond that, Apple has become really sloppy with new releases, and you're best off waiting at least six months for the major bugs to be ironed out. What's 'deletionism'? Is it the act of being fastidious regarding everything under '/usr'? If so, count me in. One of my biggest gripes with the aged Linux install ritual is having to keep the source bundle forever in case I ever decide to uninstall. Why? Because the autoconf makefile is the only rule that cleans up all the bins, docs, and shared files that were sprayed into the OS. Building from source needs an uninstall fix, but no one seems to care. (Or I simply don't understand how w/o keeping the build folder forever... Help?) And since this technique relies on setting an environment variable, you can of course put it in your .bashrc/.zshrc, restart your Terminal, and you'll never have to remember it again: This looks like a really awful solution. Does homebrew not ask the users on each run whether they want update something and warns them about the dangers? Like a responsible software? It does not. If you have an old version of a package, say OpenSSL, that was installed a year ago when you wanted PackageA, then both will be updated when you get PackageB which also depends on it. B’s bottle only refers to the new OpenSSL, and the old version is deleted so you need the new A bottle. No auto update just means you use the state of homebrew-core that you had before you ran the command, rather than updating it right before using it. It makes sense, but then I don’t do full upgrades and get delayed just like OP. I live with it. I switched to Macports back in my Mac days, used it as my daily driver for about 18 months before moving to Linux. Macports has this appearance of being 'dated', and not the cool kid compared to Homebrew. But it doesn't have an opinion on what version of software you can install. Want postgres 9.4? Go ahead and install it. Want some ancient version of some other library? That's fine. Dealing with custom port files and repositories isn't quite as easy, I suppose, but that's not really much different to setting up a debian repo in that it's a use case you won't encounter nearly as often as just installing stuff. I also used nixpkgs at one point, many moons ago. That was also nice but not the most practical on a Macbook with 128GB storage, when you also have Docker, npm/ruby dependencies, and your native Mac software in the mix. It’s a strategy. Package managers have to choose a strategy: slow, crusty and stable or bleeding edge? It’s not CentOS… it’s a local dev machine. The same scenario will occur on Arch Linux. Fortunately Homebrew has lots of helper scripts to upgrade DB’s like Postgres. You probably saw some scroll back with a command to upgrade from 12 to 13 or 13-14 etc. It will even download and install an old version in order to upgrade safely. I understand the frustration but what is the alternative? Pinning every dependency? The reason so many of your packages were updated was likely because a core dependency was upgraded like OpenSSL. It would be nice if you could run Homebrew in a mode where dependencies were statically linked if possible. On my system, "brew list | wc" shows 200 installed packages, and "brew leaves | wc" shows that I actually only intended to install 32 of them. This is a huge dependency tree full of junk I don't even know if I actually need. Static linking would save a lot of space and make your list of installed packages simpler and more understandable. In my experience, developing software on a local dev machine is precisely when fine-grained control of software stack versions is needed. This can be encapsulated with things like docker, but the "I'll globally, arbitrarily update everything to the bleeding edge" strategy isn't friendly for building and maintaining software. That capability exists: brew install postgres@12 nix. It's confusing, but we'll get there. You'll need/want: - nix, the tool/build system. - nixpkgs, the world's most up-to-date and underfunded package repository. - nix-darwin, bringing a NixOS experience to Darwin. Not perfect, but pretty damn great. And optionally, - home-manager, a nix-based opinionated tool to manage dotfiles and other stuff in your home directory. And finally, some things to avoid and experimental features to use: - do not ever use `nix env -i` to install things ala homebrew, apt, etc. This is a trap. You don't want this. - use nix flakes. Sure, it's experimental, but it's what you _do_ want. Reproducible, version-pinned builds. - if you use direnv, or are familiar with it, check out nix-direnv, which is like direnv on crack. Instead of managing your packages globally, manage them per workspace or project. I disagree with "don't use nix-env --install; use <much harder thing> instead". I see "nix-env --install" as a gateway drug to using Nix. It's effortless, and already easy to see benefits from using Nix (like you can easily get the same set of packages on different OSs). Rather, it's worth explaining why it's bad to use. The footgun I ran into was that I'd caused a messy state by installing something and forgetting I'd installed it. Maybe nix flakes and home-manager and nix-darwin are really cool.. but they're also much harder to use, and surely overwhelming for someone who just wants a way to install packages. I think it's like telling someone to learn Kubernetes when they're wondering how to serve the website they wrote. -- I think it's easier to appreciate these once you've gotten your feet wet with nix. > Rather, it's worth explaining why it's bad to use. My biggest problem is that it's not version-controlled. I don't use home-manager, and (still) haven't looked into Nix flakes. Instead, I put `export PATH="/path/to/my/repo/result/bin:$PATH"` in my ~/.bashrc, and inside that repo I have a default.nix file: You never want `nix-env --install` without it being `nix-env --install --attr`, though, because `nix-env -i` is like a billion times slower than `nix-env -iA`, and that gives people a really bad first impression. But yeah I agree, it's fine to use `nix-env -iA <package>` or whatever. It's fine when you're getting started, and it's fine for a slightly more persistent way of trying things out than `nix-shell`. I've never used nix, but it sounds interesting. What getting started guide would you recommend? Ideally one that has all these "insider tips" that you offer, so that I learn to use Nix "properly". Nix with nix-direnv is awesome! So you're saying use nix flakes instead of `nix env -i`? I've been using `nix env -i` without any issues till now, but I'll explore nix flakes. nix-env -i package by package? I think the next step in the journey is describing what you've got installed with a package: https://nixos.org/manual/nixpkgs/stable/#sec-declarative-pac... Package by package, yes. I had followed this tutorial: https://youtu.be/NYyImy-lqaA What your link describes though looks better. I’ll give it a try, thanks. A lot of the usual "it's bad" "it's not bad for me" discourse in these threads. Could people raise the bar a bit and talk about specific problems they have had with homebrew - other than the auto-update, which we've already noted can be changed by setting HOMEBREW_NO_AUTO_UPDATE=1? Thanks. Certainly. For me it started when they removed the entire scientific stack, which broke my install scripts / brewfile. Then, they removed the ability to customize builds, which broke my emacs build. Also hit the OP issues. In every case they were removing features that were working just fine, and in the process creating work for me. Got fed up, switched to macports (after checking out nix), and so far things are back to the way I like it. It's hard to pinpoint what the problem really is, but me and a few colleagues had corrupted brew installations lately, where various things would just randomly stop working after updating a single package. The trick of disabling auto updates doesn't really work when you do want things to be updated. Sometimes there are conflicts and the usual path was to upgrade everything. Not that it matters: it seems that the usual "upgrade everything" from 2 years ago has changed into "delete every folder and reinstall everything from scratch". Also, a few years ago, updating everything or reinstalling lots of things from scratch was fast. Now, it requires much more time due to the massive number of dependencies every single package has. Also, the maintainers have a super weird aversion to everything that is "too simple". I've seen packages rejected because "the build process was too simple and there are no dependencies". The excuse was that "users can build it themselves". This is not for silly stuff random people wrote on weekend, this is for 20 year old tools used in production written in C. I can't say for certain why or how it got worse for other people, but for me it started when they doubled down on dynamic linking. That's why installing takes 5x the time it used to, takes more disk space, breaks all the time and makes organising your tools a fucking mess. As for me: I've replaced most of my Brew usage with Docker (for most development tools) or with static binaries compiled weekly using Github Actions (for cli tools like youtube-dl, etc). I'm also using download links straight from language websites (Rust, Ruby, Python, DotNet, Haskell). > Also, the maintainers have a super weird aversion to everything that is "too simple". I've seen packages rejected because "the build process was too simple and there are no dependencies". The excuse was that "users can build it themselves". This is not for silly stuff random people wrote on weekend, this is for 20 year old tools used in production written in C. wtf? What could be the possible motivation for this? Is their build farm really straining or something? Can you point me to this example? > I'm also using download links straight from language websites (Rust, Ruby, Python, DotNet, Haskell). this sounds downright medieval to me, like why even use macOS for development if the software management tools are so bad that that's what you're doing > this sounds downright medieval to me, like why even use macOS for development if the software management tools are so bad that that's what you're doing Because it's only one tool that really sucks: Brew. For the languages I specified, it works quite well. Even using C++ with vendored dependencies works better than average in macOS for me. For most of those languages I mentioned, Brew already offloads the real installation and updating of the language itself to a language manager (via rvm/rbenv, nvm, pyenv, rustup, whatever), so it doesn't make a difference, except without Brew I get a more stable, officially supported, environment. Also, DotNet was a Cask until a few months ago, so they effectively were just downloading something from the official website just like me. Could you give (or point to) a description how compiling binaries using GitHub Actions can be done?
Sounds really nice. :) Sure! Here's an example with jq: https://pastebin.com/gHu8dTgQ Honestly the best option is asking maintainers of each program to add a static build to their projects in order to make their projects able to run without installing dependencies. Problem is, some maintainers are very anti-static-compiling. They deprecated the "brew cask install" then completely removed it only ~20 days later (in late December!), breaking many other projects in the process. I'm a bit wary on relying on it now. It's a shame, because it's a really useful tool (and I actually like the auto-update). My preference for MacPorts starts from the fact MacPorts keeps their files separate from the system files. To uninstall, just delete the folders. I’ve seen homebrew overwriting system files and thoroughly breaking machines (that’s no longer that easy, but more thanks to Apple). Haven’t tried many other package managers (Fink, when G3 was bleeding edge), but MacPorts has the feel of an exquisitely well engineered tool. Homebrew staff have rejected the concept of automatic upgrades for security, stability, and semver: https://github.com/Homebrew/brew/issues/3428 One thing has ruined my week is running brew upgrade $package instead of brew update $package. One of those commands will update the package and the other with update everything you’ve ever installed with Homebrew. You can disable autoupdate by setting HOMEBREW_NO_AUTO_UPDATE env https://github.com/Homebrew/brew/blob/7d31a70373edae4d8e78d9... Just to add how to use it:
HOMEBREW_NO_AUTO_UPDATE=1 brew install <formula> This seems to control whether to upgrade homebrew itself? Upgrade homebrew itself AND other packages, see https://computingforgeeks.com/prevent-homebrew-auto-update-o... In the past i used brew to install EVERYTHING, this lead to situations like in OP of DBs, compilers, interpreters and other things that i wished stay frozen to get updated by surprise Since then i moved to manage everything that needs to stay frozen by either a versioned formula/cask (python@3.9 and so on) or use brew to install a version manager (asdf,pyenv,pipx) and then use that. Results are much better once I did that Although versioned casks/formula exists you will still get minor versions - python@3.9 will update from 3.9.0 to 3.9.1 is that ever exists. so while completely protecting you from upgrades it does minimize the the risk Also, there is really no need to install virtualenv directly from brew. A very bad practice. Of course doesn't change the fact that brew did things that are unrelated to what you asked of it I started using nix. It’s difficult and search results are not often helpful. Your honesty is refreshing. I've seen countless people trying a new technology/tool, and overselling it as 'amazing' and super easy. Seeing a post like yours helps to keep the discussion grounded in reality, with tools that looks nice on paper but have practical downsides as well. I agree with this general sentiment, but I'll add that it's worth reading most posts about Nix through this lens. Finding your sea legs is rough, and IME people who completely deny this are few and far between. The reward is a pretty impressive lever. The thing I use and like most in nix is nix-shell. I set up a shell.nix file on my projects, and together with direnv the environment switches with the right dependencies for each project. Shameless plug, I wrote about this recently on medium: https://link.medium.com/cYkKMtdHRkb Nix IS amazing. It is NOT super easy. Parts of it are getting easier, and it has an exceptionally smart and helpful community. But it's not smooth and pretty and easy to pick up like Homebrew is. (And that does suck) Fight through it for another couple of weekends - the Stockholm syndrome feels sooo gooooood! I found it much more helpful to find people's own Nix setups and learn from them, by the way. Mine is at https://github.com/Smaug123/nix-dotfiles - like all such things, it's a WIP, and I'm definitely still a noob, but bits of it may be able to help. you can search via web interface - https://search.nixos.org/packages?channel=nixpkgs-unstable Finding packages is not the hard bit :P the hard bit is learning how to do anything. The Pills teach you how the language and "runtime" works, but nixpkgs is gargantuan and requires a whole lot of extra learning, as is home-manager, as is nix-darwin. Nox is much easier interface to nix. I often use that instead of nix-env. One vote in favor of macports here. I first went from homebrew to nix. Nix was fine, the concept is great but I found the language to be difficult. Ports is closer to homebrew in its simplicity, and has a big catalog (including much of the scientific stack that was removed from homebrew). MacPorts and Nix are fine to use alongside each other. I recommend using Nix alongside a couple other systems so you have escape hatches. If you use Nix with Flakes, it'll feel a lot faster (much faster than Homebrew). If you use old-school Nix, it'll be slow if you call some deprecated commands. I cannot remember why, but I chose pkgsrc (https://pkgsrc.joyent.com/install-on-osx/). It is however of varying quality. I have also used it. I switched once homebrew installed conflicting versions of OpenSSL and wasted weeks of time trying to get both MySQL and vim working at the same time. It is rough around the edges as some packages are missing if I remember right. I too only used pkgsrc when I had a Mac developer machine. Afaik mostly because of their superior dependency management. I built a feature in arkade [1] to pull in binaries for CLIs for infrastructure and developer tooling that I wanted to use - it now has 72 CLIs that you can pull down with a single command and most importantly, as near to instantly as you're going to get. For instance: arkade get kubectl@v1.22.1 yq helm faas-cli It's not got anywhere near the catalog of brew, and doesn't compile software, or help you find lib-xyz for your Yubikey, but it is really fast and has a growing community behind it. It works on MacOS, Linux, Windows and arm hosts to determine the correct download URL and pull in a binary. [1] https://github.com/alexellis/arkade Contributions are welcome. I've been using MacPorts for over a decade now, and I'm very happy with it. I certainly much prefer its default choice of install prefixes (/usr/local should be reserved for things I install manually; /opt/local doesn't clash with anything else I'm aware of). After years of dealing with it I just run most of the software from docker containers and use homebrew for random cli tools. Homebrew needs to do a better job of communicating that you shouldn't use it for project dependencies, because every time there's a Brew thread on here, a bunch of people come out complaining about this kind of thing. "Homebrew broke my build by updating PostgreSQL and Redis". They shouldn't need to communicate that, but clearly, they do. Use Docker, probably. Homebrew is for your tools, not your dependencies. Is the system you're deploying to a Mac with Homebrew? Do all your collaborators use Macs with Homebrew? If the answer to either is no, why were you trying to use Homebrew for that in the first place, even if it were otherwise good for that? Would you use Homebrew to install Node modules for your project? Python packages? No? When why are you letting it define which version of PostgreSQL this project depends on? Genuinely, is this attested as the project philosophy anywhere? Because right now https://brew.sh leads with "The Missing Package Manager for macOS", which would lead anyone to think it does the same as a package manager for Linux, and they're _full_ of developer dependency packages. The thing is, you also shouldn't be using apt or whatever for managing project dependencies, unless you and all your fellow devs match Prod exactly, both with distro and exact versions of everything. People do it anyway, but it's bad practice for exactly the same reason that using Brew for it is bad practice, and it's barely easier than doing things right (and, very early in the life of a project, it'll become less easy than doing it right). It's not about project philosophy, it's about using the right tool for the job. Now, since people keep running into this problem, maybe they ought to put a warning about that right up top in a big banner on the site, I dunno, but it's not specific to Homebrew. Imo you shouldn't use a typical Linux distro package manager for project dependencies either, unless you're only ever gonna deploy every app you're developing on that machine to the same version of the same distro. This is not because there's anything really wrong with them for what they're designed to do, but they install packages globally, and different projects may want different versions of the same thing. This was also going to be my comment, homebrew for cli tools and casks for some gui apps, all other environments in docker containers. Ditto. Anything mission critical runs in Docker via Docker-compose, so I can manage every aspect. And for any cli tool that's also mission-critical, it can be run within Docker so I can pin it or keep it bleeding edge more easily. For the important stuff, performance and ease of installation can take a side seat to stability. The fact that homebrew is awful is one of the huge reasons I don’t use MacOS for development like all my co-workers. Instead I use Windows with WSL because the company won’t allow me a Linux laptop. If I had to use a Mac my solutions would be to alway use Docker, a virtual machine, or a remote dev environment of some kind. I don't even bother with WSL anymore, most dev packages I use comes as prebuilt Windows binaries bundled as zip-packages that you can unzip anywhere and run. No dependency management, no magic updates, no PPAs, run how many versions of the same software you like, delete folder when don't need it anymore. Super easy. Granted I don't work much with compiled languages anymore so I don't need to setup some complicated tool chain, but when I need to do that I usually go for MSYS2 or WSL when it has to be Linux. What’s an actual concrete example of a problem you have with homebrew? I feel the same way. I've had a very good experience running xubuntu. I hate that you are forced to sign up for an Apple account just to install XCode which is required for Homebrew which pretty much makes the machine useless. This is not correct. All you need is the command line tools, which brew tells you to install with `xcode-select --install`. No account needed. Oh shoot thank you, for some reason when I last looked it seemed it was implied you needed the whole Xcode package. Guess I'll give that machine another go now. Funny, I use Windows with WSL and the first thing I do when I set up a new WSL instance is get homebrew (linuxbrew) and install everything else through that if available. apt/dpkg and any other package manager that scatters files all over the filesystem is just an absolute disaster that--if you do a lot of R&D tinkering--eventually renders a computer unusable after a few years. That's what happened to my first WSL instance; no apt command would work anymore because it had broken its own dependencies. > apt/dpkg and any other package manager that scatters files all over the filesystem is just an absolute disaster that--if you do a lot of R&D tinkering--eventually renders a computer unusable after a few years. That was your experience. Mine is that after using apt for a lot of R&D tinkering for 3 years, everything runs fine. apt is global. Sometimes that's great and that's what you want. Sometimes it's not. Another way to keep that in mind is to think as apt as "bad at doing cleanup". The thing is that you have to have a separation between things that you want on your system all the time, and things that you use for one or more projects. For the first, apt is perfect. For the second, usually that's where I use tools like nvm for node versions, languages package managers like npm, and also docker (which I use a lot for development databases). For a backend JS project, I would install nvm, npm and docker globally, and then for the project I would use node through nvm with a .nvmrc, install packages locally with npm and use a package.json and package-lock.json, and run the development database with docker through a npm command, usually the latest PostgreSQL version. The link between the backend and database would be made with environment variables, because that's also what I use in production. The difference between the global (apt) scope and the project (the rest) scope might seem like a failure of apt, but I personally see it as the difference between global and local scope in programming: both have their use, we separate them for a reason and we don't put the same things in them. > apt/dpkg and any other package manager that scatters files all over the filesystem is just an absolute disaster that--if you do a lot of R&D tinkering--eventually renders a computer unusable after a few years. That's what happened to my first WSL instance; no apt command would work anymore because it had broken its own dependencies. This is the craziest claim that I've ever heard about package management, I think I would love to hear a more complete story IMO this is so cringe and far from the truth. If you want a slow non-standard package manager they litters your computer and breaks things then that is Homebrew. If you want standardized installs then use apt/pacman/APK and just don't install a lot of user packages not in default repos. Homebrew is one of the slowest package managers I've ever used. When I first started using Macs 5 years ago, I explored both macports and homebrew. As soon as I saw how homebrew screwed around with root, and how it installed itself in /usr/local and required strange machinations I looked more at Macports. Ports have a long history from the BSD Unices and Macports keeps that ethos. It installs neatly in /opt and doesn't screw around with MacOS except in the approved ways (eg for Java and Python frameworks). I still use Macports and it's had everything I've needed and when I hit bugs during the Big Sur beta, they were fixed quickly and directly, while still complying with Apple's beta policies. I've dabbled with Nix, but it's a bit too* prescriptive for me. Came here hoping to see some recommendations on getting started with nix on MacOS, or maybe a favorite blog post or series. I've tried the nix.dev installation + nix-darwin instructions 4 or 5 times but never make it very far. Seems like I should generally prefer a multi-user install? Does nix-darwin play well with that? As I supposed to `sudo -i nix-channel --add / update` or not use sudo? Is unstable the generally recommended channel? Do I use sudo with nix-darwin, since it's supposed to be kind of like NixOS / system-wide? And then I usually give up, count my blessings for homebrew, and resolve to never again waste another 2 days trying to figure out nix. Rinse and repeat in 1 month. > Seems like I should generally prefer a multi-user install? Does nix-darwin play well with that? On my macOS computer, I don't have a multi-user install. Never had issues with it (but I also only just use one user account for everything). Just tried running the nix-darwin installer, and it sets up some extra users as part of that anyway. > Am I supposed to `sudo -i nix-channel --add / update` or not use sudo? Is unstable the generally recommended channel? Do I use sudo with nix-darwin, since it's supposed to be kind of like NixOS / system-wide? Once you've got nix installed, in general you don't want to "sudo nix ...". Unstable is fine; but if things break, it's easy to rollback changes with Nix, so that you can use the versions which worked. There's no need to start out with wrangling with nix-darwin. Just starting out with "nix-env --install --attr nixpkgs.<whatever>" will be closest to how homebrew is used. Before I used homebrew, I was a MacPorts user and that was a terrible experience for me. Been running with homebrew for at least a decade since, and I cannot recall any negative encounter other than the hassle of grabbing xcode command line tools when I upgrade a major version of Mac OS, which is a minor detail. I get the appeal of MacPorts & a dedicated separate package path but it caused all sorts of problems for me v. the integrated approach of homebrew. O there was another negative now I remember -- had to uninstall brew version of MacVim & install manually. But that not a big issue either & typically, all of the brew packages are CLI only for me. A while ago there was a post on HN about pkgsrc[1][2][3], which works on Mac, among other platforms. I haven't tried it, but it may be worth a shot. [1]: <https://rubenerd.com/using-netbsds-pkgsrc-everywhere-i-can/> [2]: <https://news.ycombinator.com/item?id=27293108> [3]: <http://www.pkgsrc.org/> I checked and it appears to suffer from the same problem as MacPorts in only making the ancient version of awscli available: http://cdn.netbsd.org/pub/pkgsrc/current/pkgsrc/net/py-awscl... I wasn't able to immediately find any issue tracker for pkgsrc packages, and it seems they're a "mailing list and IRC" type setup, which if true means it definitely isn't for me i agree that most people don't want to bother with mailing lists and/or IRC these days, though there are a couple of alternative options https://www.perkin.org.uk/pages/pkgsrc-binary-packages-for-o... https://github.com/joyent/pkgsrc It appears those links made the same assumption as did @alnja8a in the sibling comment: my concern is not, at this moment, bugs in pkgsrc itself rather my concern is with tracking who is working on fixing (in our example at hand) awscli-v2 Now, maybe if I go trawling through mailing lists or (shudder) IRC I can find out what the status is, but that's beyond my level of energy expenditure for some niche packaging system yeah, i definitely think your point stands the only place i know of where you can (easily) report an issue with a specific package is https://github.com/joyent/pkgsrc/issues why does the protocol by which bugs and problems get tracked matter, its not like you dont have an email address already... the bug reporting tool is mentioned on the packages page but not the most clear. http://www.pkgsrc.org/#index2h1 I did qualify "for me," indicating that system is not the way that I want to interact with a project The answer to your question is that searching years of email threads is not the same UX as navigating to a dedicated page for managing the lifecycle of a (bug|feature) that tracks when I can expect a new, or working, version of -- in this case awscli-v2 -- to show up, or (if reasonable) that I can indicate to others that I'm willing to take responsibility for doing the work to package it, in order to avoid duplicated effort Plus, again for me, I don't need more email traffic in my life I understand for you, and i understand the UX is different but i dont see how github issues are much better, compared to something like lists.sr.ht or lkml. I find github issues are just as disorganized in a lot of projects with lots of closed due to duplicate and etc. guix has a bug tracker backed on email that has the ux i think you might expect, i find at least, it worked (while i contributed to it) just as well as a non email workflow. Anyway I didnt really consider that this is a personal thing so fair enough to not want to deal with the different problems. mailing list, IRC, and CVS, that is... I gave up on brew and other package systems like it a long time ago. I found it far, far easier to install Ubuntu 20.04 and use that as a utility machine. Even on a system like that, many times when I interact with package managers like conda, it ends up compiling some huge package from scratch. I consider that a failure. I very much dislike that autoupdate is the default. This is user-hostile. I instructed you, brew to install something. Not to spend an eternity autoupdating things. I switched to MacPorts and never looked back, about two years ago. So far a much better experience than Homebrew. I'm surprised people still use postgresql from homebrew. Like honestly, I use brew for any CLI command only like zsh and git. Well maybe some programming language as well but I would never, ever install database with it unless I'm desperate. Anything that can be dockerized, just use docker. Postgres from Homebrew used to work extremely well. It was faster and more reliable than Docker, not to mention lighter on your machine. Now, it's unusable. If you've only got a Mac with 256gb of storage, then throwing away ~100gb on containers is a nightmare ...because your other 156 gigs are being eaten up by Xcode. Seriously, why the hell does Apple even sell 256gb laptops these days? For me it's just my personal one I've had for a few years, I couldn't really afford to spend more when I bought it as I was a student. I think the new MBPs start at 512gb now I use and love Postgres.app on macOS, separate instances of lots of versions of postgres at the click of a button. Docker has a considerable barrier to entry if you've never used it before. Flatpack has horrible download sizes and result in non-standard invocation command lines. You’re right. Some of us only realized our mistake when it was too late. I never understand these complaints about homebrew. Of course, that's because I set Is it a bad default? Sure, probably, spooky-action-at-a-distance is not great. But it's just a default, if you don't like it, change it, one line of shell config is simpler switching distros. There are some tips which others might already have alluded to: 1. Python has built in virtual environment support. python -m venv should do the trick.
2. Use docker for running Postgres, MySQL etc with some directory mounted for data. nix has a very steep learning curve, but I have been using it on macOS for a while now. Can’t recommend it enough. Even started using it on my personal PC, which allows me to use the same nix flake on both systems. EDIT: Personal PC is running NixOS not Windows. Nix does have a steep learning curve. But, you're also very likely to install most of what you want easily. Some other downsides: - Nix installations can take time. Nix packages can either build from source, or download from a binary cache. If it's not in the cache, it will build from source. - Nix can take up a significant of storage space. However, e.g. Nix is amazing at rolling back package installs; and installing a package with nix won't ever break another package installed by nix. If you like the phrases "pure/immutable", "declarative", "functional programming", etc. then you'll like Nix. -- If those words don't excite you, then you'll not like Nix. I didn’t think brew upgraded packages automatically? Just updated the index? Am I missing something here? They have a separate env var with language that matches OP's experience: updated with an actual hyperlink: https://github.com/Homebrew/brew/blob/3.3.2/docs/Manpage.md?... Wow, terrible misfeature. Why is that the default? I could maybe understand it if it were just dependencies, but upgrading everything is insane. It doesn't upgrade everything. It upgrades dependencies and dependents of those dependencies (which, admittedly, can feel like everything). It does this because the alternative is sometimes breaking things. A "breaking things without this example":
- you want to install something that depends on `readline`
- the binary package for the thing you want requires the latest `readline`
- this upgrades `readline` on installation Now, we have a choice. Either we upgrade _everything_ that depends on `readline` that you have installed or we knowingly break some of the things you have installed that depend on `readline`. We choose the safer option by default. If you leave it a long time between updating: you are more likely to have more dependencies updated which requires more dependents to be updated. I'm inclined to agree, but it seems Ubuntu's solution to this problem with "apt install <package> [-y]" where you are presented with a list of dependencies that will get upgraded if you choose to proceed is a much better trade off. The user can of course ignore the dependencies notice by specifying the "-y" flag to force an installation. I hadn't actually considered it until you explicitly mentioned it, but yes: I'd bet a lot of the frustration when using brew would go down if brew just chose to _say_ what it was going to do, and _ask_ if that was ok, instead of assuming the user completely delegates that choice over to brew I am aware of the "brew pin" command, and I do think it would have saved OP some heartache, but I think your suggestion of prompting before transitive changes would be the most pro-user way That would be a breaking change for most scripts so would need to be optional (which feels less useful for people). I see. That makes sense. Seems like the documentation for that environment variable could use some work, unless this behavior and the reasons for it already exist elsewhere. Happy to submit a PR if needed. I do try to `brew upgrade` at least once a day, so that probably softens the blow of these automatic upgrades and explains why I haven't run into many problems with them. > the binary package for the thing you want requires the latest `readline` Have you considered trying to install the latest version of the thing that would be satisfied with the `readline` you already have installed (if it exists), or presenting the user with a choice on whether to upgrade or abort the install? It changed a while back. Definitely surprised me in a bad way in the last year or two and I went through something similar to the OP. That said, I stuck the env variable in my shell setup and I would point out brew does include tooling to upgrade databases from one major version of Postgres to the next so you can get back and running (unless you have cause to stick with a certain version to match server setup, which is a bit of a pain). I believe a proper ‘brew update’ does indeed get ran when doing an install. I think a few years ago they made updating all packages on install the default for some reason I have moved to Nix on two of my three machines – the two laptops have Archlinux32 and MacOS Mojave. I should replace Homebrew installed packages with Nix packages on my desktop iMac too, but in the mean time I have used Macports and some Nix packages for installing new packages. I have reasonable experience with macport, just remember to enable multithreading, or it can be exraordinarily slow. I'm moving to nix. I use a combination of macports and brew, and brew's current behavior is unusable for me, so it's going to be relegated to "use brew for package X until I find an alternate solution for package X". I don't want to have to keep updating brew just to keep from having to wait ever-increasing amounts of time when I want to install a new package or upgrade a specific one. I never have these issues with macports, and won't have them with nix. My current solutions mainly relies on homebrew + Macports + conda. Macports primarily for system level dep. use this whenever packages available. Fall back to homebrew when not available on Macports. This includes brew cask which is unique. This dual setup may requires you to carefully separating the 2 when installing and updating. I do this by installing brew in custom location (not /use/local) which is now the default on ARM. Also some shell functions that selectively control the environment (essentially I use an environment that only sees port to update port, vice versa.) Lastly, conda for dev environments, mostly for Python stack. But it is also useful for other stuffs (conda is a cross platform package manager and in this sense it is similar to brew which also support Linux.) Obviously selections is not as broad comparing to brew. On computing facilities, I used to use linuxbrew to install some “system” dependencies that’s missing, including zsh, mosh, tmux, tree, etc. But obviously I cannot install in the recommended path by brew, which means compiles from source. And it is not robust (and they also say this is not a supported pattern.) I now use conda for this and is very robust (conda uses a hack to install precompiled binaries to arbitrary prefix.) Macports Yeah, macports still works fine. It has an equivalent to the OpenBSD flavors and does versioning fine. The only pain point is OS upgrades. And the fact that it "needs" Xcode installed. It does not, it does need the "command line tools" installed. Certain ports require full Xcode but not too many. If my pre-coffee memory is correct, Macports only needs command-line tools for XCode. I just said what's on the "Installing MacPorts" section of the site: https://www.macports.org/install.php > 1. Install Xcode and the Xcode Command Line Tools Seconded, I've never had a problem with Macports, it just works. One should bear in mind that it seems MacPorts takes a much more puritan approach to packaging than Homebrew, as this 18 month old ticket asking for awscli v2 (and its comment thread) shows: https://trac.macports.org/ticket/60452 I didn't dig into the process for a user to add their own ports, or share new ports with a team, in order to more fully compare the two packagers, but that ticket stood out to me I don’t get how that thread demonstrates what you say. It just looks like awscli 2 is a pain to package; what am I missing? Well, one will observe that brew manages to package it, without any similar stone throwing about the project's "hatred for pypi" or other meta commentary. It seems every project has *some* kind of quirk, but if the packaging system pays that cost once, then its users get "brew install awscli" instead of waiting 18 months for the AWS team to "see the error of their ways" or whatever one is expecting to change about that situation I apologize if "puritan" was too emotionally loaded of a word that hurt people's feelings -- I didn't think "strict" was the right word but maybe I should have used "opinionated?" My read of that ticket is it's a bunch of people saying "this looks hard, I don't want to do it." But if someone wrote a (good) pull request, I expect MacPorts would accept it. The thing about MacPorts is that it's a very community-driven project. Things happen when someone decides to go do them. (Individual maintainers are sometimes opinionated, but I wouldn't say they speak for the whole of MacPorts.) You're doing it wrong. Use Homebrew only for tools and not for critical services like databases and use a cloud server or local virtualization like VMware Fusion (which is free these days for personal use) to run services because major distro packages have far more eyes to check for stability issues. I never understand people trying to run every parts of the development environment through a local machine's third party package managers which would forbid anyone else being able to check your work right then as you're only running it as localhost, not to mention the power can be down at any moment you're not using your machine and you can't even switch machine yourself to work from easily. And then when deploying, you'll be running it on Linux anyway which would likely bring little consistency issues. Do yourself a favor, either spend $5/mo for a cloud Linux and run your stuff remotely or run a locally virtualized Linux and stop using third party package managers no one is using to run on production. And you can happily keep Homebrew and also install Homebrew on Linux to have unified tool sets on your Mac and Linux CLI tools. My experience with MacPorts has been generally good, apart from dealing with OS upgrades which are more painful than necessary, but I've honestly never tried Homebrew so I can't compare them directly. Personally, I find that the whole Homebrew "theme" where every command/thing is named after something alcohol/brewing-related (cask, bottle, formulae, tap, keg, etc) is a huge turnoff. I use brew for system cli tools (eg wget, ffmpeg) and Docker when I need a consistent environment (eg to match what I run in production) I'm surprised I've not seen Spack mentioned here. Spack especially is great for home installs. It's aimed mostly at HPC cluster users, who by default need multiple versions of compiler toolchains, programming languages, libraries, but works well on Mac too. I hate it, but Anaconda might work too, so long as it's not a corporate laptop where you need a license. sorry OP by this is a clickbait rant from someone who just did not made his homework and ended loosing time and data. - Homebrew is rolling - always was and never claimed otherwise. - OTOH for key packages (say databases) it has available versioned bottles. for postgresql one have: In short, none of what happened to the OP is due to inherent flaws in homebrew... QED. brew will auto update on install by default. why are you being this hostile while being completely wrong? I would suggest to not use brew for anything that depends on python, ruby etc. Most of the time you can use e.g. asdf to manage all your dev dependencies.
Also, most softwares allow you to install them without hombrew. (e. g. https://postgresapp.com/ for postgres).
I have seen so many installations screwed up because hombrew upgraded some packages without requiring a confirmation from the user. It's a bit unfortunate given that so many packages are on hombrew. Have a look at pkgsrc - https://www.pkgsrc.org/ It is my first choice since a while and if it's not there I go to homebrew Maybe [1] https://pkgsrc.org/ ? Check [2] https://pkgsrc.se/ before, to see if the availability and the 'freshness' of packages fits your needs. edit: [3] http://pkgsrc.joyent.com/install-on-osx/ if you don't want to compile and have binaries instead. For databases specifically, I like a little tool called DBngin. It's perfect on MacOS for installing various versions of databases. I used to install MySQL 5.7 for an older legacy application because I was finding it challenging to juggle different versions on MySQL w/ Homebrew. I know it handles MySQL, PostgreSQL, and Redis. Maybe is handles other stuff, but I haven't checked. Brew unrelated but especially for PostgreSQL on a mac I can highly recommend https://postgresapp.com Came here to comment exactly this, it’s an awesome app. I have tried macports, spack and to a limited extent homebrew. But the package manager that seems to work best for me is actually Anaconda, which is very surprising. Nowadays virtualenv is built into Python and you rarely need to install it. Try: python -m venv Homebrew is a great system for most situations. Great effort by an amazing group of volunteers. To fix the very real issue you describe, I would like to see a --isolate option for mission critical packages. All the dependencies would be contained in a separate, package specific directory within /opt or /local. The rest of the dependency management could happen without affecting the isolated package. I am Python dev and I never heard of someone using brew to install virtualenv before. I think that you are using brew for the wrong purposes, I suggest the following instead: - Python already includes the venv module, you don't need to install anything, just run `python -m venv <venv_dir>` - You should manage multiple versions with pyenv instead of brew. Pyenv can be installed with brew. Conan can be made to do what homebrew does with minimal effort, I have written some convenience wrappers around it which makes it slightly easier to use for this use case, you can have a look here: https://gitlab.com/aucampia/proj/xonan I've found that `brew pin` is becoming a necessity for things not breaking on my machines. after Postgres breakage that cost me a half day of work, and fish breakage that cost me a bunch of personal time, I try to pin things that have the high probability of breaking due to an update. not optimal, but sadly necessary. Asdf is the bees knees for dev tools. Asdf is 1000! Use it on Linux and once in WSL. Linux on a normal laptop or computer. Why not check FreeBSD? Here are some arguments why it may be worth it this time :) The same thing happened to me when I upgraded vim ! I thought why the heck would postgres requires an upgrade if I was upgrading vim's minor version ! It took ages... EDIT: for me this is a minor issue. i'll continue using `brew` as it does the job quite well 100% unrelated but I would suggest installing `pipx` when you need python packages to use from the CLI/outside a python project environment. `pipx` installs each package in its own virtualenv, so it is independent of your system python. It is really simple to compile Postgres from source and it only takes a few minutes. You can have multiple versions running on a single machine and installed in your preferred directory. If you forget to include the extensions, you can add them all later. I’ve used pkgin[1] on my Mac. It works fairly well, and I appreciated how quick the installs were. Given that macOS is a BSD variant, and given Apple's billions, it's a bit disappointing that Apple doesn't directly support (both engineering wise and financially) a BSD package manager. I avoid it at all costs. Sometimes it takes more time to figure out how to install something because guides are quick to recommend brew, but there's usually an alternative I could not get Postgres installed through NixDarwin, so I installed with Postgres.app for MacOS. With it you can easily manage Postgres versions. Really you should switch to debian if you want stable packages. It's not for the latest hotness but it achieves the stability it advertises. You may have already tried it, but you can run `brew postgresql-upgrade-database` after postgres is upgraded to fix your local databases. Just disable the auto-updates. Also, I'm 100% certain all it did was update the formulae (similar to apt-get update, NOT apt-get upgrade). I've been using homebrew for years and this is the default behavior, so unless you purposefully set it up to upgrade everything this is PEBKAC. AFAIK the only time it upgrades everything is when you do a blanket `brew upgrade`. And if you're pinned to something like postgresql@10 it will stay at that major version. Why did you want to `brew install virtualenv`? That’s no longer the current way to work with python. Virtual environments are now in the standard library: Absolutely. Brew manages pyenv, pyenv manages Python versions, venv manages Python packages / projects. Has been very reliable for the last decade or so, with my only complaint being the need for special linker flags when installing a new Python (homebrew openssl path etc), esp if doing an `--enable-optimizations` build (which I definitely recommend in the long run). brew install pyenv
pyenv install 3.9.7
cd ~/myproject
pyenv local 3.9.7
python3.9 -m venv --upgrade-deps .venv
source ./.venv/bin/activate You're off to the races. (I think I got that --upgrade-deps flag right.) I can't recommend pyenv and pipx enough. I use brew to install pyenv and pipx, and there ends my interaction with brew and python. Pyenv builds all of my python versions. Pipx installs cli tools in their own env. Precommit is also great for much more than just pre-commit hooks. It can install tools used for hooks in their own cache, no need to manage venvs yourself. Pyenv will setup Python environments under your user and replace the system Python executable with a shim. Depending on your setup, it can create a lot of confusion (I have fixed a lot of colleague machines in the past year that were broken in one way or another). Doing a `python[V.v] -m venv …` and then sourcing `activate` is conceptually much simpler and doesn’t affect any other terminal sessions you may have open. Me 11 minutes from now: "why did you use venv? They replaced that 11 minutes ago with ..." No. If you’re not an experienced python programmer then I can see that your comment might seem like a witty contribution. But I was actually providing good advice. If you're not an experienced /anything/ then I can see how that comment might have seemed like a personal attack, but I was actually only providing an observation, from someone who's been around longer than "11 minutes", about python and the rate at which it and other similar entire systems and their library of best practices changes, dickface. IMO packages like virtualenv should be brought in by a local python that's installed, not brew. This si literally why docker exists. I would never install something on a system and then complain it is being updated from time to time. The system needs to stay safe and follow a path of updates in order to cover vulnerabilities and such. Put it in docker and version lock it. Docker performs pretty poorly on macOS. Why are you using virtualenv instead of `python3 -m venv`? Just wondering. Macports has served me well for the last 18 years. None, I just use the UNIX experience provided by macOS. Why Ruby tho? pacman Pacman is unparalleled but its bleeding edge approach to updates would cause the same issues here unless you specified a version number when you're installing or something You can install Gentoo in a prefix on OSX. I used to do that. User error. You should have known about pinning versions. If this is the reason you want to change package managers, you should consider this. Either that or maybe macOS isn't the OS for you. Try gentoo. If you want package managers, go Linux. If your only problem is you don't like it surprise-updating everything, maybe you should set up an automated patching schedule. My experience is that both (Macports, Homebrew) are comically unreliable and frustrating to use. That said, with my insistence to install under $HOME, Macports seems to work slightly better. But truly, I got the best results from downloading and compiling every single flipping dependency myself. > But truly, I got the best results from downloading and compiling every single flipping dependency myself. My experience has been that very, very rarely is any installation as straightforward as "curl && ./configure && make PREFIX=whatever install", partially due to everyone in the known universe thinking they need some awesome new build system, partially due to unknown dependencies and their versions, partially due to it usually needing some patches because they only build on the author's machine, and all of that is just noise when I just want to have something running The value of Homebrew to me are the constantly bumped formulae descriptors, definitely, 100% NOT the "brew" frontend for it. So, I've found a nice middle-ground by switching Homebrew into developer mode and neutering its bottle system and using "--build-from-source" This isn't necessarily to lobby you, as much as "for your consideration" and to raise awareness for others that one need not throw the baby out with the bathwater just because the brew frontend is user-hostle > My experience has been that very, very rarely is any installation as straightforward as "curl && ./configure && make PREFIX=whatever install", partially due to everyone in the known universe thinking they need some awesome new build system [...] Aye, This is particularly impacting when the build system itself needs to be eh built. > The value of Homebrew to me are the constantly bumped formulae descriptors, definitely, 100% NOT the "brew" frontend for it. [...] and using "--build-from-source" I hadn't thought about that and I suspect you are right. Although I lack recent exposure to it and I can certainly recall getting very annoyed by it, I have to say that my experience with Gentoo's Portage/emerge is by far the least unpleasant as dealing with package managers goes. Homebrew isn’t unreliable — I’ve used it for more than a decade with no problem. What’s an actual problem you have with it? Perhaps others here could help you. I'll repeat what others have said above, but perhaps with more emphasis on the layperson's perspective. I think I started using homebrew some 4+ years ago. I have no idea what a cask, tap, pour, or whatever is. At first things "just worked" and it was great. Then came the couple of afternoons where I had an important project to finish, and I had to upgrade X. I think once it was matplotlib (for which I had to upgrade pip? ... I can't remember) or something silly like that. Everything broke, I had to reinstall all my python packages (that's when I moved to pyenv), and I did not finish my work on time. There was probably an easier way, but I was (am) a newb and I didn't know how. I remember brew was one of the first command line utilities I ever used and thought, "woah, that was easy _and_ cool!", but I no longer think I am the target audience. Someone might say the onus is on me to learn more about the tools I use, and they are probably right. I need to know my .tmux.conf or .zshrc or whatever to get the most out of those, too, and sometimes I like to dive in. For the most part, I just want to Get On With My Work and be oblivious. If you could provide some guidance, I would genuinely be grateful. For a newbie like myself, when I see comments like "just set HOMEBREW_NO_AUTO_UPDATE=1" and then back and forth on why that does and doesn't work, I decide I don't have time for this today and cross my fingers I don't need to brew install anything in the near future. And if I do, I will probably avoid it (because it will take, I don't know, hours?). Thanks. I agree the onus shouldn't be on you to figure out how to do sophisticated things, and it should basically just work without breaking your system. However! You shouldn't be using your system package manager to install Python dependencies! As you know, since you moved to pyenv. But I don't think it's fair to blame homebrew for getting you into trouble when doing that; one also wouldn't use apt to manage python packages. In fact if you look at this thread, many supposed problems with homebrew seem to be in fact problems with python package management. It's the same with all languages isn't it -- ruby and rbenv and gem, node and nvm and npm, rust and cargo. As soon as you start actually installing 3rd-party dependencies, you're better off using something language-specific to install the interpreter / compiler / toolchain, better than using homebrew or apt etc. The cask project supports large binaries and projects whose licenses are a little more poisonous than your standard brew libs. tap as far as I can tell is basically the same as PPA extensions in linux, it lets you install stuff from external repo lists. PPAs were pretty standard fare when I used to use linux more often, maybe working with package managers in your distro of choice for a while will help. In practice you usually don't need to use either cask or tap 99% of the time. There is no help for problems such as how the main developers seriously thought it was fine to have a user account own a shared system directory for many years, until Apple finally just took control over that at the OS level and forced them to figure out some marginally less borken arrangement. Macports is a tougher sell for an average user today simply because the disparity in popularity means macports library is smaller and older. But macports is engineered more correctly. > until Apple finally just took control over that at the OS level and forced them to figure out some marginally less borken arrangement. This was my biggest complaint about Homebrew. Does it mean it was finally fixed? It seems to install into /opt/homebrew/ now, whereas previously it used /usr/local. That’s progress. I've never had a problem with Macports and am wondering exactly the same thing. "Comically unreliable"?!
So when you look at a Postgres formula in the Homebrew repos and see that it declares an ‘unversioned’ dependency on openldap, for example[1], that does not mean that the formula author has declared ‘any version is fine’, i.e., it does not mean that the dependency is unversioned. It just means that the dependency's version constraints are totally unspecified except implicitly as ‘the only version of that package which is simultaneously present in the glorious monorepo of build recipes’. antlr
atkmm
autoconf
cairomm
db
elasticsearch
erlang
gcc
ghc
glibmm
go
gradle
guile
hdf5
headers
helm
isl
libfuse
llvm
lua
mbedtls
mysql
node
numpy
openexr
openjdk
openssl
pangomm
php
proj
pyqt
pyside
python
qt
ruby
swig
synth
tbb
vtk
wxwidgets
wat. depends_on "curl" # curl >= 7.55 is required
There's no way out of this that doesn't involve breaking things for Python developers who are, whether out of shortsightedness or resigned pragmatism, relying on or working around the many, various bad practices that the Python packaging 'system' currently allows. - total lack of static metadata— you must download a source package and attempt to build it in order to determine what its actual dependencies are
- relatedly, package setup scripts are way too powerful and can do literally anything, which makes automating Python packaging a nightmare
- any possible vendorization is deeply limited by the fact that Python processes can't include multiple versions of a single library
- the behavior of all Python packaging tools relies on the implicit state of PyPI, forcing a high degree of non-determinism unless you go to great lengths to take snapshots of PyPI
- native dependencies (i.e., dependencies on C libraries) are either just totally undeclared, or packaged abuse setup.py in order to manually compile them in an ad-hoc, unmanaged fashion
The nix-build command creates a symlink called `result`, so I put `/path/to/my/git/repo/result/bin` in my $PATH. with nixpkgs;
buildEnv {
name = "my-macbook-programs";
paths = [ bash gimp /* etc */ ];
}
That will: $ niv update firefox -v94.0
export HOMEBREW_NO_AUTO_UPDATE=1
Running `nix-build` will update the result/bin symlink to point at the new applications. buildEnv {
name = "my-packages";
paths = [ foo bar baz ... ];
}
It's simple, although to get it right with some programs. The advantage is that with one click you can download it, put in your path, and it won't need special paths. - Use the workflow_dispatch type so it can be manually triggered. Or schedule if you want it to run periodically.
- Install dependencies like automake/cmake and libraries using brew (I cheat here a bit)
- Clone the repository
- Perform the steps for building a static build
- Use actions/upload-artifact@v2 to upload the executable.
To get the most out of Nix, you'll need to take some time to really learn it. For managing services like postgres, it's nicest to use Nix-Darwin (a module system for managing services and programs, including automatic startup via LaunchD) but if you want to always keep the same Postgres with Nix-Darwin, you'll have to learn how to pin packages. - Nix
- pkgsrc or MacPorts ‘just in case’ they're more convenient for some package for any reason
- Homebrew, exclusively to use for Casks (which are really a separate system from the rest of Homebrew)
in my zshnev so long ago that I forgot about it, and just enjoyed a great product from a great community. export HOMEBREW_NO_AUTO_UPDATE=1
https://docs.brew.sh/Manpage#HOMEBREW_NO_INSTALL_UPGRADE (that's a fake hyperlink cause they don't put named anchors on their <li> for some reason) HOMEBREW_NO_INSTALL_UPGRADE
If set, brew install will not automatically upgrade installed but outdated formulae
That said, MacPorts is often opinionated, e.g. they package certain versions of some software and not others. xcode-select --install
- Finally, no one is forced to update everything at all.
run `brew outdated` and pick whatever you want updated then `brew update foo`... - postgresql@10
- postgresql@12
- postgresql@9.4
- postgresql@9.6
- postgresql (points to latest)
- postgresql@11
- postgresql@13
- postgresql@9.5
I’ve never had any problem with homebrew in over a decade’s usage. But yes, don’t use it for python development tooling: use pyenv and python -m venv and pip for that. python -m venv /my/directory