macOS Containers v0.0.1
macoscontainers.orghttps://github.com/macOScontainers/homebrew-formula
"macOS native containers"
Cool, this sounds interesting.
"Disable System Identity Protection."
Eesh.
"System Integrity Protection (SIP)" seems to be the correct name here, for anyone as confused as me :)
(More details here: https://support.apple.com/en-us/102149 )
Original author here. Thanks for spotting the typo, fixed.
WRT security implications of disabling SIP - I don't think OS becomes any less vulnerable than usual Linux/Windows installation.
> I don't think OS becomes any less vulnerable than usual Linux/Windows installation.
is not a good enough argument.
For the story, SIP is Apple's "rootless". Effectively the OS runs with less privileges than root. Disabling SIP significantly increases the attack surface.
That being said, I'm grateful that someone decided to do something more native for containers in macOS.
I think it's an OK argument given that most people run (and have been running with no alternative until very recently) docker in such a way that there's a trivial privesc to root. In general it seems like docker users are, overall, willing to take that tradeoff.
How so? I use docker pretty frequently, but I’m sure that my user is part of the docker group before I do, so I don’t sudo anything.
Is there anything else I should be doing security wise?
I’ve been hearing podman is more secure, but I think it’s still containerd under the hood, so idk how true that is.
In general if you can `docker run` without sudo then that means you have a trivial privesc path since you can do `docker run` with the various flags that run it without any sandboxing, get a shell, and just ask to be let out of the namespace.
The way that podman and newer versions of docker get around this is using unprivileged user namespaces. Unprivileged user namespaces are not a free lunch - in fact, they're a bit of a security disaster in their own right.
In a typical installation, being in a docker group gives you access to a socket that controls docker daemon and that daemon runs as root. `sudo` is not important in this context.
Thankfully there is rootless mode for some time now: https://docs.docker.com/engine/security/rootless/.
Podman, too, can run in rootful and rootless mode. Rootless in podman still feels to me to be more like first class citizen, as opposed to docker case.
In both cases it's important to keep in mind in which mode you operate. Both from the perspective of security and day to day operations, as some aspects of behavior will differ between those modes.
This is not necessarily the case.
On Linux, more or less the entire permissions system makes no assumption about SIP existing (as it doesn't there), so other protections are relied upon to secure the system (such as SELinux, granular directory permissions, etc.).
On both Linux and Windows, TPM and secure boot provide similar protections to SIP on macOS, but are optional (it's encouraged more forcefully on Windows 11).
Removing SIP from a system that relies on it as a basis for platform security is different than using a system that wasn't relying on it in the first place.
What exactly attack vectors you think are possible against macOS without SIP but not possible against Linux?
I think the argument is that file permissions may not be applied as rigorously, with the assumption that SIP is in effect.
SIP won't save you from wrong file permissions.
And SIP doesn't defend you from editing files in /bin. They are guarded by the fact that root filesystem is mounted read-only.
With SIP enabled it’s not possible to load arbitrary kernel extensions, for one - they must be signed.
Entitlement stealing, for example.
macOS had granular directory permissions way back when it was NeXTSTEP, long before SIP was introduced. Where are you suggesting they disappeared to?
It still has them, of course, but the concern is that after ~8 years of SIP basically ~everywhere, platform security decisions have been made assuming it is present.
This concern is definitely not totally unfounded, back in 2019 Chrome shipped an update that rendered systems with SIP disabled unbootable: https://support.google.com/chrome/thread/15235262?hl=en
The "varsectomy" bug in Chrome isn't the example you think it is, because disabling SIP was not sufficient conditions for it. There were 3 other conditions that had to be met, the most notable of which is that "/" had to be writeable by tho logged-in user, which is not the default.
This is an example of defense-in-depth being present, and defense-in-depth still failing for some users who gave escalated permissions to some installers, allowing them to run roughshod over their filesystem permissions, leaving them vulnerable to a subsequent varsectomy. If one did the same thing to their Linux system, the same thing could happen.
https://arstechnica.com/information-technology/2019/09/no-it...
The two other conditions listed in the page you link basically amount to "the buggy code has to run". Related, I think SSV [1], introduced last year, would also have blocked this bug even with SIP disabled. But none of that invalidates concern that areas of the OS we don't know about might not have the level of defence in depth that we would like - it's not like Apple has never cut corners or shipped bugs to hit a date.
1: https://support.apple.com/en-gb/guide/security/secd698747c9/...
One can have vague, general concerns about any operating system if one lets their imagination run wild, though, and one doesn't care about the presence or absence of specific examples supporting the suggestion that the "permissions system...makes assumption[s]" about SIP existing.
I think we have a rather specific example of a Chrome bug hosing OS installs. Had SIP not existed, there's virtually zero chance that bug wouldn't have been caught before shipping.
Here's a better one: https://gist.github.com/ChiChou/e3a50f00853b2fbfb1debad46e50...
I'd like to digress to your "Encouraged more forcefully" phrasing which is quite interesting if you think about it. In my view, it would mean something like pervasive dialog box ala EULA, some UI hoops you need to resolve, alike going with local account on installation.
In reality they done basically everything to force users to use secure boot. If they disabled normal boot altogether, OS adoption would suffer heavily. They could've obscured that option, but it would be found out, and enterprise users would be pissed at them because they didn't gave them a provisionable way while the way exists. So it came down to normal variables in installer registry.
However modifying, e.g making users "hack" the ISO is really as forceful as it gets without market loss.
Note: There may be more normal way today than modifying the registry of ISO, I installed 11 once when it came out.
If you really want good adoption, you’ll have to figure out a way for devs to try it out without first having to disable SIP.
Is this related to the code you tried to have merged here: https://github.com/containerd/containerd/pull/8789 ?
Nope, that PR was an attempt to upstream my changes: https://github.com/macOScontainers/containerd/commits/macos
Vanilla containerd cannot mount anything on macos.
> If you really want good adoption, you’ll have to figure out a way for devs to try it out without first having to disable SIP.
I can't stress enough how I also would like it to work with SIP enabled!
Sounds like if Apple wants developers who wants to use containers natively, they need to address the problem of not being able to offer this feature without disabling SIP.
No matter what you or I think about what's needed for adoption, technical problems get in the way of the tool working with SIP, so seems it's in Apple's ball court really.
In corporate managed laptops it may not be an option to disable SIP.
Understatement of the year. I am sure there are some places where being caught doing something like that (without authorization) could result in one of those “my hands are tied, I have to fire you” situations.
Think places where security is a big deal, like finance, military, aerospace, critical infrastructure etc.
Docker Desktop doesn’t require me to disable SIP. Why would I use this if it requires that?
To containerize darwin binaries?
This is not an alternative to remotely connecting to a VM to control Linux containers (which DfM is)
And what is the benefit of that? Who would use that and for what? Containers solved deployment, and software distribution problems related to diversity of runtime environments on linux. It also has some security benefits, but their adoption was successful because it solved real world everyday problems. It eventually allowed better utilization of servers.
They did not have such success on windows, despite Windows also having a container subsystem, as windows servers already did this with IIS web app containerization.
On MacOS desktop software distribution is largely a solved problem since ages. On MacOS/Darwin servers... are there such in industrial use apart from some research installations?
> Who would use that and for what?
I believe this project can be useful for CI and testing scenarios.
> On MacOS desktop software distribution is largely a solved problem since ages
Are you talking about App Store? Or Homebrew? Or MacPorts? Or... Wait, isn't this too many tools for a problem that was solved?
So why don’t they run a Darwin VM that doesn’t need SIP and control it in the same manner?
Docker For Max marshals filesystem events over a VM host/guest boundary which can grind the most powerful computers to a halt if you’re sharing directories between the host and guest. For example, at my last company we developed Python apps and ran them in Docker for Mac containers by mounting the source code directory into the containers (so we wouldn’t need a build step) but as our project grew the filesystem event marshaling became exponentially slower until we eschewed Docker from our dev iteration loop entirely (the fidelity benefits weren’t worth the performance hit). Note: there are lots of projects and hacks that claim to solve this problem but none made an ounce of difference.
The alternative is to use a performant file sharing system through the VM isolation. Some people use NFS, I personally use VMware Fusion + vmhgfs.
Sure it does not attempt to map fsevents<->inotify 1:1 but honestly I can live with that limitation given that it's a 10x performance increase compared to the DfM kitchensink.
I do go native darwin when I can / it makes sense.
Is that something you can configure in DfM or are you saying you sort of roll your own DfM alternative? Also, what does "VM isolation" mean here? How is NFS or VMware Fusion more "isolated" than DfM?
> Is that something you can configure in DfM
No
> or are you saying you sort of roll your own DfM alternative?
Yes, I set up a NixOS VM and use DOCKER_HOST=ssh://docker@<ip>
If you want to have it easy you can roll with lima/colima (but I found the fs sharing slower than vmhgfs)
> Also, what does "VM isolation" mean here?
The host/guest boundary. The guest is just like any another, remote machine. DsM adds smoke and mirrors to make it look like guest and host are one.
> How is NFS or VMware Fusion more "isolated" than DfM?
It's not, both are VMs and need a way to expose the host fs to the guest, NFS or vmhgfs are a means to that.
Understood. Thanks for the explanation/clarification!
> until we eschewed Docker from our dev iteration loop entirely
Can you say anything more about what you did instead, and how it ended up working out?
We used native MacOS processes and it worked out fine. In practice I don't recall any bugs due to different behavior between our dev environment (MacOS) and our production environment (Linux) and we saved a ton of time during development. Docker just wasn't worth it. This worked for us because our app didn't depend on any platform specific behavior.
Of course, if you are using Go or some other language with a fast, static cross-compilation step, you don't need to mount a source code volume into your container, you can just rebuild the whole container image or rebuild on the host and `docker cp` the new binary onto the target container.
Docker Desktop for macOS does full-blown hardware virtualization, which is just silly.
This should have faster file access than Docker Desktop. For large projects, disabling SIP may be worth the increase in performance.
Docker Desktop runs an entire second kernel in a VM.
And that’s a good thing? I’m not sure the benefits of containers would be worth disabling SIP.
Maybe for an automated test CI/CD running system (which is probably the main use-case), but not on anything that users would interact with.
> And that’s a good thing?
That's a technical limitation.
Technical limitations aren’t excuses for a bad design. If it’s not a good design due to a technical limit, the answer isn’t to sacrifice security for functionality.
If it really isn’t technically possible (which I think you might be able to do in a Darwin VM), then maybe this approach isn’t a good idea.
> Technical limitations aren’t excuses for a bad design.
Tomorrow Apple might decide it is safe to chroot with SIP enabled (I actually do not understand why they restrict it, chroot is a tool to increase security). Does that suddenly convert bad design into a good design? But this is exactly the same design.
Design operates within the contexts of a system. A perfectly viable design can be made bad by a change in the system for which it was designed.
Does this current design require disabling SIP? Then I don’t think it is worth my effort to use (for my use case). If Apple changes the system in the future, my opinion might change.
But a design cannot be judged as good or bad outside of the context for which it was designed.
If you feel this is a better way to tackle the problem, then talk to Apple about it.
> I don't think OS becomes any less vulnerable than usual Linux/Windows installation.
A modern Linux with SELinux enabled (the default in e.g. Fedora) running apps inside rootless containers (Podman doesn’t even need a daemon) is likely much more secure than your default MacOS or Windows.
If you're talking about desktop use, not really. All the software in a default Fedora desktop install, runs as unconfined by SELinux policies.
Where SELinux really shines is in server installs.
I think you're probably @totallywrong
TBH, Fedora Silverblue with the parent commenter's approach it's far more secure.
Well Linux powers just about all the most important systems in the world and SELinux was originally developed by the NSA, but I'd love to get your insights.
No, that's not true. Turning off SIP is usually about equivalent to allowing instant escalation to root privileges; sometimes even worse.
Can you please expand on this? It wasn't my understanding at all.
A decade ago SIP didn't exist at all. I thought disabling SIP just put us back to how things were in Yosemite. What changed?
Apple relied on it being turned on and started introducing permission checks where you'd have things like "do thing as root" and check for some flag being set that is protected by SIP. If you have it turned on, there are no issues, because the check blocks both root and non-root users! But if you turn it off, now the "do thing as root" is available to any account because the flag is editable.
I'm more interested in this as part of a CI/CD process for releasing macOS software. For an isolated build worker - yeah, sure, I'll go ahead and disable SIP.
Not every workload is running on an endpoint connected to a human via keyboard and screen.
>For an isolated build worker - yeah, sure, I'll go ahead and disable SIP.
Isn't this especially dangerous on a build worker? All your source code goes in and you (presumably) use the binaries that come out across the rest of your infrastructure. Compromising a build worker in a persistent fashion due to lack of SIP seems like it could do some serious[1] harm...
Depends on your threat modeling. Are you running untrusted code on the worker (maybe you have an org with thousands of engineers, maybe you're running builds from a public fork)? Sure, that's an issue. Are you a small startup? Take the convenience and focus on the bigger fish to fry.
That's what it takes to implement container-ish technology on macOS. Hopefully one day Darwin will get real namespaces as Linux does.
In the context of an internal CI or something, it's probably fine.
Given that the vast majority of CI runs on linux without any equivalent of SIP, I'd have thought it were a complete non-issue
You might want to run CI/CD on your Mac builds
"v0.0.1"
There's probably a hundred things that are not right just yet and they know it, let's not be overly negative
(although the discussion on what it is and what it does definitely is interesting)
OrbStack doesn’t require breaking security: https://orbstack.dev/
Does OrbStack allow you to create MacOS Containers? It doesn't seem like it from their website. It seems like it just runs on MacOS to let you create Linux containers.
OrbStack is about Linux containers. What we're talking about in this topic is the only software in the world for macOS containers.
Then I wonder why the parent comment mentioned OrbStack as if it were somehow comparable or an alternative to this.
> "Disable System Identity Protection."
This is the first thing I do on any Mac OS system before I start using it.
This kind of makes me wonder why you'd have a Mac at all (I'm sure there are use cases).
Wouldn't a Linux device, or Linux running on a Mac suit you better?
For me, the security picture is one of the main features of the eco-system even if it's very restrictive - disabling SIP undermines it more or less completely.
>Wouldn't a Linux device, or Linux running on a Mac suit you better?
Maybe they want a unixy desktop with working sound ?
Half joking, but that's my use case - homebrew is pretty great, most developers use a Mac in my domains of interest so it's always supported.
Linux is just too much work (and I'm using Fedora on my desktop). SIP is just false positives and annoyance.
I'm on the fence about M/ARM switch since I still see a lot of friction with containers so I might be looking at framework for my next device. Or just go all in on client/server development model.
> homebrew is pretty great
I hear this said a lot in passing, and I'm really curious what people mean when they say this.
I can use it to get stuff I need to do work and I don't remember the last time I couldn't. Upgrades sometimes leave me broken but it's usually ironed out fast because everyone is using it. If my dependencies are that locked down I'm using docker and special environments anyway. Linux can get tedious with upgrades breaking or dependency mismatches. Homebrew is probably the largest homogenous community - if you're doing something relatively popular there will be a lot of noise when stuff breaks. Linux is spread out across various distros/repos/package managers, overlap of users with your problem is a lot smaller.
Beats all the package management experiences I've had on Windows, admittedly I have not tried to use Windows for work for >1 year.
Homebrew is available for Linux as well since approx 3yrs now. I've been using it without issues https://docs.brew.sh/Homebrew-on-Linux
OK but what I like about homebrew on mac is that when I'm having an issue with "popular stack X broke after updating" it's probably me and >10k other people out there, so by the time I hit the problem it's already under investigation on GH. I'm not sure the same would apply to homebrew on Linux - even if you ignore the differences between distros - how popular is homebrew on linux and linux desktop in comparison ?
I wish more of those 10k people would help get others off of a package manager that is so fragile and convoluted that updating so often leads to popular things breaking.
Things like macports and pkgsrc do things in an arguably much simpler, more unixy way, without the contortions that so often seem to leave homebrew in a bind after routine operations like updating.
I’ve never experienced a broken homebrew and I’ve used Mac for years
The comment was in response to parent's stated complaint, namely having to wait for someone else to resolve issues with popular packages being broken after an update, which has been the experience of more than one user.
Neither have I. My main complaint is that it's slow.
I’m curious what’s the benefit? I use homebrew as a Linux package manager for MacOS. On Linux I just use the distros package manager
If you need to build something from source (my use-case: Vim, so I can change which language bindings exist in the resulting build) it can sometimes be a lot easier than cloning and using the "raw" C/Make build system.
Also, assuming a downstream distro like Debian or Ubuntu, what's in Homebrew is likely a more up to date package. You could fiddle with adding/using Debian testing or some PPA, or... you could just use Homebrew.
(FWIW: I use Arch and the AUR on my desktop Linux installs these days, and it's essentially the same process. But still using Homebrew on the Mac, and occasionally in Linux when I'm not on a desktop)
I mean, it installs stuff more or less reliably for me so I’d say that’s pretty great. I’m sure it has issues (I’ve had problems trying to install old versions of packages for example) but I don’t think I’ve ever had “brew install <whatever>” fail to install what I wanted
The experience is really seamless, mostly it just works… which IMO is nothing special as they mostly support installing the latest and maybe a couple of other versions, and software that requires weird stuff will likely be packaged in a self contained way.
They mean they haven’t tried another package manager for Mac, like pkgsrc or macports.
I switched to it after using both Fink and macports and it was the first time I felt like I had a package manager as reliable to apt on Linux. Both Macports and Fink would routinely break my whole system by putting extra versions of system libraries where other applications could find them. Since, I’ve switched to nix.
Perhaps they are talking about Homebrew the package manager [1].
[1] https://brew.sh
I wonder what people mean when they say "Homebrew is pretty great", I do not wonder what they mean when they say "Homebrew".
They mean that they have an overall positive opinion of Homebrew, more than “not bad” but less than “amazing”.
As someone who would readily say that Homebrew is pretty great, this is exactly how I feel about Homebrew.
For example people do not use homebrew to install python3 on a fresh install of mac os and get a python that spews a bunch of messages to stderr about the version of libc6.so and cannot speak TLS.
It only upgrades PostgreSQL 3 major versions up alongside without asking.
Think this issue was fixed a few years ago
Not sure I agree, when I use brew it still regularly updates random unconnected crap through multiple major releases. And of course it takes ages doing so
I was referring to your postgresql example, those became rudimentary @versioned packages
It does what it says on the tin and installs the plethora of software available on homebrew with a simple command, which is pretty great
It's plenty solid, and there's a good library of packages on there. It's a package manager, doesn't take more than that.
I use AMD64 containers on my M1 Pro under Docker Desktop with zero problems. It’s about 10-20% slower than my AMD64 Linux machine on average, which is usually fast enough.
You ever give OrbStack a go? I used Docker Desktop before it but find OrbStack to be a lot faster and native feeling.
No, I’ll give it a look, thanks for the tip.
This is a fair point. For folks who want a “just works” Unix system, disabling SIP probably makes sense. But most it’s still very niche.
Is there a way to make a separate partition of MacOS and have one copy with SIP and one without?
> But most it’s still very niche.
I'd say that the whole containerization topic is niche
> Is there a way to make a separate partition of MacOS and have one copy with SIP and one without?
I think you can install macOS VM on your macOS host and disable SIP _inside VM_.
depending of what you mean by friction with containers, there may be different solutins. for example, as a temporary solution on Docker when you don't want/have time to support both arm and x86, you can do something like this: FROM --platform=linux/x86_64 ubuntu:23.04
i haven't found any issues with it that i could not get over in the past 2+ years of m1. most of the containers are available on both architectures anyway. the performance improvement was totally worth it, i won't even talk about the heating issue with intel.
> i haven't found any issues with it that i could not get over in the past 2+ years of m1.
I'm currently running a Journal of Open Source Software x86 container on aarch64 and it's terribly slow. Takes 12GB of RAM and 3 minutes to build a LaTeX document, see https://github.com/openjournals/inara/issues/30. Any tips?
Building a single-page LaTeX document as a PDF can be done in less than 0.3s when the environment is set up correctly. I don't know anything about this particular container.
> I'm currently running a Journal of Open Source Software x86 container on aarch64 and it's terribly slow
There's your issue. Use an aarch64 image.
> Maybe they want a unixy desktop with working sound ?
In my experience, this has not been an issue for the past 10-15 years atleast. Before that there were some problems with few (external) soundcards or random cpu spikes with the mixers.
However, the UX can still improve. Switching audio outputs with multiple outputs like external displays etc is not very smooth or intuitive.
Some bluetooth headsets have issues but I've had those with a mac as well.
I'm running Linux everyday and I really wouldn't recommend it for any serious audio stuff.
There is great audio software coming to Linux (Bitwig, Reaper, etc) which is great but the underlying infrastructure is a mess.
There are like 3-4 audio subsystems running, I never know which one is it, setting latency is wizardry and sometimes it doesn't run at all. It's usually fine when I run stuff like Spotify, VLC, or Youtube in Firefox, so for user-level audio, Linux is fine IMO. But when I run something where I care about latency and multichannel output, it's hit or miss. It runs fine one day and then I get no sound on another or distorted sound or sound playing at wrong speed and wrong pitch (yay, 44,1 vs 48).
Maybe it's the distros I'm using, maybe there are some that work better, but the UX isn't as great as with macOS. On Manjaro, update sometimes get audio notification removed from tray and I can't change volume using mouse or dedicated keys. Then I have to look for few hours for a solution only to have the same thing happen again three months later (same with brightness keys on laptop). On Ubuntu Studio with an external soundcard, I get randomly distorted sound or no sound at all. So it's easier to use some shitty onboard sound, great.
I like Linux, I use Linux daily, but sound on Linux is terrible. It's much better than it was, yes, but still terrible. For anything more than "play a song here", macOS is much better.
Fedora Silverblue with Pipewire will just use a single subsystem and that's it. Inmutable OS and the rest of software it's Flatpak. The issues are gone. Oh, you need a proper devel environment with dnf/rpm? Just use "toolbox enter" and install all the complex envs under that container.
It sounds like your specifically fighting your distros problem, or something that its not your understanding.
I would not consider Linux sound Terrible, but to be fair, I only use it every day for regular development tasks for the last 10 years. Maybe I've become accustomed to whatever problem you see that I don't.
Anecdotal evidence, but here it goes.
I moved from Linux to M1 MacBook recently. I know my greps and vims, but I was tired of audio glitches during high CPU usage, system not waking up from sleep, total OS freezes, super loud fans, and so on.
Now I get none of that. I don't think I've ever heard the fans. Audio just works, everything is super snappy. It always wakes up. I'm no longer afraid of bluetooth.
And on top of that, setting my $DAYJOB VPN took three minutes and it just works, where on Linux I had constant problems with DNS breaking, and setting it up was always an hour of work, praying I got the config files right this time.
It really seems to be "unixy desktop with working sound", the best of both worlds.
Exactly my experience. After 15 years, I became an apple fanboy in 15 days. I still do hate losing my muscle memory on some bash shortcuts, but I'd say it was very much worth it.
Homebrew has GNU utils to smooth over the differences in package options.
[0] https://formulae.brew.sh/formula/coreutils [1] https://apple.stackexchange.com/questions/69223/how-to-replace-mac-os-x-utilities-with-gnu-core-utilitiesYou can just install a recent bash with homebrew and use that as your terminal.
Karabiner and applescripts can get you pretty far
There's only a 1% chance these days on Linux that your sound won't work or your computer won't sleep when you close the lid or your wifi won't work, or your ethernet, or your cooling, or a peripheral, or CPU/memory spikes.
And a 90% chance it'll be at least one such thing.
All of those things work fine on every computer in my household that runs Linux. This spans thinkpad, dell and ASUS laptops, Dell desktops, home-built gamer type desktops, a few raspberry pi's, and a SFF PC we use to run Kodi on the main TV.
I do find it amusing in a thread about how you have to turn off a core security feature to be able to use containers properly on a Mac that the discussion immediately turns to how bad Linux sound drivers supposedly are. Honestly, I went in the other direction (Mac to Linux) and I've found the waters to be just fine. I don't know if I just have the magic touch or something, but ¯\_(ツ)_/¯.
Haha now that you mention it touch (trackpad) absolutely sucks on Linux when I last tried it - I couldn't even get palm rejection to work.
But this isn't bashing on linux desktop (I would use it if Mac wasn't an option) as much as giving a reason why people would use MacOS despite being annoyed by SIP.
Tell us what distro you recommend?
The sound is an internal meme where the linux devops would regularly have to drop off calls to talk (restart to unmute :)).
On my desktop I couldn't even boot installer without running with safe mode, otherwise I'd just get stuck on a blank screen (ancient 1050 TI GPU and standard desktop components otherwise, so not exotic/new stuff).
I've used linux desktop for >decade and Gnome shell feels like home but these days I feel like I don't have the time for linux adventures. Maybe I'll mix it up with my next device, but I'm not reading great things about AMD power modes and Linux.
> The sound is an internal meme where the linux devops would regularly have to drop off calls to talk (restart to unmute :)).
Hmmm once every full moon MS Teams running on ungoogled chrome do not seem to realize my Bose BT Headset is paired and available (and in that case I just use the internal soundcard) but I have seen people having sound issues on MS teams and needing to reboot regardless of the OS they were using. Windows, Linux, even some MacOS users so I wouldn't use that as a generalization.
> In my experience, this has not been an issue for the past 10-15 years atleast. Before that there were some problems with few (external) soundcards or random cpu spikes with the mixers.
May be some confusion. To run linux on a newer Mac with "Apple Silicon" (ARM based), you need to go through a lot of hoops and much work needs to be done still for a stable environment. Check out https://asahilinux.org/about/
Or maybe you thought they meant running linux in general on a PC (Intel x86 32/64 bit)? In that case I agree - driver issues like that have been mostly ironed out by now.
I'm getting early PulseAudio vibes with PipeWire though. BT audio devices stuttering and sometimes losing audio completely regardless of output devices unless I restart the daemon. I guess it stabilizes again during next few years.
I would think most people choose macOS over Linux for the familiarity with the UI/UX and software availability (Adobe and MS stuff).
I would expect a very small number people making this choice over security concerns.
Windows is constantly forces people with popups to makes scary security decisions and if they press no, the world stops working
> Wouldn't a Linux device, or Linux running on a Mac suit you better?
No? Maybe you're preferring Mac OS for getting stuff done, exchange work with the outside world and/or use non-historic software (like any commercial desktop app such as idk Photoshop, Sketch, Audio, 3D, CAD s/w, etc., etc.) and still are a developer?
Or even doing something esoteric such as using office software without wanting to throw your notebook out of the window?
Only way to properly develop iOS apps, I'm afraid.
Connecting to a Mac Agent with Visual Studio on Windows gave me nothing but headaches.
I'm not your parent post but surely the easy answer to why use a mac at all is simply how great the hardware is. The 2020 M1 Air is nearly 3 years old and still a brilliant machine.
No, the hardware drivers are very polished and never give any problems. And the hardware itself is beautiful but I suspect wouldn't integrate as well with the OS if it were running Linux.
In my opinion, MacOS is just a nicer experience on the whole than (Ubuntu or Red Hat) Linux (I have limited experience with other distros, but they are all pretty comparable to the big distros at best.
I only ever really had one goofy driver/deep OS bug in Mac - something with the location daemon would cause the wireless internet connection to cut out repeatedly. That bug was left behind with that machine when I left that company, and didn't appear in my next macbook pro.
Linux is just always a struggle with drivers, subtle bugs, and other misc friction. It's not a dealbreaker - ubuntu 22.04 is still my daily driver, but it's very much enough that I would prefer a mac for most development.
For example, if I run a software update, it quietly breaks the fn keys to change screen brightness, and when the machine wakes from sleep, the screen stays black. I figured out after much trial that running ubuntu-drivers fixes it, but it's a pain. I'd rather just turn off auto-updates.
Also the Command key for keyboard shortcuts is brilliant and just works across the whole system. On linux I have to use ctrl-shift to copy/paste and I haven't found a good workaround yet.
Work gives me a laptop, I use it. I'd prefer Linux, but that's not what I get.
I ran desktop Linux (Manjaro) for years in an enterprise job with certified hardware, as a Linux sysadmin. I regularly had issues and spent 5-10% of my time troubleshooting the OS or a janky application.
Conversely, macOS is broadly 'production grade'. It mostly 'just works' (with a number of tweaks - including SIP -, hacks etc on initial config, most power users automate with dotfiles). It has a drastically better UI, first class terminals and unixy support, and most code built for it has a higher level of shine.
I am confident enough to deploy alternate security implementations for the convenience of full FS control, as I know many power users are. Disabling SIP is a bad idea for those who don't understand it, the same as disabling Windows Defender or forwarding NAT on your router.
SIP on consumer laptops/desktops is security theater. It only really makes sense on web-servers. For people for whom this is a deal breaker, can you describe a concrete scenario where SIP would actually protect you?
SIP protects you from an app accidentally rendering your system unbootable, for example.
> why you'd have a Mac at all
My work provides it. Everyone else uses it and I don't want to be the one with a different setup.
The hardware is what's good in Macs, with Apple Silicon, not the software, which I use begrudgingly.
So I take it you don't use Apple Pay or anything...?
Edit: to be clear for the people who may not know, Apple Pay does not work with SIP disabled. ;P
Is Apple Pay that popular?
On mobile absolutely [1]. Not sure on the computer, but I use it all the time.
[1] https://capitaloneshopping.com/research/apple-pay-statistics...
Apple Pay rocks, it has saved me a few times when I forgot my wallet. And it’s faster to use than pulling out my wallet.
I don't know whether it is popular everywhere, but it is certainly popular everywhere I go (anywhere in the US, and western Europe). I absolutely love it.
I use it whenever I can, whether on laptop or mobile.
No, I don't use apple pay. I don't have any apple device for personal use at all, and I wouldn't use my work MacBook for personal payments.
You can forward Apple Pay requests to your other devices.
Could you elaborate why? I’m new to macOS, I’d love to learn more about it.
SIP is a feature that protects you from malicious actors with root (admin) access on your device. After they've encrypted your photos and drives and changed your passwords, it prevents them from making your machine unbootable by deleting or altering system binaries. As a side effect of this protection, you give up certain freedoms to customize your system.
https://github.com/koekeishiya/yabai
For instance requires SIP to be disabled.
Don't disable security features because of random comments from randos.
Also, don't install shit anywhere but your home directory as you unless you want to break your system in an irreparably, unmaintainable, or unsupported manner.
If you're using sudo, you're already doing it wrong.
> Also, don't install shit anywhere but your home directory as you unless you want to break your system in an irreparably, unmaintainable, or unsupported manner.
I guess you don't use Homebrew or MacPorts?
Civilized people don’t install things in “/“.
I'm not sure if you were arguing against the parent comment or supporting it but did you actually mean "/" and not "~/"?
Reads like support, and I'm 99% sure they mean "/"
Bizarre
... the second thing being scanning for Malware, presumably
Then you don't know how to use it.
Oh wow… please do not do that folks!
I disable SIP because I don't want to use an OS where I cannot debug programs which did not consent to be debugged. macOS makes it impossible to inspect failure states I encounter in normal usage (like I can on Linux with debuginfod enabled, or Windows with .pdb files) without first rebuilding the program in debug mode.
I’ve never had problems debugging anything with SIP on.
My experience is that I was unable to debug Maestral (a Python program) crashing in native Cocoa bindings in LLDB, because the interpreter did not allow itself to be debugged. Mozilla says Firefox acts the same way at https://firefox-source-docs.mozilla.org/contributing/debuggi.... It's possible there's some workaround I'm not aware of, but I ended up disabling SIP just like every non-macOS computer I own.
Were you debugging a local install of python, or the default python installed as a part of the OS?
If the latter, it's definitely a footgun but you shouldn't be using it for applications. It's a legacy binary for scripts.
You can't debug system processes or anything with the hardened runtime enabled.
Surely you can do this with xattr?
Sincerely, why is this your recommendation? What does this protect against and why do you consider it a concern?
https://eclecticlight.co/2019/06/19/what-is-sip-and-when-is-...
It’s a core part of how macOS is prevents unauthorised modifications to the operating system and file system by malicious software.
Help doc: https://support.apple.com/en-us/102149#:~:text=System%20Inte....
TLDR: it restricts even the root user from modifying system files. Like the ones that would otherwise be the target of malware.
I understand that of course, I'm much more curious why you think this is a concern for people, and/or why you consider this an effective protection against said unknown actors.
I want that feature on cause I dont want stuff I dont know about looking and changing stuff I dont know about, whether it's running with escalated privileges or not
I doubt you (or any human) is capable of enumerating what you don't want looked at. Frankly, I doubt most of this unknown area is covered by SIP at all, and it would be extremely odd if it did. Perhaps you might consider arguing for actual permissions rather than arbitrarily walling off the OS in a way that tangentially benefits the monopoly Apple holds over their own computers.
Wouldn't it be far easier to enumerate what you want an app to access?
SIP means not messing with the system files, enumerated thusly: /System.
Enumerating what I do want an app to access is handled by Gatekeeper.
> SIP means not messing with the system files, enumerated thusly: /System.
...and all its children, which is effectively the entire operating system
> Enumerating what I do want an app to access is handled by Gatekeeper.
Gatekeeper is not capable of this.
It's among the things Gatekeeper does, isn't it? As configured with PPPC?
It's actually a larger list available in /System/Library/Sandbox/rootless.conf
Why would it be running with escalated privileges if you don't know what it is ?
I feel like you're assuming that applications have to be honest about what they are when they request a user-prompted permission. SIP makes that irrelevant.
No, I'm assuming that you know what you install and that apps run with the same rights your user has. Your user can't touch /System, so shouldn't the app
I always disable SIP on every Mac I own. It's not very useful and I can run whatever I want on my hardware.
I hope that's sarcasm.
https://support.apple.com/en-us/102149 https://support.apple.com/guide/security/system-integrity-pr...
Why would it be sarcasm? If I am root and want to write to /bin or /System, I should be able to, even if it might be dangerous. I get why Apple does it for normal consumers but for people who know what they're doing, it should be available, which it is by disabling it.
I'm sure they'll overcome this! Rootless containers are a thing after all!
> Disable System Identity [sic] Protection.
Aaand, it's stillborn. Not happening.
Hopefully if Darwin containers take off and produce an ecosystem of containers this could pressure Apple to implement process namespacing in Darwin in a SIP compatible way.
Denial is not a river in Egypt.
Wouldn't recommend this to people prone to identity crises
How does this work?
Fundamentally, containers are about namespace/isolation of a bunch of OS interfaces, so file system functions, network functions, memory management, process functions, etc, can all pretend like they're the only game in town, but crucially without having to virtualize out the kernel.
Does XNU have such namespacing functionality across all its interfaces?
Furthermore, the existing container ecosystem assumes a Linux syscall interface. [1]. Does macOS provide that? I expect not.
The way Docker Desktop (and podman.io) implement "containers on macOS" is a bit of a cop-out: they actually run a Linux virtual machine (using Hypervisor.framework/hvf), and have that just provide the container environment.
Is that what this project is doing? But then, how could it run a macOS container?
[1] based on the foundation that Linux, unlike BSDs, has a stable syscall interface!
This introduces a long-standing problem that is really the main issue I have with Docker/Podman/containers on anything other than Linux - you have a dedicated VM that needs memory to be set aside for it exclusively.
This is fine if you have a 32/64GB machine, but less so on an 8GB non-upgradeable laptop.
I get it - memory is relatively cheap these days - and manufacturers that are building memory-limited devices are really only doing it to fleece you on obscene upgrade fees at the time of purchase - but it would be nice if there was a more elegant solution to this on Windows and macOS.
WSL 1 had a solution to this that clearly took a lot of work to put together, wherein they'd have a Linux kernel running side-by-side as a Windows sub-process so that the memory pool was shared. Unfortunately it might have been too much work as they scrapped it entirely for WSL 2 and just used essentially the same VM route.
If anyone knows of any projects trying to work around that problem I'd love to hear about it. If Apple really wanted to bring the development community back on board, focusing on these kind of use cases would be great, sadly it seems someone over there has taken the view that scrapping butterfly keys and the touchbar is "enough".
Say what you will about Microsoft, but they've focused really hard on developer use cases for decades, and it shows.
> If anyone knows of any projects trying to work around that problem I'd love to hear about it.
Containers are namespaced processes. These processes exec against the corresponding kernel they require. There is no workaround: if you have an ELF binary calling Linux syscalls it can only run on a Linux kernel†, so to run that you need a VM††. It's not as bad as it appears thanks to memory ballooning†††.
Conversely if you want to exec a Windows binary in a container, the Windows kernel needs to provide process namespacing features (which it does). And if you want to exec a Darwin binary in a container, then the Darwin kernel needs to provide process namespacing features (which it doesn't).
† WSL1 was implementing the Linux syscall API on the Windows kernel, which proved to be much more complex than it appears to be.
†† Or colinux (https://en.wikipedia.org/wiki/Cooperative_Linux), or user-mode Linux (https://en.wikipedia.org/wiki/User-mode_Linux).
The challenge of running OCI containers in every OS is a bit similar to the problem of running the same binary in every OS.
I think the only true solutions are (a) OS vendors develop their own native container platforms with UX similar or better than Docker (b) OS vendors agree on some common ABI standard
I suppose running a windows binary under wine in a Linux container also belong somewhere on this excellent summary.
> running a x86 windows binary - looking for 32bit stuff under SysWOW64 - under x86_64 wine in a x86_64 Linux container AOT translated to aarch64 by Rosetta 2 for Linux that switched the ARM M1 memory model to Total Store Ordering on a aarch64 Linux kernel under Virtualisation.framework on arm64 darwin.
FTFY ;) (and thanks!)
> WSL1 was implementing the Linux syscall API on the Windows kernel, which proved to be much more complex than it appears to be.
I've long wondered, and again now that Chat GPT is proving so adept at coding, if this translation layer could be automated. Do humans actually have to hand-code each syscall? Or are there just enough edge conditions that can't be automated?
The hard part isn't so much writing the code as it is testing the code. The WINE server is decades old, porting API calls to another kernel isn't an entirely new or novel process. Getting all the software to work right is an ongoing process though, and it requires careful deliberation over what works and what doesn't. WINE code is still being refined to this day in response to new titles and old bugfixes.
So, I won't preclude the idea of AI helping, but I think human effort is still the bottleneck for projects like this. Even if AI could write perfect code 100% of the time, testing and troubleshooting would probably still be the larger timesink.
WSL 2 now allows for memory reclaim from the VM, essentially fixing this
https://devblogs.microsoft.com/commandline/windows-subsystem...autoMemoryReclaim – Makes the WSL VM shrink in memory as you use it by reclaiming cached memoryI think someone told me that the relies on a feature of the CPU itself and MS is the only OS vendor to actually implement it
This is supposed to be possible on macOS, but it's basically completely broken (just doesn't work) due to a bug. Apple is aware of it but unfortunately hasn't said much more.
Obviously that limits the options, but I'll still be taking one last shot at using creative workarounds to tackle the memory problem in OrbStack (another containers-on-macOS product).
VM dynamic allocation of ram has been a standard feature of KVM, hyper-v and VMware for the better part of 10 years now.
Recent versions of Docker Desktop have a "Resource saver"[0] mode. It will de-allocate memory and CPU when containers aren't being used. If there's a leaf icon next to your Docker icon then it's in resource saver mode.
I imagine over time it will get smarter too. Right now it waits for no containers to be running for 30 seconds and enables resource saving mode but who knows what could happen in the future. Maybe it can internally profile and estimate load based on evaluating runtime stats of your contains and dynamically change the VM's resources on the fly and then expose a +% over provision threshold option or a way to turn off dynamic resource saver mode.
One of my issues too. The tools/ecosystem are Linux driven and doing anything on Windows requires a Linux 'stub' of some sort (VM, WSL, etc). I am comfortable with both OS's but all of my coworkers are Microsoft OS tied. 0% Linux experience. Cannot really introduce a managed kub/docker/containers into the picture without ending up being 'support' - not happening! The cloud is an option but that still leaves the developer experience (local laptop) short changed.
Edit: It has been a while since I last looked at this. Looks like containerd is, perhaps, a native option
WSL1 didn't run Linux kernel anywhere.
WSL2 doesn't really use VMs in the traditional sense. msft have invested heavily in lightweight virtualisation (eg better memory management) for certain security features, which allowed WSL to perform well enough without the maintenance/support overhead of WSL1
However WSL for Docker works differently depending on the containers configuration.
When Windows containers are being used, it is just there to keep the docker daemon happy.
The transitively linked docs have some answers:
rund is an experimental containerd shim for running macOS containers on macOS. rund doesn’t offer the usual level of container isolation that is achievable on other OSes due to limited macOS kernel API. What rund provides: - Filesystem isolation via chroot(2) - Cleanup of container processes using process group - OCI Runtime Specification compatibility (to the extent it is possible on macOS) - Host-network mode only - bind mountsSo essentially a chroot with a bit of make-up and a lot of marketing?
Except for bind mounts (not even overlayfs...) there isn't much interesting.
> - Host-network mode only
Yeah expect a lot of things to break in subtle ways... most containers are developed kinda expecting you have your own network namespace (and that no one else is using ports)
Original author here.
> So essentially a chroot with a bit of make-up
Well.
1. It is not trivial to properly set up a chroot on macOS. If you try to find a working guide/tool that works with modern macOS, I doubt you'll find anything (at least, I failed, even though tried very hard) 2. I believe that ability to package stuff into a Docker image distributable via already existing infrastructure and compatible with already existing tools maybe "a bit of make-up", but it is an important makeup. 3. Kubernetes recently got HostProcesses for Windows: https://kubernetes.io/blog/2022/12/13/windows-host-process-c.... They are even less isolated from host than chroot and still, people find them useful for certain scenarios.
> and a lot of marketing?
Thanks for "a lot of marketing", that made me chuckle. My own submission got buried yesterday with humble 8 points: https://news.ycombinator.com/item?id=37640688
Great effort. I get why you call it container - but sounds more like jail or cheroot would give more appropriate expectations; like "tooling to build and run Darwin containers in a macOS chroot"?
I didn't want to use "jail" term because it is mostly unheard of outside of FreeBSD.
Container definition is very stretched nowadays. Look at Windows HostProcesses in Kubernetes [1]. They don't have neither process, network nor device isolation from the host.
I also plan to try macOS sandbox-exec tool, which should offer additional isolation from the host.
[1]: https://kubernetes.io/blog/2022/12/13/windows-host-process-c...
If the parent process of the container here changes its bootstrap port to itself or disinherits it then it could also create an isolated mach namespace, restricting access to mach/XPC services.
overlayfs and bind mounts are orthogonal:
- bind mounting solves the exposition of filesystem within the root pivot.
- overlayfs solves the persistence efficiency issue using a layered union fs.
> most containers are developed
Most Linux (and Windows) containers. Since these are macOS containers there are no containers developed yet so by definition there is nothing to break.
> bind mounts via MacFuse
> How does this work?
Essentially, like this:
https://ericchiang.github.io/post/containers-from-scratch/
https://earthly.dev/blog/chroot/
> The way Docker Desktop (and podman.io) implement "containers on macOS" is a bit of a cop-out
It's not, it's a requirement for running Linux containers: https://news.ycombinator.com/item?id=37656401
DfM is more like running the CLI locally to a remote Linux machine, and all it does is conveniently expose /Users in the same place through the VM folder share so that you have the convenient illusion that it happens locally.
If Darwin had process namespacing features it would not make it magically able to run Linux processes.
> Does XNU have such namespacing functionality across all its interfaces?
I don't think so, but some Docker features could be implemented using XNU sandboxing AFAIK
> Furthermore, the existing container ecosystem assumes a Linux syscall interface. [1]. Does macOS provide that? I expect not.
This project is about running macOS containers on macOS. It's not about running Linux containers.
> > Does XNU have such namespacing functionality across all its interfaces?
> I don't think so, but some Docker features could be implemented using XNU sandboxing AFAIK
Theoretically, probably, for coarse-grained yes/no things? I don't think it's able to go much further than "you can use the local network and/or internet" and "you can read/write to the filesystem location corresponding to you bundle identifier `com.foo.bar`" but not "hey let me present you with a namespaced view of loopback or process list".
Also not sure if it can be dynamically set by a parent process for a child? Seems like it's very bundle oriented (except maybe for Apple processes) so not very practical.
> Also not sure if it can be dynamically set by a parent process for a child?
Yes, it can. See sandbox-exec tool. And I actually plan to use it: https://github.com/macOScontainers/rund/issues/15
Oh my I totally forgot about sandbox-exec!
I played with it some time ago, can't recall the context but it was about build systems / packaging (maybe nix?), doing the configure/make/make install with reduced privileges.
> Furthermore, the existing container ecosystem assumes a Linux syscall interface. [1]. Does macOS provide that? I expect not.
There is more to the container ecosystem than Linux containers; Windows native containers function much the same way (well, in two ways, with VM-backing or the traditional kernel syscall interface, but with Windows syscalls).
I am also interested in the API boundary. It seems that there are too options:
1. Rely on system call stability. This is like Linux containers but unlike Linux macOS doesn't provide a stable system call API. So this would break when the system updates with a change that modifies the system call API.
2. Install the host libraries into the container at runtime. This should provide as much stability as macOS apps usually have. It may also be beneficial as you wouldn't be embedding these into every container.
It seems like 2 would be preferable. However it may be a bit weird when building as the libraries you build against would be updated without the container being aware, but this is unlikely to break anything unless they are copying them to new paths which seems unlikely.
> macOS doesn't provide a stable system call API
I'm really wondering, do you have any links about macOS syscall stability over versions?
Generally speaking macOS does not guarantee syscall stability, and does not generally guarantee compatibility for any binaries not linked to `libSystem.dylib` (that is the supported ABI boundary)[1]. This has a number of implications, including (but not limited to):
* The most obvious is the commonly mentioned fact that syscalls may change. Here is an example where golang program broke because they were directly using the `gettimeofday()` syscalls[2].
* The interface between the kernel and the dynamic linker (which is required since ABI stability for statically linked executables is not guaranteed) is private and may change between versions. That means if your chroot contains a `dyld` from an OS version that is not the same as the host kernel it may not work.
* The format of the dyld shared cache changes most releases, which means you can't just use the old dyld that matches the host kernel in your chroot because it may not work with the dyld shared cache for the OS you are trying to run in the chroot.
* The system maintains a number of security policies around platform binaries, and those binaries are enumerated as part of the static trust cache[3]. Depending on what you are doing and what permissions it needs you may not be able to even run the system binaries from another release of macOS.
In practice you can often get away with a slight skew (~1 year), but you can rarely get away with skews of more than 2-3 years.
[1]: https://developer.apple.com/library/archive/qa/qa1118/_index...
[2]: https://github.com/golang/go/issues/16606
[3]: https://support.apple.com/guide/security/trust-caches-sec7d3...
I do not. I'm not really a macOS user but have heard this mentioned many times. I remember one recent example is Golang switched from doing raw syscalls to using the macOS libc due to this issue. That was probably ~5 years ago now so you may be able to dig up examples there.
I can't help but feel like this is an X/Y problem. Apps on MacOS shouldn't need containerization to function.
I get the point of isolation for build/test situations. But Apple provides a neat virtualization framework, and you get security + isolation + reproducibility + decent performance.
It seems like if you feel the need to containerize the userspace on MacOS you're using MacOS wrong. It's not the same thing as the Linux userspace, and doesn't have the same kernel features that would let you do so cleanly or performantly.
Orbstack is moving mountains to provide Linux-native perf and support for containers and it still makes me beg the question: why are devs allergic to just using Linux natively? At least I understand why Orbstack is useful, I don't know why containerizing MacOS itself is.
> But Apple provides a neat virtualization framework, and you get security + isolation + reproducibility + decent performance.
You also get limits on how many VMs your machine can run, each VM needs gobs of storage and locked-out RAM blocks, and sharing directories between the host and guest, compared to bind mounts, is something that makes me remember for my root canal dental jobs wistfully.
I've only used it with Linux kernels, but that has not been my experience with the new Virtualization.framework. The perf of virtio shares is reasonably fast.
I can see how you'd need a crap ton of disk for MacOS virtualization, but again, why do you need it?
If it's isolation for builds, fix your build. If it's isolation for tests, live with it. If it's for running your app, write your app to properly run in the app sandbox.
> If it's isolation for builds, fix your build
Isolation fixes the problem for free and is more similar to how I build for Linux. Why would I want another solution?
MacOS apps are designed to be self contained and not require isolation, unlike most Linux distros, which are designed to use FHS and share their state/dependencies with everything else on the system.
Further you can sandbox processes just fine on MacOS without needing to mock the whole userspace, like you do on Linux. This will give you the same degree of isolation that a container does.
> reasonably fast.
But still not "as fast as it gets". And I want it to be as fast as it gets.
Neither is using macfuse as a stand in for overlayfs and bind mounts.
My point is "as fast as it gets" is using MacOS how MacOS is designed: ie, through sandboxing and not containerization that pretends the MacOS userspace is Linux. It's not Linux.
There is a fundamental trade off between isolation and performance. You cannot securely share resources without overhead.
FUSE is only used for bind mounts. If you write to a directory that belongs to container, you get the raw speed of host OS.
How do you support layers without overlayfs?
It's not me, it's containerd "native" snapshotter. It turns to be efficient on macOS thanks to `clonefile` syscall.
What's the licensing situation on this? Would I be distributing parts of macOS in my containers? I don't think Apple is OK with that.
Or is this just the fully open source Darwin core? That wouldn't likely be super compatible with a ton of production software? I need more explanation of what is actually going on here because it sounds like a good way to get sued.
> What's the licensing situation on this?
1. This project didn't take explicit permission from Apple to redistribute binaries
2. There are multiple jurisdictions where you don't need to explicitly have such permission, it is implied by law
3. Usage of this software implies you already have macOS system. I'm not a lawyer, but it looks to be covered by section 3 of macOS EULA.
4. There are existing precedents of redistribution of macOS binaries for multiple years aready:
- https://github.com/cirruslabs/macos-image-templates/pkgs/con...
- https://hub.docker.com/r/sickcodes/docker-osx
- https://app.vagrantup.com/jhcook/boxes/macos-sierra
And so on.
> Would I be distributing parts of macOS in my containers?
Unless you're producing fully static binaries (or static enough that they don't bind to non-redistributable things) it'd be a yes (it would not be much of a container if it needed non-packaged things)
The screenshot points out a ghcr.io URL that lands on these packages: https://github.com/orgs/macOScontainers/packages?repo_name=m...
Edit: There's a note here†, so at least there is some consideration for licensing. No idea if it holds ground.
† https://github.com/macOScontainers/macos-jail/blob/9b1d5b141...
Counting the hours until this gets DMCA'd out of existence
Anecdotally:
- there has been images like that circulating on well-known platforms† for a long time.
- I never heard of a single DMCA on any of these, even though they were on much clearer violation of the license.
Apple might be pulling a Hackintosh manoeuvre here.
† an example, there are many others: https://app.vagrantup.com/jhcook/boxes/macos-sierra
folks have been building macos vm images, sharing them, and virtualizing macos on macos for a decade or so now.
is this that much different?
Reminds me: Still waiting for native ARM support on GitHub Actions https://github.com/actions/runner-images/issues/5631
There are Cirrus Runners which is a service of managed GitHub Actions Runners powered by M2 chips. But there is no free option for OSS yet. https://tart.run/integrations/github-actions
This one is ridiculous. This should already exist. Until GitHub builds it, you can use GitHub Actions to kick your builds off but run them remotely on Earthly Cloud (https://earthly.dev/). Even the free tier includes arm64 remote runners.
Note: I work at Earthly, but I'm not wrong about this being a good, free, arm64-native workflow for GitHub Actions.
You mean, Earthly has macOS arm64 runners? I am failing to find any info on its website.
Isn’t that service shutting down in <1 week?
We shut down CI. Earthly satellites are still going strong.
https://github.com/macOScontainers/macos-jail - new code
https://github.com/macOScontainers/rund - new code
https://github.com/macOScontainers/moby - fork, 6 commits
https://github.com/macOScontainers/buildkit - fork, 4 commits
https://github.com/macOScontainers/containerd - fork, 5 commits
Would be interesting to see if they can get moby/buildkit/containerd changes upstreamed
This is a failed attempt to upstream part of containerd changes: https://github.com/containerd/containerd/pull/8789
Other part of containerd changes waits for gods-know-what: https://github.com/containerd/containerd/pull/9054
But I haven't gave up yet.
Just merged #9054.
Sorry for that we had to revert #8789, but we are looking forward to seeing that PR submitted again with an alternative abstraction interface.
Yeah, I undestand the situation with #8789.
If you want to follow my upstream attempts, subscribe to
System Integrity Protection sounds really important. What does it do normally, and why does this tool require it to be disabled?
SIP is secure boot for macOS.
It's intended to prevent malware from changing system files due to rogue permissions or escalation. With SIP enabled, even the root/sudo user doesn't have rights to change these files.
It also refuses to boot a system with drivers that are not signed by Apple, so as to deter malware from using drivers as an attack vector.
> SIP is secure boot for macOS.
Not really. «Secure Boot» is intended to secure the boot process through signature verification. However the security model is completely broken, https://arstechnica.com/information-technology/2023/03/unkil...
SIP is a protection layer which protects system files from modification also after the system is booted.
Apple ships a non-broken secure boot, unlike almost everyone else.
It's meant to protect the base OS from persistent malware infections. https://support.apple.com/en-us/102149
Also against accidentally deleting /usr.
Why should anyone trust this website and download the software? There's no indication who made it. Could be malware for all I know.
Looks like it's open source https://github.com/macOScontainers
Unfortunately, that doesn't answer their question.
It answers both of their questions. Git lets you see the author and they can audit the code if they like.
IMO that’s not good enough, especially when disabling SIP is involved.
We don’t even have certainty that the human running the account is who they say they are (anyone can make a GitHub account and make it look like a real person).
Not everyone who wants to use a container system understands the underlying code of that container system. If I’m a web developer using Docker Desktop or podman to build my PHP app, I’m not necessarily going to understand the code written in Go when my specialty is PHP.
yes it does. There’s only one contributor for most of it and you can click to see his profile.
With the source code available and the primary contributor clear, what more could anyone want? Certainly it’s a bit much for one to ask for a security audit they themselves won’t do
> With the source code available and the primary contributor clear, what more could anyone want?
Reproducible builds. :)
How hard is it to make a GitHub profile?
If I am a developer using podman/Docker to build my PHP images, am I expected to understand code written in Go?
These are all acceptable risks until someone is asking me to disable SIP.
> There's no indication who made it.
It clearly links to the GitHub where you can click to see all contributors
I suppose the answer to your question is “people who want macOS containers”, whoever they are. As far as malware, I’d employ whatever your standard practices are for installing GitHub projects
Which answer to this question would sufficiently comfort you?
I feel cheated by Apple a little bit.
I bought an Apple Silicon machine after their presentation claiming that they would have first class docker support, but the reality has been that while the first docker worked well as it was translated, now it wants to default to arm containers and it has become very difficult to use because it doesn't want to use Rosetta 2 containers.
The whole point of using docker is to use the same containers in production as you use in development, so having docker default to these random arm containers means that my containers aren't exactly production, because they are arm based and the servers are not.
I understand that docker is the developer of docker software, but I really wish I could just click a button and force intel based containers in docker as the default and have to opt-in to arm.
If anyone has an easy solution to this let me know. I don't want to spend hours and hours figuring out docker on my mac.
Try OrbStack ;) It’s the only docker daemon alternative for Mac that makes working with docker acceptable.
+1 on Orbstack! def been a huge difference from the desktop client, and supports K8s too
Thanks! I'll check it out.
export DOCKER_DEFAULT_PLATFORM=linux/amd64
This is amazingly cursed and I'd love to see this become viable
TIL SIP blocks chroot. I wonder why?
I imagine because if an adversary can a chroot environment, then they can trick anything using that chroot to use different binaries.
I have these thoughts about Homebrew also.
macOS apps have to be signed and notarised to run without a warning, which is a pretty big part of the defence picture for this software - the certificates can be revoked at any time to block the software if malicious behaviour is identified.
However, if I install Homebrew, then install python, then install a pip package, there's really no kind of scanning/notarization/checking happening at all. I wonder if this is something Apple has ever looked into - it seems like the exact scenario where you'd want to sandbox it away from the rest of the system.
> However, if I install Homebrew, then install python, then install a pip package, there's really no kind of scanning/notarization/checking happening at all.
There is: you are running that pip package in a chain of processes: Terminal (or iTerm, or whatever) - your shell - python - pip package. In this chain, Terminal has "Developer Tools" privilege, which allows you to run software, that does not meet the system's security policy.
You can disable this privilege in System settings, Privacy and Security panel.
EDIT: Having looked at this again, I'm not sure the Developer Tools privilege is even required to run essentially arbitrary software in Terminal. Homebrew still works without Developer Tools installed, as do interpreted runtimes. If I can even run an arbitrary shell script then the privilege is pointless, which makes the case for a dedicated sandbox for Terminal even more important.
This is my point, Terminal with Developer Tools privilege is essentially granting carte blanche privilege to every sub-process running there, which is not really necessary.
If I could pop a Terminal with a restrictive scope, allowing read and write access only within that directory tree it would be a much better situation than we have now. macOS' permission system isn't really that far away anyway - I can already disable Terminal's permissions to access anything else same as other apps.
Essentially a basic and very limited not-even-a-chroot would be a vast improvement and would still offer an adequate sandbox.
Yeah, but we need to analyze picture as a whole. And by default, chroot is banned while running as root arbitrary stuff downloaded from the internets is not.
Only in terminal and only if you have developer tools installed. Otherwise, it is not.
That's indeed the point of chroot. It's a feature not a bug.
This is a cool idea and an impressive project.
At the same time, I don't truly understand why anyone would need to use it. If your preference is to totally work with macOS, then I'm sure this would be perfect for that. Otherwise, what's the advantage?
VMs have really come a long way. Every major OS today has a virtualization framework that makes running another OS extremely performant. Docker on macOS uses a virtual machine, but so what? Performance of individual containers, in my experience, isn't really a problem unless you're doing something with the GPU, and even then there are ways to deal with that. Even a fully-emulated VM using QEMU (without hypervisor or KVM) won't have any noticeable performance penalties in many cases.
IMO, there's a much greater advantage to sticking with Linux. Even if the host isn't Linux, developing and deploying with Linux guests provides a tremendous level of consistency and portability.
But maybe I'll be proven wrong by this project someday soon!
What my dream is that the User Mode Linux is made into a cross-platform userspace binary that translates syscalls transparently between itself and the host. So you might get "drivers" that talk to Windows, Linux, *BSDs, Darwin, it manages memory in an efficient (for the host) way, and enables you to run any kinds of wild experiments with, say, virtualized and passed-through serial devices, USB devices, networking, bind-mounting from the host and image mounts. And yes, containers. All of that without needing host root in most cases.
Of course the drawback would be that the host would see just a fat Linux process and its child processes, much like you can see qemu, but it could be an interesting thing nonetheless, if even for shits and giggles of it.
For a Docker-like OCI experience on MacOS without disabling SIP, check out TartVM. Happy user here.
When macOS runs on Unix kernel and Linux systems are the best supported for containerisation and I assume are much more lightweight than macOS, I personally don't see any reason to run macOS in a container.
You would do it so you can run mac software in a container.
I think I get that part :) but realistically if anyone wants to run macOS software they are most likely macOS users already. Whereas containerisation is useful to aid with development and deployment. Have you come across anyone who runs production software on macOS? :)
One might develop a game that by some kind of a miracle releases for macOS too. So they way to run CI tests on macOS. Or they might target iOS. And use macOS build machines to produce builds. The world is not only about web, you know?
Can anyone speak to how the macOS runners on GitHub actions work? It would seem from this post that containers of any kind for macOS are a brand new thing..
AFAIK GitHub action runners all use VMs and not containers
Yes, they are VMs
I'm probably getting confused between containers and VMs but, sorry, I don't understand what the difference is in the case of MacOS?
It is the same for any OS. Virtual machine boots a separate instance of the whole OS. This is slow, this is often too much isolated (you can't easily/effectively share files between host and guest), you need to set artificial limits on VM disk/memory/cpu. On the other side, containers work in the context of host OS, what means less overhead and easier interaction with host.
I see! Thank you! :)
Despite the SIP problem, this is really exciting.
the amount of engineering hours wasted making macos usable for backend dev work and then wasted again from inefficiency due to that failure is staggering.
linux is great. macos is great. windows is great too. for their intended purposes.
it’s horseless carriages all the way down.
MacOS is - by choice - an Apple controlled walled garden.
Trying to break out of that is an exercise in futility.
Can you come up with situations where I would run a container instead of just running an app or sys service?
caveat: this is based on rund. Extract from the readme:
rund is an experimental containerd shim for running macOS containers on macOS.
rund doesn’t offer the usual level of container isolation that is achievable on other OSes due to limited macOS kernel API.
What rund provides:
Filesystem isolation via chroot(2)
Cleanup of container processes using process group
OCI Runtime Specification compatibility (to the extent it is possible on macOS)
Host-network mode only
bind mountsI wonder if something comparable can be achieved with ptrace. I've tried to experiment with that but the API seems a bit unclear to say the least
I use MacOS and am very positive about it. I have lots of reasons to run Linux containers. What are some reasons I might want to run a MacOS container?
At work, this would've been super helpful for compiling macOS libraries. We ended up using a macOS cross-compiler [0] instead.
I would imagine it would enable you to run MacOS specific pipelines like building a project and have it work semi-portably across different machines/users.
Yes, I can see it could be useful for a company making MacOS software. But for a company whose developers use MacOS but whose product has nothing specifically to do with MacOS, my instinct is that if local environment reproducibility concerns get to that point then linux containers are the answer since they'll be more closely related to CI/prod envs. Or are you thinking maybe that's wrong for large companies with lots of developers using MacOS?
This uses macfuse, a closed source software
I think MacOs is also closed source.
Unrelated to containers themselves: how do you make a patch when no version was released? I mean, people call this "semantic" versioning, but then spit in the face of those semantics...
Original author here. I wanted to clearly indicate early-prealpha-unstable-not-for-production-yet state of this software. Using "1.0.0" and even "1.0.0-alpha" would give false expectations about maturity of this project.
This was more of a stab at "semantic" part of the semantic versioning (which similar to <div> in semantic Web... found its own semantics that don't follow from its definitions). You are definitely not the only one using it like this.
I believe this is the problem with the format of semantic version which seem to assume that releases only happen to software ready to be... released :)
My preferred course of action in such situations is not specify a version at all.
> My preferred course of action in such situations is not specify a version at all.
This doesn't work because, well, I do make releases and they need some numbers)
That's a contradiction. You cannot release a software that you didn't release. :)
If you release it, it has to start with version one. Zero is for non-released versions.
Okay, let's call them "labeled code snapshots")
0.1.0 might make more sense, but still not completely (backwards compatible with what?). Then again, it doesn't really matter. It's not like someone's going to accidentally install 0.0.1 because of semantic versioning not being 100% correct.
Version 0?
Oh, it was easier than I thought: https://semver.org/#spec-item-2
So, no leading zeros, ta-da!A normal version number MUST take the form X.Y.Z where X, Y, and Z are non-negative integers, and MUST NOT contain leading zeroes. X is the major version, Y is the minor version, and Z is the patch version. Each element MUST increase numerically. For instance: 1.9.0 -> 1.10.0 -> 1.11.0.Oh, wait. The spec was written by some... big brain:
So... my reading of this "definition" is that there's really no need for three digits, if major is zero... Then why on earth would you have two digits? Also, if no pubic API at this point, then why have versions at all? I mean, you clearly shouldn't be specifying anything with zero major version as a dependency because it should be illegal to depend on a library w/o public API... Then, again, why have versions in this situation? And if the argument is that its for internal use, then why standardize it for external use?Major version zero (0.y.z) is for initial development. Anything MAY change at any time. The public API SHOULD NOT be considered stable.Just two paragraphs below. How lovely.
Haha, what a mess)
I believe at least some semantic version parsers would not accept version zero. I'd need to find the definition though. I could be wrong of course.
Can I run them on a normal PC with Linux?
No. For the same reason you cannot run Windows containers on Linux. You need a working macOS kernel.
7G images, wow
I believe that could be optimized in the future. At least, by splitting amd64/arm64 code.
Can anybody explain, what's the point of Mac containers? Almost nobody uses Mac for container based deployments.
At work, this would've been super helpful for compiling macOS libraries. We ended up using a macOS cross-compiler [0] instead.
CI/CD workflows most likely. And devshops that have standadised on docker containers for their stacks (mac-based devs in such places suffer a fair amount of papercuts today). Then I guess there are people that are very security minded that might want to run all userland executables in containers (although this project here is not for them I'd say).
> CI/CD workflows most likely
Yep, this is primary goal of this project.
It's sad to see so many negative comments for this. I get it's not an ideal place to start for macOS containers, but it's a start. Apple isn't doing it, so the community has to. Once you have a start, you can iterate on it. It might not be great now, but hopefully this makes it possible in a year or so. Who knows, maybe this is the kick Apple needs, and maybe they'll hire the devs of this project to fully work on this.
Sorry, not disabling SIP for something that I can already do without needing to nobble security policies (and have them reset/impossible due to MDM). If there was user/networking space in Darwin then maybe I'd be interested but...
> something that I can already do
I would be very curious as to how you already run darwin containers.
The only alternative is spinning a macOS VM (including relying on macOS CI machines as a remote job executor)
I mean in terms of functionality, this doesn't give me anything extra to what using docker would do on macos (granted it's via a linux based VM). From an end user perspective there's no real difference, but I don't have to entirely disable SIP just to use it.
Something like namespaces or proper jails on darwin would be super cool, but not at the expense of other security measures and chroot-ish outcome imho. Maybe this works for some, but not me :)
> I mean in terms of functionality, this doesn't give me anything extra to what using docker would do on macos (granted it's via a linux based VM)
Ah, I understand your angel, in that your use case is to run namespaced processes that achieve some functional purpose irrespective of the underlying kernel/platform, which is totally fair.
> not at the expense of other security measures
Not for me either... that is, not in a host OS, maybe a dedicated VM; I consider this to be as it says on the tin, 0.0.1, a thing that would help bootstrap an ecosystem of containers, which would push towards Apple adding namespaces or jails (oh, hell yeah, JailKit!) to darwin.
> this doesn't give me anything extra to what using docker would do on macos
You're missing the point. This project DOES use docker.
I mean docker machine/desktop (kinda implicit of running docker on macos) not the cli or anything :)
It’s remarkable that Apple doesn’t have a first party solution to this yet. They used be, or aspire to be, at the forefront of OS research.“The most advanced Unix”.
They’re not even trying, now.
A/UX wasn't that much advanced, and most technology interesting stuff in NeXTSTEP, and OS X, is completely unrelated to UNIX.
Off the top of my head:
launchd inspired systemd.
Spotlight (real time indexing and notification) is something I miss in Linux today.
64bit Unix layer on consumer hardware (G5).
All of that stuff was not a first ever implementation, of course, but it was well executed and led the way.
All of that was more than a decade ago.
All of that appeared first in HP-UX, Solaris and Aix.
Xenix and Coherent were the first UNIX on consumer hardware.
Which is why I said: “All of that stuff was not a first ever implementation, of course, but it was well executed and led the way.”
Apple is rarely the first at doing something but it's often the first that's good and cheap enough that people care about.
Which makes the point that there is hardly any UNIX inovation coming from them.
Please cite a USENIX research paper from Apple.
Apple didn’t invent the GUI, touchscreen or the concept of unifying init, cron, etc.
But it was after they released their implementations that those ideas caught on.
I hope you can see the point I’m making.
UNIX was already winning the server room and workstation market before Apple, that is why they came up with A/UX in first place.
The Hollywood studios that now use Apple, would be using SGI previously.
On iDevices, UNIX APIs aren't even that relevant for app development, even basic stuff like networking has been superceeded by Objective-C specific APIs.
So no, I don't see anything UNIX related where Apple has helped to caught on.
Moving beyond UNIX, now that is a thing NeXT and Apple have done a lot.
> So no, I don't see anything UNIX related where Apple has helped to caught on
Perhaps if you read “Rethinking PID 1”, from Lennart Poettering in 2010, who originally wrote systemd along with Kay Sievers
http://0pointer.de/blog/projects/systemd.html
“But first, let's clear a few things up: is this kind of logic new? No, it certainly is not. The most prominent system that works like this is Apple's launchd system: on MacOS the listening of the sockets is pulled out of all daemons and done by launchd. The services themselves hence can all start up in parallel and dependencies need not to be configured for them. And that is actually a really ingenious design, and the primary reason why MacOS manages to provide the fantastic boot-up times it provides. I can highly recommend this video where the launchd folks explain what they are doing. Unfortunately this idea never really took on outside of the Apple camp.”
Other than that, your answer has nothing to do with what I wrote.
Maybe they should have spent some time actually looking into Solaris Service Management Facility, or Windows Server SCM for that matter.
I’m sure they did, it’s even stated in the first paragraph I quoted: “But first, let's clear a few things up: is this kind of logic new? No, it certainly is not.”
It’s just that influence is not all about being first.
You may argue that Quartz inspired Wayland, but I wouldn’t be so sure, maybe that was just obvious given the hardware of today.
Non UNIX display technology inspired Wayland.
Amiga, Atari, BeOS, OS/2, Windows, classical Mac OS,...
cries in Asahi Linux
macbook is the best laptop there is but macos...
can't wait for a stable release of Asahi and permission from corporate to install it even in a VM somehow. probably won't happen, but one can dream.
You're missing the point of Asahi, by the looks of it. It's not intended to be run in a VM.
I mean, I won't be allowed to install it side by side on my work laptop in a million years. GPU acceleration would be nice in a VM if macOS can pass it through, which I've no idea if it can.
It's not really intended to be run as a standalone distro (there's a fedora version though) and (afaiu) the point is to understand the bootloader process and stuff like GPU support from linux.
I know company policy moves glacially with these things (been there, got the faded t-shirt!) so yea, you're probably right there. Technically you could probably still use linux MDM instead of MacOS for mgmt, but getting that past IT is nigh on impossible imho also.
As long as the architecture satisfies Popeck and Goldberg, why shouldn't it?
The point of Asahi is to provide linux drivers for Apple hardware. What would be the point? You can already run Linux in a VM on arm macOS today with good performances. You don't even need to disable SIP.
Passing through a device and using it in a VM.
I hope their software quality is better than what the page looks like in a small form factor.