A Base Filesystem Project for macOS
kext.ioIt's always good to see open source projects published, especially for tricky and often under-documented stuff like file system kexts.
Not to derail from this fine post, but I wanted to give a shoutout to FUSE for macOS (previously known as OSXFUSE). As an outsider from the project - merely a happy user - it seems to be a well maintained FUSE implementation for macOS, with signed kexts available. https://osxfuse.github.io/
According to Dropbox it has performance & security risks: https://blogs.dropbox.com/tech/2016/05/going-deeper-with-pro...
According to Dropbox, it has performance implications and is complicated thereby introducing a large attack surface. For some reason, they believe their own kernel module is less complicated and easier to reason about than FUSE, despite having more in-kernel code and not being open source; I would venture to say their words about security are more about control of the attack surface rather than any actual implications about the quality of the FUSE code.
According to Dropbox it is okay to circumvent security mechanisms of the operating system (https://news.ycombinator.com/item?id=12463338) unless the media discovers your shady way of tricking the user into granting you full system access.
Their lost a lot of their credibility in my opinion.
why are there kexts in file system user space extensions? in OSX multiple driver types can exist entirely in user space, with no kext. [ former Apple engineer here ]
Are you saying you can write a 100% user-space "driver" that allows for `mount`ing into the regular file system? How does that work? Can a non-root user supply the code running to serve /some/path that is visible for any process on the system, even root?
In order to deploy and run your own kernel extensions on recent versions of the Mac operating system you will need to disable System Integrity Protection (SIP). Otherwise you will need a paid Apple developer account and ask Apple for a special codesigning certificate for kernel extensions.
https://developer.apple.com/library/content/documentation/Se...
Just yesterday I was thinking how all of the platforms are still using vfat as a go between. There really should be a new FS "for them all"
All attempts at such will fail. vfat won because it works in Windows without additional drivers. Anything you come up with to replace it will also have to work in Windows without additional drivers.
It will also have have a workable implementation in what is essentially equivalent to the public domain, and be simple enough to process that hardware manufacturers can read it with a microcontroller.
I would love to see it happen, but I am not optimistic.
Already exists, UDF. Built-in support on macOS, Windows, Linux.
Another option is NTFS, the one limitation is by default it is read only on macOs.
> vfat won because it works in Windows without additional drivers.
I would not understate that it is also very simple to write a driver for. The reasons for this have high overlap with the fact that the ondisk structure is crap. But it does mean that many people were able to write drivers.
I wrote a driver for it on an 8-bit chip that had 16kB of code space and 2kB of RAM. It was ugly, and it could only hold two sectors at a time in memory (if I recall, one from the FAT and one from the current file). Despite it being ugly... it worked! Best of luck getting pretty much any other standard filesystem running in that.
as mentioned above, ntfs works, but you'll need a driver for linux and macOS, which isn't too bad, and easier than getting unix file systems on windows.
There's also no fsck for NTFS on linux, so in the event of a power failure (or out-of-battery), you're stuck with a useless FS until you visit someplace with windows/osx machines.
You can run ntfsfix, a part of the ntfs-3g suite, to fix fs errors.
vfat only won in the sense that if you're forced to use windows you still have to use vfat. In every other context there is a better option. I haven't needed to move files to a windows box in 15 years so—to me—it's a massive surprise anyone still uses it at all.
Just don't use USB keys; they're terrible security-wise and you're forced to use technology built for your grandmother.
That is the only sense in which a 'universal file system' can win. There is no other context. Your personal usage habits are completely irrelevant ("I haven't had to shoot a gun since the first gulf war, so to me it's a massive surprise that anyone still fights wars at all.")
I don't use USB keys. I don't have any windows systems. But I do have UEFI, and that is based on FAT as well. Disregarding my grandmother's technical prowess, this technology is here to stay, and sticking your head in the sand about it doesn't advance any causes.
You're an outlier.
I live in an environment where I need to move files between several builds of Windows, Linux and macOS - and often where there is no network access because of security policy (misguided policy perhaps). Thumb Drives are the only way to function.
Can you say anything more about the environment? I'm curious.
Smells like the IT department of BigCo or public agencies to me. Been there, seen that. Literally terabytes of Debian binary packages transferred by hand with USB keys and disks between the dev network and the secure network.
Oh, and the USB serial numbers of the keys had to be whitelisted on the secure network. What a fun that one was. They didn't even stop doing that after I showed them that I could modify the firmware of some of the sticks using software from some obscure Russian site and thus bypass the control...
(Does anyone know by chance how that website is/was named? All I remember is that it held hundreds of manufacturer tools for all kinds of USB stick controllers/flash chips)
I work in the public service industry - many of the systems I work on are isolated from the internet, and from other networks even - so I often have to sneakernet software updates and other things around to all these various machines. The no network thing is reasonable when you consider the requirements these systems operate under - its also an easy way for folks with no system administration abilities to ensure some security and stability.
You could call it a universal disk format. https://en.wikipedia.org/wiki/Universal_Disk_Format
I tried UDF for a few months on a macOS/Linux/Windows triple boot. It doesn't work reliably enough. I've switched to ext2 using e2fsd and Paragon (which I paid for) on Windows and macOS respectively. Fingers crossed.
I wish someone would port F2FS to Windows and macOS. This could be a good starting point for macOS.
What sort of issues did you have with udf?
Windows threw up CRC errors. I am not blaming Windows here, that would be unfair. Perhaps it was the only one that checked CRCs on read and one of the others didn't write CRCs on write.
My use case was to store many large git repo clones and MP3s.
I think today it's exFat instead of vfat. That seems to be supported by a lot of phones and devices for large storage.
No, exFat is patent encumbered and isn't as simple and well documented as vfat (so non-Windows implementations are worse). vfat is still the common denominator.
https://en.wikipedia.org/wiki/ExFAT#Restrictive_licensing_an...
exFAT suddenly dirty-dismounts both on windows and osx, and is hard to mount again for regular user. Tried it as common r/w storage for bootcamp -- complete trash, cannot be used for long-connected drives, hyperactive torrents, big files. Copying some files forth-back is okay though.
Far from being "simple to implement", it's patent encumbered, making implementations for many OSs highly unlikely.
via fuse, ntfs is pretty broadly available... iirc there was a recent post about efforts to bring a fuse alternative for windows in (which should bring ext3 and btrfs)...
I tend to just use ntfs for most of my external storage and cifs for network storage only because it's easier to get on my non-windows systems than nfs and ext3 are to get working on windows... ymmv though.
> about efforts to bring a fuse alternative for windows in
You mean dokany? 'cause that one is one quite stable project.
Pretty soon, we'll just use XKCD numbers and not bother linking. We'll just call something or other "hey, that's a 927"
We should create a URI standard for that; xkcd://927
Should be simple enough: https://github.com/chungy/xkcd-uri
> Just made for fun, not expected to see any wide use.
Well that's jinxed it.
Let me propose the xkcd2 scheme, to be used like xkcd2:927, because who really needs those two slashes?!
My proposal: xkcd3, works as xkcd3:927 AND xkcd3://927
And that's why Linux desktop is going nowhere.
hi, what about xkcd4:ⅽⅿⅹⅹⅶⅹ or xkcd5:٩٢٧ (note: may not appear in correct reading order due to technical limitations)
I propose a browser powered by deep machine learning that lets you just write 927 and automatically figures out that it refers to an XKCD by inferring the context from looking at the content on the rest of the page.
Echoes of "Shaka, when the walls fell."
Akin to this is the keybase.io encrypted file system. It's in beta and brilliantly implemented so far.