Alexander Yue (@Alezander9) on X

Here is a failure we keep having at Browser Use. It is not a bug.

Three agents share one workspace. Each agent starts. Each pulls the workspace from an S3 bucket. Each does good work. Each pushes the workspace back to the same bucket. Work is lost.

The system does exactly what we built it to do.

We built a shared filesystem out of copies, and copies do not share. We knew that. We did it anyway, because it is what everyone does.

This article is about the pain of files across machines. Then it is about a set of filesystems that solved these problems decades ago, but lost anyway.

Filesystem Pain

Pain 1. Many agents in a workspace

An agent needs files. The files live in a bucket. So the agent copies them down at the start and copies them up at the end.

This is not a shared workspace. This is three private workspaces and a race.

The classic fix is a lock. But a lock makes the agents wait, and the whole point of many agents is that they do not wait. The other classic fix is merge. But merge needs to know what changed, and a bucket only knows the final bytes.

Pain 2. User file upload

The user has a spreadsheet. The agent needs it.

USER LAPTOP ----> API ----> S3 ----> AGENT VM

Every arrow extra code, every arrow can fail. And the last arrow is the worst one, because the agent may be currently running and changing files.

Pain 3. The user file download

The agent made twelve files. The user wants to see them.

We built a viewer. The viewer reads from the bucket. The bucket is behind the agent, so the viewer is always outdated.

Pain 4. The user wants to edit a file

The user edits a file while the agent writes the same file. Now there are two writers and no merge. If two agents are running, there are three writers.

So we removed the feature. Because we have no model for what "the file" means when two actors hold it.

That is a strange place to be. Editing a shared file is a solved problem in Google Docs. It is not solved in our stack, because our stack thinks a file is a blob of bytes with one owner.

Pain 5. The browser's files

The agent runs on one machine. The browser runs on a different machine. The browser downloads a PDF. The PDF lands on the browser machine's disk.

The agent cannot read it. It has to do a workaround.

We did not decide against this bridge. We never proposed it. The idea did not come up, because "two machines have two filesystems" is such a deep assumption that it does not feel like a decision.

So instead the agent reads the content from browser, and types it out again into a file on its own machine. The data makes the trip through a language model. That is our file transfer protocol.

The five pains are one pain

Look at them together.

Four islands. We move bytes between them by hand. Every path is custom. Every path is fallible. And because making paths is a hassle, we build fewer of them than we should.

Where do we assume this is it?

The filesystem in your head was designed for a machine with one disk, one processor, and a small number of humans typing at terminals. That machine was real, and the design was excellent for it.

That design says a few things so quietly that we forget they are choices:

None of these are laws. They are defaults. And defaults are the strongest force in engineering, because you do not argue with them. You do not even see them.

What else could it have been?

A. Multics

MIT, General Electric and Bell Labs, mid-1960s.

In Unix you open a file, then copy its bytes into a buffer, then work on the buffer, then copy the buffer back. This is the idea of "files" we all know. Multics deleted the middle.

Every piece of stored information was a segment, and a segment was directly addressable by the processor. There was no load step. Access rights were checked by hardware on every single reference.

This is called a single-level store. It is not dead: IBM i, running on Power systems in banks today, still works this way. It came from IBM's Future Systems project and shipped in the System/38 in 1978.

B. VMS Files-11

DEC, 1970s. Inherited from RSX-11's ODS-1, which got the idea from ITS and TENEX before that.

The version number was part of the filename, in the filesystem, for every file, without asking. PURGE deleted old ones when you wanted the space back.

Users of that era did not think of this as a feature. It was just how files worked. Then we all moved to systems without it, and we rebuilt it badly, over and over, as report_final_v2_ACTUAL.txt.

C. Plan 9

Bell Labs, from the late 1980s. Rob Pike, Ken Thompson, Dave Presotto, Phil Winterbottom and others, with Dennis Ritchie running the department. This is the lab that produced Unix, deciding that Unix was not finished.

Two ideas carry the whole system:

Every resource is a file server - the network stack, the window system, the process table, a remote machine's disk. All of them speak one small protocol called 9P.

Every process has a private namespace. You build your view of the tree by mounting servers into it. Two processes on one machine can see completely different filesystems.

Because remote things mount exactly like local things, "use the network card on that other computer" is one command. Not an API. Not a library. A mount.

AT&T sold it in 1995 for $350. It was moved to the back burner within a year. Eric Raymond's explanation is the honest one: it was not enough better than Unix to displace Unix. It had no applications and no drivers.

D. Coda

Carnegie Mellon, from 1987, under Mahadev Satyanarayanan. A descendant of AFS.

Every distributed filesystem before this treated a network failure as a fault. Coda treated it as a state.

The client hoards files you marked as important, so that when the link drops you already have them. This is the same problem OneDrive and Dropbox and iCloud solve, except Coda was solving it in 1987 with a real conflict model instead of a "keep both copies" dialog box.

E. BeFS

Be Inc., 1996-1997. Dominic Giampaolo and Cyril Meurillon, in about ten months.

Files had typed attributes. Attributes were indexed in B+trees. So you could ask the filesystem questions, and the answers behaved like folders.

The saved query is a file containing the question, not the answer. Open it and it re-runs.

This idea keeps being attempted and keeps dying. Microsoft tried three times: Object File System in Cairo, then Storage+, then WinFS. WinFS was demoed in October 2003, pulled from Vista in August 2004, and dissolved into other products in June 2006. Bill Gates called it his greatest disappointment.

Note also that WinFS was not a filesystem. It was SQL Server files sitting on top of NTFS. The marketing said filesystem; the architecture said database layer.

F. Venti

Sean Quinlan and Sean Dorward, Bell Labs, 2002. Part of Plan 9.

Some consequences fall out for free:

Identical blocks are stored once. Deduplication is not a feature; it is a side effect.

Writes never overwrite. The store is append-only.

A client can verify the server by re-hashing what it gets back.

Venti predates git by three years, and git's own documentation describes git as "a content-addressable filesystem with a VCS user interface written on top of it."

G. LFS and NILFS

The concept is Ousterhout and Douglis, 1988. The design and implementation is Rosenblum and Ousterhout at Berkeley, SOSP 1991, in the Sprite OS.

NILFS2, from NTT in 2005, is the version that lives in mainline Linux today. It takes a checkpoint every few seconds, automatically. Any checkpoint can be promoted to a snapshot and mounted read-only while the filesystem is still being written to.

Honesty: the hard part of a log-structured filesystem is the cleaner, the garbage collector that reclaims space from the tail. It is where these designs get complicated. NILFS2 still has no fsck.

H. Elephant

Santry, Feeley, Hutchinson, Veitch, Carton and Ofir. University of British Columbia with HP Labs, SOSP 1999.

It is tempting to file this under "keeps everything forever". That is backwards, and that is the point. Elephant's contribution is retention policy per file:

The interesting question was: which versions are landmarks? A human saving every thirty seconds produces mostly noise and occasionally a milestone. The filesystem tried to tell them apart.

Exhibit I. Tahoe-LAFS

Brian Warner and Zooko Wilcox-O'Hearn, from 2006.

Capabilities come in strengths, and you can only hand out something weaker than you hold. A read-write holder can give a read-only cap. A read-only holder cannot manufacture a write cap. Delegation only ever diminishes.

The design principle has a name: POLA, the Principle Of Least Authority.

The same idea appears in FreeBSD's Capsicum (2010), where a process gives up the ability to name global paths and can only act through file descriptors it already holds. And in Google's Fuchsia, where there is no global filesystem namespace at all - each component gets a namespace built from the handles its parent chose to grant.

For agents: this is the real answer to "what can this sub-agent touch". Not a sandbox path prefix that we check in application code. A handle it either has or does not have.

Exhibit J.

Two virtual filesystems run at enormous scale right now, inside two companies, and almost nobody outside those companies uses them.

CitC (Google). Described publicly in CACM in 2016. A FUSE filesystem giving every engineer a view of the entire monorepo without cloning it. The average workspace holds fewer than ten actual files; everything else is materialized on demand. Peak load is around 800,000 file reads per second.

EdenFS (Meta). Same shape. Lazily fetches file contents from source control as they are touched. Uses FUSE on Linux, NFS on macOS, ProjFS on Windows.

Both of these know exactly which files were read, because every read passes through them. Google's Piper logs reads durably; that log is used to find out who accessed a file before it was removed.

Change

Every filesystem you use was designed with a human in mind. That human is slow. That human gets confused. That human needs undo, because they make mistakes and feel bad about them. That human will not tolerate a system where a folder is a query and a file has forty versions with hashes for names.

An agent does not get confused by a namespace with four thousand mounts. An agent does not mind that a file is addressed by its SHA-256. An agent is happy to work in a world where nothing is ever overwritten and everything is a version, because an agent has perfect patience for bookkeeping and no nostalgia for the desktop metaphor.

Most of these designs did not lose because they were wrong. They lost because they were unpleasant for people. Plan 9 was not enough better than Unix to be worth abandoning Unix. WinFS collapsed under its own complexity. Versioning filesystems asked users to think about version numbers, and users did not want to.

These are all human objections.

We now have a class of user for whom none of those objections apply.