Container Registry on the Edge | Evan Pratten

A significant number of my personal projects at some point involve wishing I could put a container registry closer to users.

In my past few (breif) attempts, I've looked at re-implementing the OCI specification as a Cloudflare Worker, backed by an R2 bucket, which has the dual benefits of being: fast, and cheap to operate.

Unfortunately, that also comes with the downside of being yet another thing I need to maintain as the underlying specification evolves and, to be honest, I already do enough of that at work, I don't need even more target-chasing in my personal tools as well.

Zot

A little while ago, Justin and I were chatting about OCI registries, and he brought up Zot, a single-binary batteries-included alternative to Distribution (the reference OCI registry backend).

Zot is managed by a pretty simple configuration file, and provides a reaonsably pretty web interface (on top of, of course, being a good OCI Registry).

These features, combined (most importantly) with not needing to maintain my own OCI spec implementation mean that Zot is pretty much the perfect fit for my needs.

Well, except for the "being close to the users" bit...

Cloudflare Containers

Earlier in the year, we released Cloudflare Containers, a way to run medium-lifetime container workloads in-region to service users' requests with arbitrary linux binaries.

The idea is fairly simple. A developer uploads a container manifest and some scheduling constraints to Cloudflare, then calls a few special functions from a Worker to request a nearby instance to warm up, and pass some arbitrary request data to it.

Note "nearby", Containers do not run in exactly the same place the user's request landed, they run in a place that makes sense from a latency and capacity perspective while still fufilling the developer-defined scheduling constraints.

Zot to the Edge

There's something kind of funny about using a third-party container registry and runtime to bootstrap a container registry for myself, but glossing over that, since Zot itself is a pretty slim container image, it's a pretty good fit to be scheduled onto the Cloudflare edge.

Crucially, Zot supports S3-like storage backends, which means we can run it entirely statelessly, backed by R2, giving me nearly the experience I was aiming for in my "just write it from scratch" attempts.

So, let's put Zot on the edge!

Defining manifests

Like I said earlier, to get a workload running on Cloudflare Containers, the developer needs to provide a couple manifests.

First, a container definition (fill in your own values if you are copying this):

# ./Dockerfile
FROM ghcr.io/project-zot/zot-linux-amd64:v2.1.21

COPY <<EOF /etc/zot/config.json
{
  "distSpecVersion": "1.1.1",
  "storage": {
    "rootDirectory": "/tmp/zot",
    "dedupe": false,
    "storageDriver": {
      "name": "s3",
      "rootdirectory": "/registry",
      "region": "auto",
      "regionendpoint": "https://<YOUR_ACCOUNT_ID_HERE>.r2.cloudflarestorage.com",
      "bucket": "<YOUR_BUCKET_NAME_HERE>",
      "forcepathstyle": true,
      "secure": true,
      "skipverify": false
    }
  },
  "http": {
    "address": "0.0.0.0",
    "port": "5000",
    "compat": ["docker2s2"]
  },
  "extensions": {
    "search": {
      "enable": true
    },
    "ui": {
      "enable": true
    }
  },
  "log": {
    "level": "info"
  }
}
EOF

..and a Wrangler config:

# ./wrangler.toml
name = "zot"
main = "main.js"
compatibility_date = "2026-09-01"

[observability]
enabled = true

[[containers]]
class_name = "ZotContainer"
image = "./Dockerfile"
max_instances = 1
instance_type = "basic"

[[durable_objects.bindings]]
class_name = "ZotContainer"
name = "ZOT_CONTAINER"

[[migrations]]
tag = "v1"
new_sqlite_classes = ["ZotContainer"]

Request handling

With the manifests in place, a simple Worker can be used to pass requests through to the container:

// ./main.js
import { Container, getContainer } from "@cloudflare/containers";

export class ZotContainer extends Container {
  defaultPort = 5000;
  sleepAfter = "10m";
}

export default {
  async fetch(request, env) {
    const container = getContainer(env.ZOT_CONTAINER);

    await container.startAndWaitForPorts({
      startOptions: {
        envVars: {
          AWS_ACCESS_KEY_ID: "<LOAD_SECRET_HERE>",
          AWS_SECRET_ACCESS_KEY: "<LOAD_SECRET_HERE>",
        },
      },
    });

    return container.fetch(request);
  },
};

Note that the way I've configured things in this example, one instance of Zot will exist at any point in time globally, scheduled closest to the first user to call it. Durable Objects scheduling hints can help you place this instance manually.

For advanced users, try bumping the container limit and giving Zot a global write lock to work with.

How it works

This setup works quite well actually, so far my only complaint is cold-start times, which can be worked around to some degree by tweaking sleepAfter and keeping a heartbeat probe going against Zot.

I've also been exploring the use of Cloudflare Access as an auth source for Zot, but I have not yet settled on an appropriate way to do that smoothly.