GitHub - suda/dvsync: 🐳️ Easy and secure way to copy data between Docker volumes, even across data centers

Easy and secure way to copy data between Docker volumes, even across VPCs or data centers.

dvsync is a set of two containers running OpenSSH, ngrok and rsync that automatically create a secure, encrypted channel between each other and enable easy way to migrate data stored in Docker volumes.

Running

⚠️ To establish a secure channel, dvsync uses ngrok and you need to pass your NGROK_AUTHTOKEN which can be found in ngrok dashboard.

dvsync will synchronize contents of its /data directory, therefore whatever data you want to sync, should be mounted under it.

Here are example ways to run it using Docker CLI, Docker Compose / Swarm or Kubernetes. Note that you can mix those, making data migration much easier.

Docker CLI

Start a server where you want to copy data from:

docker run --rm -e NGROK_AUTHTOKEN="$NGROK_AUTHTOKEN" \
  --mount source=MY_SOURCE_VOLUME,target=/data,readonly \
  ghcr.io/suda/dvsync-server

Once the server started, look into the logs and copy the DVSYNC_TOKEN
Start the client where you want tot copy the data to:

docker run --rm -e DVSYNC_TOKEN="$DVSYNC_TOKEN" \
  --mount source=MY_TARGET_VOLUME,target=/data \
  ghcr.io/suda/dvsync-client

Using local source/target

Alternatively, if you want to copy this data to your local machine, you can mount a host directory as well:

docker run --rm -e DVSYNC_TOKEN="$DVSYNC_TOKEN" \
  -v $PWD:/data \
  ghcr.io/suda/dvsync-client

This can also be done other way around, when you start the server locally if you need to copy local data into the data center.

Docker Compose / Swarm

Start a server where you want to copy data from:

version: '3.6'
services:
  dvsync-server:
    image: 'ghcr.io/suda/dvsync-server'
    environment:
      NGROK_AUTHTOKEN: ${NGROK_AUTHTOKEN}
    volumes:
      - type: volume
        source: MY_SOURCE_VOLUME
        target: /data
        read_only: true
volumes:
  MY_SOURCE_VOLUME:

Once the server started, look into the logs and copy the DVSYNC_TOKEN
Start the client where you want tot copy the data to:

version: '3.6'
services:
  dvsync-server:
    image: 'ghcr.io/suda/dvsync-client'
    environment:
      DVSYNC_TOKEN: ${DVSYNC_TOKEN}
    volumes:
      - type: volume
        source: MY_TARGET_VOLUME
        target: /data
volumes:
  MY_SOURCE_VOLUME:

Kubernetes

Start a server where you want to copy data from:

apiVersion: v1
kind: Pod
metadata:
  name: dvsync-server
spec:
  containers:
  - image: ghcr.io/suda/dvsync-server
    name: dvsync-server
    env:
    - name: NGROK_AUTHTOKEN
      value: "REPLACE WITH YOUR NGROK_AUTHTOKEN"
    volumeMounts:
    - mountPath: /data
      name: MY_SOURCE_VOLUME
  volumes:
  - name: MY_SOURCE_VOLUME

Once the server started, look into the logs and copy the DVSYNC_TOKEN
Start the client where you want tot copy the data to:

apiVersion: v1
kind: Pod
metadata:
  name: dvsync-client
spec:
  containers:
  - image: ghcr.io/suda/dvsync-client
    name: dvsync-client
    env:
    - name: DVSYNC_TOKEN
      value: "REPLACE WITH YOUR DVSYNC_TOKEN"
    volumeMounts:
    - mountPath: /data
      name: MY_TARGET_VOLUME
  volumes:
  - name: MY_TARGET_VOLUME

Contributing

All contributions (no matter if small) are always welcome.

To see how you can help and where to start see Contributing file.