Migrate a Peertube instance from Debian to FreeBSD

12 min read Original article ↗

       2417 words, 12 minutes

Not matter how hard I try, I can’t stand and learn those systemd and ip censored swearing stuff on Linux. Call me an old fart with muscle memory amnesia but I just want to manage my servers using rc scripts, ifconfig and route commands, and read my logs in syslog flat format. Anyway, my Peertube instance was running on Debian and I wanted to use some BSD instead of Linux.

So I decided to run Peertube on FreeBSD , while migrating the account and its data from the already running Debian instance.

FreeBSD installation

FreeBSD is installed as a bhyve machine on my OmniOS hypervisor .

It uses three (3) virtual disks so that I can manage the OS, database and Peertube storage independently. You don’t have to, but I already encountered some need for more space and it is easier to manage this way.

I also use two (2) network interfaces so separate Peertube traffic from Admin flows. You don’t have to.

Creating the FreeBSD instance is a simple as downloading the installation ISO file, creating the bhyve configuration file and starting the bhyve zone.

# wget -O /zones/iso/FreeBSD-14.3-RELEASE-amd64-bootonly.iso \
  https://download.freebsd.org/releases/amd64/amd64/ISO-IMAGES/14.3/FreeBSD-14.3-RELEASE-amd64-bootonly.iso

# zadm create -b bhyve peertube
{
   "autoboot" : "true",
   "bootdisk" : {
         "blocksize" : "8k",
         "path" : "tank/zones/peertube/root",
         "size" : "16G",
         "sparse" : "false"
   },
   "brand" : "bhyve",
   "cdrom" : [ "/zones/iso/FreeBSD-14.3-RELEASE-amd64-bootonly.iso" ],
   "disk" : [
      {
         "blocksize" : "8K",
         "path" : "tank/zones/peertube/pgsql",
         "size" : "5G",
         "sparse" : "false"
      },
      {
         "blocksize" : "8K",
         "path" : "tank/zones/peertube/data",
         "size" : "256G",
         "sparse" : "false"
      }
   ],
   "ip-type" : "exclusive",
   "net" : [
      {
         "global-nic" : "igb0",
         "mac-addr" : "fe:ed:ca:fe:00:01",
         "physical" : "peertube0"
      },
      {
         "global-nic" : "admin0",
         "physical" : "peertube1"
      }
   ],
   "ram" : "16G",
   "rng" : "on",
   "vcpus" : "8",
   "vnc" : "off",
   "zonename" : "peertube",
   "zonepath" : "/zones/peertube"
}
# zadm start -c peertube

FreeBSD then boots and ends asking for the console type. I used “xterm” for the fun but “vt100” is probably is safer option, even for bhyve console through SSH.

Starting secondary installer on ttyv0
Starting primary installer on ttyu0

Welcome to FreeBSD!

Please choose the appropriate terminal type for your system.
Common console types are:
   ansi     Standard ANSI terminal
   vt100    VT100 or compatible terminal
   xterm    xterm terminal emulator (or compatible)

Console type [vt100]: xterm

From the install wizard, select “Install” and proceed as usual.

I choose the “Auto (UFS)” disk partition layout. Yes, UFS, not ZFS!
The main reason is OmniOS already uses ZFS. And, as advised on the Fediverse, ZFS over ZFS is probably a bad idea.

The OS is installed on vtbd0, the first (16GB) disk.
The partition scheme is “GPT”; because this is what bhyve on OmniOS offers.

When the installation is complete, select “Shutdown”, quit the console and remove the ISO from the zone config. The VM can now be started as normal. The serial console is available by default, without specifying anything at the boot loader prompt.

Additional storage

I didn’t bother setting up all disks from the installation wizard. So after reading the Adding Disks section of the FreeBSD Handbook, I connected to the server using SSH and configured the database and Peertube storage.

First, learn about the available disks:

# sysctl kern.disks
kern.disks: cd0 vtbd2 vtbd1 vtbd0

Configure the first disk to host the future database content:

# gpart show vtbd1
gpart: No such geom: vtbd1.

# gpart create -s BSD vtbd1
vtbd1 created

# gpart add -t freebsd-ufs vtbd1
vtbd1a added

# gpart show vtbd1
=>       0  10485760  vtbd1  BSD  (5.0G)
         0  10485760      1  freebsd-ufs  (5.0G)

# newfs -Uj /dev/vtbd1a
/dev/vtbd1a: 5120.0MB (10485760 sectors) block size 32768, fragment size 4096
        using 9 cylinder groups of 625.22MB, 20007 blks, 80128 inodes.
        with soft updates
super-block backups (for fsck_ffs -b #) at:
 192, 1280640, 2561088, 3841536, 5121984, 6402432, 7682880, 8963328, 10243776
Using inode 4 in cg 0 for 41943040 byte journal
newfs: soft updates journaling set

Configure the second disk to host the futur Peertube content:

# gpart show vtbd2
gpart: No such geom: vtbd2.

# gpart create -s BSD vtbd2
vtbd2 created

# gpart add -t freebsd-ufs vtbd2
vtbd2a added

# gpart show vtbd2
=>        0  268435456  vtbd2  BSD  (128G)
          0  268435456      1  freebsd-ufs  (128G)

# newfs -Uj /dev/vtbd2a
/dev/vtbd2a: 131072.0MB (268435456 sectors) block size 32768, fragment size 4096
        using 210 cylinder groups of 625.22MB, 20007 blks, 80128 inodes.
        with soft updates
super-block backups (for fsck_ffs -b #) at:
 192, 1280640, 2561088, 3841536, 5121984, 6402432, 7682880, 8963328, 10243776, 11524224,
(...)
 258650688, 259931136, 261211584, 262492032, 263772480, 265052928, 266333376, 267613824
Using inode 4 in cg 0 for 655589376 byte journal
newfs: Journal file fragmented.
newfs: soft updates journaling set

Finally configure the storage to be mounted to the relevant place:

# mkdir -p /var/db/postgres /var/www/peertube

# vi /etc/fstab
(...)
/dev/vtbd1a     /var/db/postgres        ufs     rw      1       2
/dev/vtbd2a     /var/www/peertube       ufs     rw      1       2

# mount -t ufs -a

# mount | grep ufs
/dev/vtbd0p2 on /                 (ufs, local, soft-updates, journaled soft-updates)
/dev/vtbd1a  on /var/db/postgres  (ufs, local, soft-updates, journaled soft-updates)
/dev/vtbd2a  on /var/www/peertube (ufs, local, soft-updates, journaled soft-updates)

# df -h | awk 'NR==1 || /vtb/'
Filesystem      Size    Used   Avail Capacity  Mounted on
/dev/vtbd0p2     14G    2.3G     11G    18%    /
/dev/vtbd0p1    260M    1.3M    259M     1%    /boot/efi
/dev/vtbd1a     4.8G     40M    4.4G     1%    /var/db/postgres
/dev/vtbd2a     124G    625M    113G     1%    /var/www/peertube

Sending emails

I am using the DragonFly Mail Agent (DMA) to send daily emails via a remote authenticated SMTP relay.

# cat > /etc/dma/dma.conf
SMARTHOST relay.example.com
PORT 587
AUTHPATH /etc/dma/auth.conf
SECURETRANSFER
STARTTLS
^D

# cat > /etc/dma/auth.conf
change_user|relay.example.com:change_passwd
^D
# chmod 0640 /etc/dma/auth.conf

# grep '^root:' /etc/mail/aliases
root:   hostmaster@example.com

Filtering packets

The server is exposed on the Wild Wild Web so I like to add a bit of filtering on the network interfaces.
Of course, using pf , because What else?
The configuration is done using the bhyve console to avoid chopping my own legs…

# sysrc pf_enable=YES
# sysrc pflog_enable=YES

# vi /etc/pf.conf
(...)
pass in on $wan_if proto tcp         \
  from any to ($wan_if) port ssh     \
  label "in-$dstport" modulate state
pass in on $wan_if proto tcp         \
  to ($wan_if) port { http, https }  \
  label "in-$dstport" modulate state
(...)
# service pf start
# service pflog start

Peertube dependencies

Read the Production Guide to learn about the official way to install and configure Peertube.

The required binary packages are installed using pkg:

# pkg install -y sudo bash wget git python nginx pkgconf \
  postgresql17-server postgresql17-contrib redis openssl \
  node npm yarn ffmpeg unzip

I went for PostgreSQL 17 rather than 13 because as-of-today, 17 has a further EOL date.

Redis

Redis works out of the box.

# sysrc redis_enable="YES"
# service redis start

Sometimes, Redis has a lower default maxclients value than what Peertube expects. This leads to Redis errors described here . On FreeBSD, the defaults are ok for using with Peertube.

# redis-cli config get maxclients
1) "maxclients"
2) "10000"

PostgreSQL

When PostgreSQL is being installed, a detailed pkg-message is displayed to help you start with it.

# vi /etc/login.conf
(...)
postgres:\
        :lang=en_US.UTF-8:\
        :setenv=LC_COLLATE=C:\
        :tc=default:

# cap_mkdb /etc/login.conf

# sysrc postgresql_enable="YES"
# sysrc postgresql_login_class="postgres"

# service postgresql initdb --data-checksums
# service postgresql start

Peertube uses a dedicated database and PostgreSQL user.

# sudo -u postgres createuser -P peertube
Enter password for new role:        << same as in production.yaml
Enter it again:                     << same as in production.yaml

# sudo -u postgres createdb -O peertube -E UTF8 -T template0 peertube_prod

# sudo -u postgres psql -c "CREATE EXTENSION pg_trgm;" peertube_prod
# sudo -u postgres psql -c "CREATE EXTENSION unaccent;" peertube_prod

I do my PostgreSQL backup and maintenance via an rsnapshot mechanism.
But the FreeBSD package provides a generic script you may want to use.

The port is set up to use autovacuum for new databases, but you might also want to vacuum and perhaps backup your database regularly. There is a periodic script, /usr/local/etc/periodic/daily/502.pgsql, that you may find useful. You can use it to backup and perform vacuum on all databases nightly. Per default, it performs `vacuum analyze’. See the script for instructions. For autovacuum settings, please review ~postgres/data/postgresql.conf.

Peertube installation

As I do on OpenBSD, I tend to run daemons as an unprivileged user. So let’s create a dedicated one for Peertube.

# pw group add -n _peertube -g 2000
# pw  user add -n _peertube -u 2000         \
  -c "Peertube daemon" -d /var/www/peertube \
  -g 2000 -s /usr/sbin/nologin

# chown _peertube:_peertube /var/www/peertube

Switch to the _peertube user to install Peertube.

# sudo -u _peertube bash -l

Check the latest available version.

$ VERSION=$(curl -s https://api.github.com/repos/chocobozzz/peertube/releases/latest | \
  grep tag_name | cut -d '"' -f 4) && \
  echo "Latest Peertube version is $VERSION"
Latest Peertube version is v7.2.3

Create the required directories

$ cd /var/www/peertube
$ mkdir config storage versions
$ chmod 750 config/

Download and extract the Peertube archive.

$ cd /var/www/peertube/versions
$ wget -q "https://github.com/Chocobozzz/PeerTube/releases/download/${VERSION}/peertube-${VERSION}.zip"
$ unzip -q peertube-${VERSION}.zip
$ rm peertube-${VERSION}.zip

Install Peertube

$ cd /var/www/peertube
$ ln -s versions/peertube-${VERSION} ./peertube-latest
$ cd ./peertube-latest
$ npm run install-node-dependencies -- --production

From there, Peertube can be run using a node incantation. But I’d rather use an rc.d script and Peertube provides one.

# install -m 0555                                            \
  /var/www/peertube/peertube-latest/support/freebsd/peertube \
  /usr/local/etc/rc.d/

# mkdir /var/log/peertube
# chown -R _peertube:_peertube /var/log/peertube

# sysrc peertube_enable="YES"

Because I’m using the _peertube unprivileged user, I had to edit the rc.d script accordingly. Peertube can then be managed from the rc.d script.

# service peertube start

Web publication

Peertube only listens on localhost. To expose the service on the Wild Wild Web, I’m using nginx; the relayd port looks a bit old.

Let’s Encrypt certificate

The TLS certificate is obtained from Let’s Encrypt . Following the Fediverse recommendation, I decided to use acme.sh to maintain the TLS certificate.

# pkg install acme.sh

# mkdir -p /etc/ssl/private /var/www/acme

# acme.sh --server letsencrypt --issue --webroot /var/www/acme \
  --domain peertube.example.com                                \
  --key-file /etc/ssl/private/peertube.example.com.key         \
  --fullchain-file /etc/ssl/peertube.example.com.crt

# crontab -e
# min hour dmonth month dweek command
0-30  2-4  *      *     *     /usr/local/sbin/acme.sh --cron       \
                              --renew-hook "service nginx restart" \
                              >/dev/null

Both FreeBSD and acme.sh have opinions on where to store the certificates and how to deal with /.well-known/acme-challenge/. But memory muscle decided I would use the same locations I’m used to with OpenBSD…

nginx

Peertube provides a ready-to-use nginx configuration file.

The problem is it lacks some extra config bits I want to use. And its format doesn’t make it easy to just include it. So I simply used it as-is and added a few stuff at its beginning. On upgrade times, using diff or vimdiff makes it trivial how to apply new default parameters.

# mkdir /var/www/htdocs
# chown www:www /var/www/htdocs

# cp /var/www/peertube/peertube-latest/support/nginx/peertube \
     /usr/local/etc/nginx/nginx.conf
# vi /usr/local/etc/nginx/nginx.conf

When done, enable and start nginx.

# sysrc nginx_enable="YES"
# service nginx start

Et voilà! If you’re starting from 0, you should have a brand new Peertube instance running on FreeBSD. 🥳

Data migration

The official Peertube Migration documentation is available here . It is probably worth reading it… like, do it!-)

So I had my old Debian instance already running, with active synchronizations. The new FreeBSD instance was installed but not started until I went through the following steps:

  • I copied the old (Debian) Peertube configuration to the new (FreeBSD) server.
  • I fully synchronized the to-be-old (Debian) Peertube storage directory to the new (FreeBSD) Peertube server.
  • I waited a couple of hours to complete the sync.
  • I decided on the day1 to proceed to the switch and stopped the to-be-old (Debian) nginx and peertube processes.
  • I synchronized the to-be-old (Debian) Peertube storage directory to the new (FreeBSD) Peertube server a last time.
    This time, it only tooks seconds.
  • I transferred the to-be-old (Debian) Peertube database content to the new server and imported the data into (FreeBSD) PostgreSQL.
  • I turned down the Debian network interface and turned on the FreeBSD one.
    Yes, both use the same IP rather than using a DNS switch.
  • I started peertube and nginx on the new (FreeBSD) server.
  • I kept a eye on the logs and the Web GUI to see if everything looked ok.
    Spoiler alert: it did.

1 Fun fact: it turned out that I switched from Debian to FreeBSD on the exact Debian 13 Release day.

Copying Peertube configuration

The configuration copy was done using scp and didn’t require any modification.

# scp -p debian.example.com:/var/www/peertube/config/production.yaml \
  /var/www/peertube/config/production.yaml

Full storage data transfer

To minimize downtime, copy the storage data while the old server is still working. Do this as many time as required.

# rsync -avz --delete \
  debian.example.com:/var/www/peertube/storage/ \
  /var/www/peertube/storage/

Incremental data transfer and Peertube switch

When ready to migrate the service, with downtime, stop the old Peertube services and transfer every data to the new server. I stopped nginx too so that no HTTP/501 were send to clients and I don’t have to change the configuration; for easier rollback, should this be required.

(old peertube)# systemctl stop nginx
(old peertube)# systemctl stop peertube

Synchronize the /storage/ content one last time:

# rsync -avz --delete \
  debian.example.com:/var/www/peertube/storage/ \
  /var/www/peertube/storage/
# chown -R _peertube:_peertube /var/www/peertube/storage

Dump and restore PostgreSQL data from the old to the new server:

(old peertube)# sudo -u peertube pg_dump -Fc peertube_prod > \
                /tmp/peertube_prod-dump.db

(new peertube)# scp -p debian.example.com:/tmp/peertube_prod-dump.db \
                /tmp/peertube_prod-dump.db
(new peertube)# sudo -u postgres pg_restore -c -C \
                -d postgres /tmp/peertube_prod-dump.db

Start the services and check the logs:

# service peertube start
# service nginx start

# tail -f /var/log/peertube/peertube.log
# tail -f /var/log/nginx/error.log

One more thing

I configured newsyslog so that logs rotation happens every night.

# cat /usr/local/etc/newsyslog.conf.d/peertube.conf
# nginx configuration file for newsyslog.conf
#
# logfilename                   mode count size when flags [/pid_file] [sig_num]
/var/log/nginx/error.log        640  7     *    $D0  BZ    /var/run/nginx.pid SIGUSR1
/var/log/nginx/access.log       640  7     *    $D0  BZ    /var/run/nginx.pid SIGUSR1
/var/log/peertube/peertube.log  640  7     *    $D0  BZ

It is probably smarter to use Jails to isolate the various services. But I was not sure Peertube would work properly so I did a “simple” install. Some time in the future, I’ll move each services in their dedicated jail.

Peertube provides telemetry out of the box. It is enable from its configuration file.

# vi /var/www/peertube/config/production.yaml
(...)
open_telemetry:
  metrics:
    enabled: true
  prometheus_exporter:
    hostname: '192.0.2.666'
    port: 9091
# service peertube restart

I used VictoriaMetrics Agent to grab those metrics and store them into VictoriaMetrics. Then used Grafana to render them. As of now, the dashboard looks bad. I may post more about it when it is finished.

After a couple of weeks, I have not had any trouble. It runs as good as the previous Linux instance.

Now your turn!