Daniel Hnyk

8 min read Original article ↗

I run Mattermost for my four-person household. It replaced Slack after Slack started deleting message history on the free tier, and it has worked well ever since. Until I was reading through the deprecated features page while planning an upgrade to v11 and found that they had removed Bleve search.

Mattermost dropped the pure-Go embedded search engine that had served small self-hosted instances for years. The replacement path requires Elasticsearch and the enterprise license (unsure about the latter, but I think it's true). That is a lot of infrastructure for a household chat server, and the community is frustrated.

I am grateful for Mattermost. It is well-engineered software that has handled years of our family chat and file sharing without drama. Mattermost is a company with employees, they need to make money, and I am not paying them anything. But I am also not going to pay for an Enterprise license for four people.

The thing about open source

But the software is AGPL, and the Bleve code was still there in the git history, in the v10.5.0 tag. The SearchEngineInterface contract had not changed between v10 and v11. The integration points, the config struct, the broker wiring were all stable. Restoring Bleve from v10 into a v11 fork was a well-defined surgical task. Claude Code enters the room...

...it resurrected the Bleve engine in ten minutes. Two thousand five hundred lines of Go (which I don't understand nor can code in), wired back into a codebase it was removed from, tests passing. A few years ago a senior Go engineer who knew the codebase would have spent a careful afternoon on this. I supervised while Claude Code found the right files in the v10 tag, identified the integration points in v11, made the additive changes, updated go.mod, and wrote a commit message explaining the reasoning.

Open source plus capable AI coding tools let a single person maintain far more than before. Code that would have required a Go contractor or an upstream patch and a long wait, you can do yourself in minutes, if the license permits it. The "if the license permits it" part is doing a lot of work there. AGPL made this possible. A proprietary codebase would not have.

What the fork does

The fork is hnykda/mattermost on the bleve-restore branch. It is based on the v11.5.x tag with exactly these changes over upstream:

  • server/platform/services/searchengine/bleveengine/ restored from v10.5.0
  • server/platform/services/searchengine/searchengine.go adds BleveEngine to the Broker struct
  • server/public/model/config.go adds the BleveSettings struct and wires it into config defaults
  • server/go.mod adds github.com/blevesearch/bleve/v2

The webapp and client come from the official Mattermost release tarball unchanged. Only the server binary is compiled from source, which keeps build times reasonable. The Dockerfile has three stages: compile the binary from the fork, download the official release tarball and swap the binary, copy into a distroless runtime image:

ARG MM_VERSION=11.5.1
ARG GO_VERSION=1.24

# Stage 1: Build server binary from the fork
FROM golang:${GO_VERSION}-bookworm AS builder
WORKDIR /build
COPY mattermost/server/ .
RUN --mount=type=cache,target=/root/go/pkg/mod \
    go mod download
RUN --mount=type=cache,target=/root/go/pkg/mod \
    --mount=type=cache,target=/root/.cache/go-build \
    GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build \
        -trimpath -tags production \
        -o bin/linux_amd64/mattermost \
        ./cmd/mattermost

# Stage 2: Download official release tarball, replace binary
FROM ubuntu:noble AS package
ARG MM_VERSION
RUN apt-get update && apt-get install -y ca-certificates curl && rm -rf /var/lib/apt/lists/*
RUN curl -fsSL \
    "https://releases.mattermost.com/${MM_VERSION}/mattermost-${MM_VERSION}-linux-amd64.tar.gz" \
    | tar -xz \
    && groupadd --gid 2000 mattermost \
    && useradd --uid 2000 --gid 2000 --home-dir /mattermost mattermost \
    && mkdir -p /mattermost/data /mattermost/plugins /mattermost/client/plugins \
    && chown -R mattermost:mattermost /mattermost
COPY --from=builder /build/bin/linux_amd64/mattermost /mattermost/bin/mattermost

# Stage 3: Minimal distroless runtime
FROM gcr.io/distroless/base-debian12
ENV PATH="/mattermost/bin:${PATH}"
COPY --from=package --chown=2000:2000 /mattermost /mattermost
USER mattermost
EXPOSE 8065
CMD ["/mattermost/bin/mattermost"]

The fork is included as a git submodule so CI checks out the right commit of the Bleve-restored branch alongside the Dockerfile.

There is no Bleve section in the System Console on Team Edition. The Elasticsearch UI is enterprise-only and Mattermost uses the same UI for both engines, so neither appears. You configure Bleve through environment variables:

MM_BLEVESETTINGS_INDEXDIR=/mattermost/data/bleve-indexes
MM_BLEVESETTINGS_ENABLEINDEXING=true
MM_BLEVESETTINGS_ENABLESEARCHING=true
MM_BLEVESETTINGS_ENABLEAUTOCOMPLETE=true

To populate the index for existing messages, you trigger a bulk indexing job through the API using a personal access token (Profile -> Security -> Personal Access Tokens):

curl -X POST https://your-mattermost-url/api/v4/jobs \
  -H "Authorization: Bearer <your-token>" \
  -H "Content-Type: application/json" \
  -d '{"type":"bleve_post_indexing"}'

New messages are indexed automatically as they arrive.

Deploying on Kubernetes

I run this on a single-node Kubernetes cluster using the official Mattermost Helm chart with a custom image built from the fork. A few things in the values file need explaining because they are not obvious.

The deployment strategy has to be Recreate rather than RollingUpdate. Bleve uses file locks on the index directory, so if a new pod starts before the old one terminates, it will fail trying to open the index files. Recreate kills the old pod first and accepts a short downtime window. Fine for a household instance.

The other non-obvious part: the official image uses a distroless base, which has no working directory. Relative paths like ./data/ resolve to /data/ on the read-only root filesystem and fail silently. All path settings must be absolute:

image:
  repository: registry.your-host/mattermost-custom
  # tag is always set by CI (sha-<commit>); no default so accidental deploys fail loudly
  imagePullPolicy: Always

mysql:
  enabled: false

externalDB:
  enabled: true
  externalDriverType: "postgres"
  # Chart prepends "postgres://" so omit it here
  externalConnectionString: "user:pass@your-postgres:5432/mattermost?sslmode=disable"

persistence:
  data:
    enabled: true
    size: 10Gi
    storageClass: "database-storage"  # Retain policy to survive helm uninstalls
    accessMode: ReadWriteOnce
  plugins:
    enabled: true
    size: 2Gi
    storageClass: "database-storage"
    accessMode: ReadWriteOnce

# Recreate strategy required: Bleve uses file locks on the index directory.
# RollingUpdate would cause the new pod to fail opening the index while the
# old pod still holds the lock.
deploymentStrategy:
  type: Recreate

extraEnvVars:
  - name: MM_SERVICESETTINGS_SITEURL
    value: "https://mattermost.your-domain.com"

  # Absolute paths required - distroless has no WORKDIR so relative
  # paths (./data/, ./plugins/) resolve to / on the read-only root filesystem
  - name: MM_FILESETTINGS_DIRECTORY
    value: "/mattermost/data/"
  - name: MM_PLUGINSETTINGS_DIRECTORY
    value: "/mattermost/plugins/"
  - name: MM_PLUGINSETTINGS_CLIENTDIRECTORY
    value: "/mattermost/client/plugins"

  # Bleve - indexes stored in the persistent data volume, survive pod restarts
  - name: MM_BLEVESETTINGS_INDEXDIR
    value: "/mattermost/data/bleve-indexes"
  - name: MM_BLEVESETTINGS_ENABLEINDEXING
    value: "true"
  - name: MM_BLEVESETTINGS_ENABLESEARCHING
    value: "true"
  - name: MM_BLEVESETTINGS_ENABLEAUTOCOMPLETE
    value: "true"

securityContext:
  readOnlyRootFilesystem: true
  runAsNonRoot: true
  runAsUser: 2000
  fsGroup: 2000

# /tmp must be writable even with a read-only root filesystem
extraVolumes:
  - name: tmp
    emptyDir: {}
extraVolumeMounts:
  - name: tmp
    mountPath: /tmp

The above covers the non-obvious parts. You would also want ingress with TLS, resource limits, and health probe timings adjusted for a slower startup, but those are standard.

One more thing: tuning it for our household

Once search was working, I noticed an annoyance. The restored Bleve code uses the standard analyzer, which tokenizes at word boundaries and lowercases. It works, but searching "running" won't find "run", and common filler words like "the" or "is" get indexed and matched against.

More relevant for us: our household messages are a mix of English and Czech. "A", "ale", "bez", "bylo", Czech stop words that appear in most sentences and add noise to search results.

Since I had the fork open, fixing this was a small change. Bleve ships with an en analyzer (Porter stemming, English stop words, possessive filter) and a cs stop word list, both bundled in the same package with no extra dependencies. I wrote a custom en_cs analyzer that combines them:

  • Unicode tokenization
  • English possessive filter ('s)
  • Lowercase
  • English stop words
  • Czech stop words
  • Porter stemmer

Twenty-odd lines of Go, and now the search index is tuned for how we write. I keep thinking about this part. I restored something that was removed, and then I kept going and made it better for my specific situation. No upstream would prioritize a four-person Czech-English household, but it was within reach once I was in the codebase.

The new mapping only applies to fresh indexes, so after deploying I had to purge the old ones and re-index. The purge API was also something restored in the fork. Full circle.

Is it worth it

Search works now, in both languages. Jack can find messages by content, stems match for English, and Czech filler words no longer dominate the results. The index files have survived several pod restarts and a few deploys.

The upgrade path for the next Mattermost version is a rebase:

git fetch upstream
git checkout bleve-restore
git rebase v11.x.0
git push origin bleve-restore --force-with-lease

The integration points are stable. Conflicts are unlikely. CI rebuilds and redeploys.