How this site runs · Mostly Operational

The rules I gave myself

• Private. No public Kubernetes API, no inbound ports, no public load balancer for traffic.
• Under €100 a month for all of Azure, with alerts before it gets close.
• Everything as code. Terraform for Azure and Cloudflare, Git for what runs in the cluster.
• No secrets in Git. Not in manifests, not in tfvars, not in CI.
• As few upstream apps in the cluster as possible. No ingress controller, cert-manager or External Secrets operator: the Cloudflare Tunnel and the Secrets Store CSI driver add-on for Key Vault do those jobs. Where an AKS add-on is light, I use it instead of installing and patching a chart myself. Where it isn't, a small upstream install wins, because everything shares one 4 GiB node: the Flux extension's two Azure agents took about 90 MiB, the KEDA add-on reserved 420m of CPU and 420 MiB, and the workload identity webhook 200m of CPU to add four environment variables and a token to a pod (measured 31 October and 1 November 2026). Flux now runs just its two controllers, KEDA one replica of each part without webhooks, and pods set up their identity themselves.

How a request gets here

DNS for kalinowski.io points at Cloudflare, which terminates TLS and serves cached static assets (images, CSS, JavaScript) from the edge. Pages and cache misses go down a Cloudflare Tunnel. Two cloudflared pods inside the cluster dial out to Cloudflare and keep that connection open, so the cluster never accepts an inbound connection from the internet. The tunnel routes by path, most specific first: /api/status, /api/ask, /api/comments and /api/newsletter/ go to their own Go services, the rest of/api/ to the contact form, and everything else to nginx serving the static build of this site. Every public route is described in the API reference, which the services' own tests keep honest. Inside the cluster, Cilium NetworkPolicies only let cloudflared reach these pods. The exceptions are listed one by one: status-api may ask comments-api for its counts, two services may read from the website pod, and the site's own pods may send their logs and traces to the collector.

What runs in the cluster

Workload	What it does	Talks to
www	nginx serving the static Astro build (2 replicas)	nothing
contact-api	Contact form	Azure Communication Services Email
status-api	Data for the live status page	Kubernetes API (read-only), Azure Cost Management, Cloudflare analytics, Google Analytics
ask-api	The Ask AI assistant	Cloudflare Workers AI, the site's own text
comments-api	Blog comments and their moderation	Azure Table Storage
newsletter-api	Newsletter sign-up, unsubscribe and delivery reports	Azure Table Storage, Azure Communication Services Email
newsletter-digest	CronJob, hourly: reminders, removing unconfirmed sign-ups and bounced addresses, and queuing the Monday digest (the same image as newsletter-api)	Azure Table Storage and Queue Storage
newsletter-send	A Job that KEDA starts when mail is waiting in the queue: sends it (the same image again)	Azure Queue Storage, Azure Communication Services Email
table-backup	CronJob, Sundays 02:17 UTC: copies the comments and newsletter tables to blobs	Azure Table and Blob Storage
otel-collector	Node and pod metrics, and the services' logs and traces	Kubernetes API (read-only), Grafana Cloud
cloudflared	The outbound tunnel (2 replicas)	Cloudflare
flux	Applies the config repo to the cluster, its own install included (two controllers: source and kustomize)	GitHub (read-only deploy key), Kubernetes API
kyverno	Refuses pods in the site's namespace whose images aren't signed by this site's build (admission controller only)	Kubernetes API, Container Registry
keda	Scales the Scale to zero demo's pods between 0 and 5, and starts the newsletter's send job	Kubernetes API, Azure Queue Storage (read-only)
headlamp	An owner-only cluster UI, behind Cloudflare Access and Entra sign-in. The pod has no rights of its own: every call uses the signed-in user's token	Kubernetes API

The five APIs and the backup job are small Go programs on distroless images. They run as non-root with read-only filesystems and idle at a few MiB of memory each. That matters on a single 4 GiB node that also runs Flux, Cilium and the CSI driver.

How changes get deployed

The cluster's API server has no public endpoint, so CI can't push to it. Instead the cluster pulls. The site repo has one folder per image: the website, the five Go services, the backup job and the metrics collector.

• Every change is a pull request. GitHub Actions gives each image a tag from its own content (a hash of its folder), builds only the images whose tag doesn't exist yet, and pushes them, privately, to Azure Container Registry. It signs in with a federated identity: Entra trusts the repo's GitHub tokens, so CI holds no Azure secret, and it can push images and nothing else. Before a pull request is opened, the same security scans CI runs go first on my laptop: Trivy for the Terraform, the Dockerfiles, npm and stray secrets, govulncheck for the Go services. Posts get checked for broken links, missing alt text and grammar. When the website changes, its image is started and checked like a visitor would see it: every internal link and anchor, accessibility (axe), sideways scrolling on a phone, layout shift and how much JavaScript and CSS each kind of page loads. A typo fix in a post builds the website and nothing else.
• Merging reuses those images. The content doesn't change in a squash-merge, so neither does the tag, and main goes straight to the release and the deploy.
• Every image is signed. The release signs each image with Notation, using a key that was generated inside its own Key Vault and can't be exported: CI asks the vault to sign and never holds the key. The certificate is self-signed on purpose, because only my own checks need to trust it. The deploy verifies every signature before it bumps a tag, and inside the cluster Kyverno refuses any pod in the site's namespace whose image isn't signed with that key. That check fails closed: if Kyverno can't verify a signature, new pods aren't admitted, while the ones already running carry on, so a Kyverno outage blocks rollouts, not the site. Each new version's release page gets a software bill of materials (CycloneDX, from Trivy). Released images, their SBOMs and signatures are locked in the registry, so a release can't be overwritten or deleted.
• Every component has a version. Commit messages follow conventional commits, so each changed component gets its own release (website-1.0.0, status-api-1.0.0) with notes grouped into new posts, features, fixes and maintenance. The status page shows those versions.
• Once a week, everything is rebuilt anyway. Content tags have a blind spot: an image whose code hasn't changed is never built again, so a security fix in its base image (Alpine for nginx, distroless for the Go services) would only arrive with my next commit to it. Every Sunday at 03:23 UTC a scheduled run builds every image without the layer cache, on freshly pulled base images, and adds the ISO week to the tag (-r202643), so it's a new image for the same code. Each one is released as a patch version and rolled out like any other change. The first run, on 25 October, started five hours late (GitHub runs scheduled jobs when it has room), rebuilt all eight images in nine minutes and deployed them.
• Then the cluster pulls. The workflow bumps the tags in a separate config repo. Flux, which keeps its own install in the same repo, watches it and applies whatever is on main, so only the pods whose tag changed roll. The node pulls the images with its own managed identity, so there's no registry password anywhere. Git is the source of truth, and rolling back means reverting a commit.

From merge to live takes two to three minutes. More on the content-addressed tags, and managing a private cluster without a jumpbox.

Identity and secrets

Every workload that talks to Azure has its own workload identity: a Kubernetes service account federated with an Azure managed identity. Pods get short-lived tokens, and there are no client secrets or connection strings for those Azure API calls. AKS has a webhook that adds the token to each pod; it reserved 200m of CPU for that, so it's off, and each pod that calls Azure mounts its token and sets the few environment variables itself. Each identity gets only the roles it needs:

• contact-api has a custom role that can send email and nothing else.
• status-api has Cost Management Reader on this project's resource groups only.
• comments-api has a data role on one table, not the storage account.
• newsletter-api can add and remove subscribers but has no permission to read them; only the digest job, with its own identity, can read the list it sends to.
• table-backup can read the tables it backs up and create blobs, but not read or delete them, so it can't erase an earlier backup.

The same trick works across clouds. status-api reads Google Analytics with Workload Identity Federation: Google trusts the cluster's OIDC issuer, so the pod swaps its own Kubernetes token for a short-lived Google token. There's no service-account key file anywhere.

The few stored secrets live in Azure Key Vault, which allows only Entra ID with Azure RBAC. Its firewall admits the cluster's subnet and my current IP, nothing else, and the storage account with the Terraform state is locked down the same way:

• the tunnel token and the Turnstile key;
• Cloudflare API tokens for analytics (read-only) and for Workers AI;
• Grafana Cloud tokens: the status page's (read metrics, mark deploys) and the collector's (write metrics);
• the key Event Grid presents when it delivers the newsletter's delivery reports.

Pods read them through the Secrets Store CSI driver. Terraform writes the Cloudflare-generated ones directly, so their values don't need to be copied by hand into a pipeline or a local config file.

The contact form posts to contact-api, a small Go service. Before anything is sent, it checks the request's origin, a honeypot field, the input (including header-injection attempts), per-visitor and global rate limits, and a Cloudflare Turnstile token verified server-side. Then it sends the message through Azure Communication Services Email, authenticated with its workload identity and a custom role that can only send mail. It sends as [email protected], a verified subdomain with its own SPF and DKIM records, so the apex records my Outlook mail depends on are untouched. The full story.

The live status page

/status shows the platform as it runs, from status-api:

• node health, pod counts, deployed versions, and whether Flux is in sync with Git;
• this month's Azure cost against the budget;
• Cloudflare edge traffic next to Google Analytics, side by side;
• comment counts;
• the site seen from outside: whether each check passes in each location, uptime, the certificate, and 24 hours of response times.

Each source refreshes on its own schedule, and failures back off instead of hammering an API. A source that's down only makes its own tiles stale, and the slower sections say when they were last updated. The Kubernetes access is a read-only ClusterRole. The page never shows IPs, pod names or resource IDs. How the status page works.

Monitoring from outside

A status page served by the cluster can't report that the cluster is down. So the checks run somewhere else: Grafana Cloud, on the free tier, set up entirely in Terraform. Probes in Frankfurt, London and Stockholm request the home page and the status API every 5 minutes. Every 15 minutes they also ask whether the newsletter can still send and whether every scheduled job (the digest and the weekly backup) succeeded recently. Two failures within 15 minutes send me an alert, and so does a certificate with less than a week left.

The first day of checks was also one of the busiest for changes: 28 deploys between the checks going live (21 October, 14:11 UTC) and 22 October, 12:30 UTC, and 100% uptime from all three locations. A website deploy starts the new pods before it stops the old ones, so there's always one serving. The probes run every few minutes, so a blip shorter than that could slip between them. The live numbers are on the status page.

Inside the cluster, one small OpenTelemetry Collector pushes node and pod metrics to the same Grafana stack every minute. The Go services send it their logs and traces too, over OTLP, so one request can be followed from the website's API through to the Azure and Cloudflare calls it makes, and a backup or a newsletter run shows up as one trace. Kubernetes events from the site's namespaces (a job started, a pod that couldn't be scheduled, KEDA scaling) go to the same logs, so a failed run leaves a trace after its pod is gone. The services push rather than the collector reading log files from the node, so it needs no access to the host. It's the one upstream app added for monitoring, built with only the components it uses. Its Grafana token comes from Key Vault through workload identity, like every other secret. A series on how this is set up starts on 4 January.

Scheduled posts

There's no CMS. A post written ahead of time goes into a queue folder in Git, with its publish date in the front-matter. A nightly GitHub Actions job moves every post whose date has passed into the blog, checks it the same way a pull request would, and commits it, and that commit deploys like any other change. If a check fails, the post stays in the queue and the job opens an issue. A GitOps CMS, and its first night.

Security at the edge

Most of the security work happens before a request reaches the cluster, and all of it is in Terraform or the repo:

• a signed DNS zone (DNSSEC), CAA records that name the only certificate authorities allowed to issue, and DMARC that rejects spoofed mail from every subdomain;
• HTTPS only: plain HTTP is redirected, browsers are told to remember that for a year (HSTS), and TLS below 1.2 is refused;
• a strict Content Security Policy, with every inline script allowed by its hash, plus headers that stop other sites framing these pages or using browser features they don't need.

The After the A record series covers each layer.

Bots, without the JavaScript challenge

Cloudflare's Bot Fight Mode injects a detection script into every page. On a slow phone that script took about 1.5 seconds of processing before anything else could run, so I turned it off, together with the JavaScript detection it depends on. In Lighthouse's mobile test (2 runs per page, 2026-10-23), total blocking time went from 350-630 ms to 0-80 ms and the best-practices score from 79-82 to 100.

What keeps unwanted traffic out instead:

• a WAF custom rule that blocks paths this site never serves, the ones vulnerability scanners try first, so those requests never reach the cluster;
• an edge rate limit on every endpoint that writes something (contact, comments, Ask AI and the newsletter sign-up): 10 requests per 10 seconds per IP, then a block;
• a Cloudflare Turnstile token on every form, verified by the service itself, plus per-visitor limits inside each service;
• Cloudflare Access in front of everything admin, so those pages aren't reachable without a one-time code.

Ask AI

The Ask AI page answers questions about my work in English or Polish, using only this website's text. Every build writes the site's text, split into sections, to a JSON index. ask-apifetches it from the website pod, ranks the sections with BM25 (no vector database), and sends the best few to Qwen3 30B on Cloudflare Workers AI. Questions about cost, uptime, versions or visitors also get the status page's live numbers. It counts the cost Cloudflare reports for each answer against the free daily allowance and stops before it runs out. When it does, the page shows a sleeping Worker until midnight UTC. It has the same guards as the contact form, keeps no question logs, and renders answers as plain text. How it works.

The same index and the live numbers are open to visitors' own assistants too: ask-api also serves a read-only MCP endpoint, so Claude, ChatGPT or an editor can search the posts, read one and check the status page's numbers. Their own model does the answering, so it doesn't use this site's daily allowance. The address is on the Ask AI page.

Comments are the platform's first state. comments-api stores them in Azure Table Storage, in an account with shared keys disabled and a firewall that admits only the AKS subnet, through a free service endpoint. Everything is moderated: a new comment waits until I approve it. The moderation page sits behind Cloudflare Access, which sends a one-time PIN to my email, and comments-api verifies the Access token again itself. Only a name and the comment are stored: no email, no IP address. Comments are shown as plain text. The plan andwhat actually happened.

The newsletter

New posts also go out by email, once a week on Monday, with no mailing-list provider. Signing up is a double opt-in: nothing is sent to an address until someone opens the link in the confirmation mail. The confirmation token travels in the link's #fragment, which browsers never send to the server, so a mail scanner that follows links can't confirm an address on its own.

Subscriber addresses are the only personal data on the platform, so they get their own storage account and the strictest roles. newsletter-api, the part the internet can reach, can add and delete subscribers but can't read a single one: rows are keyed by a hash of each reader's token, so confirming and unsubscribing work without ever listing the table. Only the hourly job can read it. That has a price: the API can't count subscribers either, so the job writes the totals to a stats row on every run.

Mail waits in an Azure Storage Queue. No sending job runs on a timer: KEDA, already in the cluster for the Scale to zero demo, looks at the queue every 30 seconds and starts a send Job only when mail is waiting. The Job is the same Go binary as the API; it takes at most 10 mails, sends them and exits. A Monday digest goes in as batches of 10 that become visible 10 minutes apart, so sending stays near 60 an hour, under Azure Communication Services' limit of 100 an hour on a custom domain (30 a minute, raisable through support; Microsoft Learn, checked 23 October 2026), which the contact form shares. When ACS says no, the mail simply reappears in the queue 10 minutes later. Confirmation mails don't wait for any of this: the API sends them at once and queues them only when ACS asks it to slow down. An hourly CronJob does the rest: reminders, removing unconfirmed sign-ups and bounced addresses, and queuing the digest. Every mail has a one-click unsubscribe header (RFC 8058). Delivery reports come back through Azure Event Grid, the platform's first event-driven piece: a webhook through the same tunnel, with a key from Key Vault, and a hard bounce removes the address. No address has bounced yet, so that last step is tested but not yet seen in production.

A health endpoint fails when the hourly job stops running, keeps failing or mail waits in the queue for three hours, and Grafana checks it every 15 minutes like the rest of the site. Storage costs less than a cent a month, Event Grid stays inside its free 100,000 operations, and ACS charges per mail.

The API

The services' public routes (the status data, comments, Ask AI, the contact form and the newsletter) are described in one OpenAPI 3.1 document, openapi.json. Each service keeps its part in an openapi.yaml next to its code, and its go test sends requests to the real handlers and checks every response against it, so a renamed field or an undocumented status code fails the build. The API reference is generated from that document when the site builds: static HTML, no viewer script.

Analytics and consent

Google Analytics loads only after a visitor says yes in a self-hosted cookie banner (orestbida/cookieconsent, bundled with the site rather than loaded from a third party). Until then nothing is sent to Google; saying no later removes its cookies. That makes Analytics a consenting subset of visitors. The status page shows Cloudflare's numbers, which count every request, right next to Analytics, so the gap is visible. The privacy page lists every cookie.

The backlog

The backlog page comes from the GitHub issues I plan with. They live in the infrastructure repo, with a GitHub Projects board on top for the order of work and the publish dates of upcoming posts. When an issue labelled public changes, a GitHub Actions workflow exports the public ones to a JSON file in the config repo. It uses the workflow's built-in token, so there's no stored token to rotate. Flux turns that file into a ConfigMap mounted into the website pod, so a change shows up a few minutes later without a rebuild. Only each issue's title, labels, dates and a "Public summary" section leave GitHub. How the export works.

Above the issues, the page has a release journal: the latest release of each component, with its change lines. The deploy job writes it as releases.json into the same ConfigMap as the backlog, so a release appears with the deploy that ships it, not a few minutes later. Only the version, the date and the change lines leave GitHub; issue references are removed, because the planning issues are private. The weekly rebuild releases every component on the same morning with no code changes, so those releases fold into one row instead of one per component.

Crawlers and link previews

Cloudflare's managed robots.txt, configured in Terraform, welcomes search engines and AI search assistants but opts out of AI model training. Every page ships schema.org structured data and a link-preview image generated at build time. The sitemap dates each post by its last change, Cloudflare tells search engines when a page changes (Crawler Hints), and the domain is verified in Google Search Console with a DNS record that's also in Terraform.

What it costs

The estimate, from Azure's retail prices for North Europe (checked October 2026). The real bill, updated a few times a day, is on the status page.

An Azure budget alerts at 50% and 80% of actual spend and at 100% of forecast. The big savings come from what isn't there: no paid control-plane tier, no NAT Gateway, no Bastion or jumpbox (the cluster is managed with az aks command invoke and GitOps), and no Container Insights.

Everything outside Azure is on a free tier: Cloudflare (tunnel, Turnstile, Access for up to 50 users, the Workers AI daily allowance), Google Analytics and the Google Cloud project used for federation, Grafana Cloud, and GitHub Actions. Adding the status page, the assistant, comments, monitoring, the newsletter and backups didn't move the total; moving the images to Azure Container Registry, which removed the last registry token, added about €4.50.

Trade-offs I accepted

• One small node, no SLA. If the node is replaced, the site is briefly down. Cloudflare caches static assets but not HTML by default, so pages go down with the node. For a personal site, a few minutes of downtime during maintenance beats paying for a second node. It has headroom, though: in a ten-minute k6 test from my laptop with 100 simulated visitors (1 November 2026) it served 50,491 requests without a failure, at a p95 of 93 ms and a CPU peak of 0.46 of its 2 cores. That's a lower bound, not a capacity figure. The load test.
• A private endpoint I could have skipped. A public API server with an IP allowlist would save about €7 a month. Not having a public control plane at all is worth more than that to me.
• Key Vault on a public endpoint. A private endpoint would cost as much again. Entra ID, RBAC-only access and a firewall that admits the cluster's subnet and one IP are a reasonable line for a handful of non-critical secrets.
• State in memory. One replica per service. The assistant's daily budget counter, the rate limits and the comments cache all live in pod memory. A restart resets the counters: Cloudflare's own limit is the backstop for the assistant, and the comments are reloaded from Table Storage. A second replica would need that state somewhere shared.
• Scrapers can read the pages. Without Bot Fight Mode, a scraper can fetch every page, the way any reader can. The pages are public, static files that nginx serves for next to nothing; the forms are what need protecting, and they keep Turnstile and the rate limits.
• A signing vault without a firewall. GitHub's runners have no fixed addresses a firewall could admit, so the vault with the signing key is open to the network. It holds that one key, which can't be exported, and CI's identity may only sign with it. A registry firewall would need the Premium tier, at about ten times the price of Basic.

Read more

• My website runs on Kubernetes. Here's how, and why.: every component, and what broke along the way.
• I added a contact form. Then came the email, bots and DNS.: the Go service, its security checks and the email DNS decisions.
• A private AKS cluster, minus the jumpbox.
• The live status page: what it shows and how it gets the numbers.
• Deploy only what changed: content-addressed image tags.
• How much traffic can one small node take?: a load test against the live site.
• An AI assistant for €0 a month, and a Worker who naps.
• Comments on a static site: the plan and what actually happened.
• A public backlog from private issues, with no token to rotate.
• A GitOps CMS: scheduled posts with Git and a cron job.
• After the A record: the security series, starting with DNS.

The stack