GitHub - skittleson/mqtt_broker_esp: A standalone MQTT 3.1.1 broker that runs entirely on a $10 microcontroller.

Most MQTT setups need a Raspberry Pi, a cloud account, or a Linux box running Mosquitto. This puts the entire broker on an ESP32-S3, written from scratch in ~2,800 lines of C. No external MQTT library, just lwIP BSD sockets. 100 concurrent clients, QoS 0/1, retained messages, OTA updates, a Tasmota-style web portal, scheduled publishes, Berry scripting, and an SNTPv4 server. All on an 8 MB chip you can power from a USB battery.

Why I built this

I built this for a 10-year (or longer) deployment lifetime on $10 of silicon. The kind of thing you screw to a wall, plug in, and walk away from for a decade. No telemetry. No phone-home updates. No hard-coded policy that governments can change out from under it... DST rules, NTP upstreams, regulatory bands all live in user-editable NVS, not in the firmware. Storage decisions get sanity-checked against NVS wear... a write-per-PUBLISH would shred the flash in months, so the retained store and in-flight tables stay in PSRAM.

That goal forced a few hard choices. No external MQTT library. The broker is ~2,800 lines of custom C against lwIP BSD sockets. I keep needing a small local MQTT broker in places where dragging a Raspberry Pi doesn't fit the shape of the problem... trade shows, ag, isolated IoT VLANs at home, the back of a truck. Pi means an SD card to corrupt, an OS to update, and ~3W of idle draw. Cloud broker means an account, an internet connection, and latency. Neither shape fits.

How it gets built and stays honest

I used Claude Code heavily to write this. That's not interesting on its own. What keeps it honest is ~129 pytest assertions running against a live ESP32 on every release. Protocol conformance, wildcards, retained binary payloads with MD5 verification, 50-client concurrency, NTP server behavior. The tests run on real hardware, not a host build, because a broker that passes on x86 tells you nothing about whether it survives a flaky 2.4GHz radio at the back of a truck. AI accelerates. Tests on real hardware are the only thing that decides whether the code is right.

Built for home automation, IoT sensor fleets, and edge deployments where "plug it in and forget" is the actual requirement.

What's surprising on a microcontroller

Quick start

# Build & flash (requires ESP-IDF v5.5+)
source $IDF_PATH/export.sh
git clone https://github.com/skittleson/mqtt_broker_esp.git
cd mqtt_broker_esp
idf.py build flash monitor

On first boot the device comes up as a WiFi AP:

1. Connect to mqtt-broker (password mqtt1234)
2. Open http://192.168.25.1, enter your WiFi credentials
3. After reboot it joins your network and starts the broker on port 1883
4. Discover it: avahi-browse -rt _mqtt._tcp → it advertises as mqtt_broker.local

mosquitto_sub -h mqtt_broker.local -t "test/#" -v &
mosquitto_pub -h mqtt_broker.local -t "test/hello" -m "world"

Schedule a Tasmota-style publish without leaving the broker:

# Set your timezone on /settings first (dropdown of ~40 presets)
curl -u user:pass http://mqtt_broker.local/settings   # → Time (NTP) → pick a preset

# Then arm slot 1 to publish at 17:00 every weekday
TOKEN=$(curl -s -u user:pass http://mqtt_broker.local/api/csrf | jq -r .token)
curl -u user:pass -H "X-CSRF-Token: $TOKEN" -H 'Content-Type: application/json' \
     -X PUT http://mqtt_broker.local/api/timers/1 \
     -d '{"a":1,"r":1,"hm":1020,"d":"-MTWTF-","tp":"cmnd/tasmotas/POWER","pl":"ON","l":"lights on"}'
# → {"saved":true,"n":1,"next_fire_unix":...}

Features

• Full MQTT 3.1.1 — CONNECT, SUBSCRIBE, PUBLISH, PUBACK, UNSUBSCRIBE, PINGREQ, DISCONNECT
• QoS 0 and QoS 1 in both directions, with in-flight retry tables and min(pub, granted) delivery
• 100 concurrent clients, 2,048 subscriptions, 16 KB payloads (binary-safe up to 64 KB retained)
• Retained messages with configurable TTL, PSRAM-backed, FIFO eviction at 80% PSRAM
• Wildcards — +, #, and $SYS topic protection per spec §4.7
• Authentication — optional MQTT username/password + Basic Auth on the portal
• Echo detection — tracks per-topic publish frequency and detects feedback loops (configurable count + sliding window). Exposed via /api/echo-detected + reset button on the dashboard. How it works →
• Web portal — Tasmota-style dark UI: dashboard, live /clients, settings, OTA, firmware rollback
• Scheduled publishes — 16 Tasmota-style timer slots (Arm / Repeat / HH:MM local / day mask / window jitter / publish topic+payload+QoS+retain). DST-aware via POSIX TZ (preset dropdown for ~40 common zones). Persists in NVS. Per-slot test-fire and master pause.
• OTA updates — file upload, URL fetch, dual partitions, manual rollback button
• Built-in NTP — SNTP client + SNTPv4 server on UDP :123, mDNS _ntp._udp advertisement, <±50 ms drift, drift compensation while free-running
• Ethernet gateway mode (optional) — W5500 SPI + NAPT to bridge an isolated IoT WiFi to your LAN
• mDNS — reachable as <hostname>.local, advertises _mqtt._tcp, _http._tcp, _ntp._udp
• WS2812 status LED — visual boot/connect/run/AP state
• No external MQTT library — single C codebase, the only non-IDF dep is espressif/led_strip
• Berry scripting — embedded Berry v1.1.0 VM on CPU 1 (see below)

Hardware

Any ESP32-S3 board with PSRAM works. The W5500 on the Waveshare board is only needed for Ethernet gateway mode.

Web portal

_{→ Full portal tour with desktop + mobile screenshots of every page}

• / — dashboard with WiFi/broker stats, MQTT auth state, device info, echo detection widget
• /clients — live MQTT clients (ID, IP, uptime, subs, in-flight, published, keepalive) + WiFi AP clients (MAC, RSSI). Polls /api/clients every 3 s, pause button, tab-hidden backoff
• /timers — 16 scheduled-publish slots. List view shows armed state (● / ◐ / —), repeat icon, time, day mask, topic. Edit form has Tasmota-parity fields (Arm / Repeat / Time / Window / Days / Topic / Payload / QoS / Retain) plus a live Next fire: today at 17:00 (in 4h 12m) line. Master pause pill in the header. Mobile collapses to stacked cards below 600 px
• /settings — broker port, auth, buffer size, retain, AP credentials, hostname, NAPT, NTP, echo detection config, timezone dropdown (~40 IANA presets + free-form POSIX TZ). Save & Reboot with confirm dialog and countdown page
• /update — file upload, URL fetch, rollback button showing the other partition's version
• /time — live clock, NTP client+server status, recent SNTP clients, force-resync
• /berry — Berry scripting manager (see Berry scripting below)
• JSON API — /api/status, /api/clients, /api/time, /api/timers (GET list, PUT/DELETE per slot — CSRF-protected), /api/echo-detected (GET), /api/echo-reset (POST — CSRF-protected), /api/ping (open, for liveness), /api/berry/status, /api/berry/log

Full endpoint and JSON reference: docs/api.md.

Berry scripting

The broker embeds Berry v1.1.0 as a lightweight automation layer. Scripts run on a dedicated FreeRTOS task (CPU 1) and never block the broker's select() loop.

/berry — 4 named script slots, each with a label, enable toggle, and up to 2,000 bytes of Berry code. Enabled slots run in order on every boot/restart. Edit inline, run one-off snippets in the REPL pane.

Available modules

# Subscribe to MQTT topics
mqtt.subscribe("sensor/+", def(topic, payload)
  print(topic + " -> " + payload)
end)
mqtt.publish("status/broker", "online")

# Make HTTP requests — returns [status_code, body_string]
import json
var r = http.get("http://192.168.1.100/api/status")
if r[0] == 200
  var obj = json.load(r[1])       # parse JSON body
  print(str(obj["power"]))
end

var r2 = http.post("http://host/webhook", "payload text")
print(r2[1])                      # plain text response as-is

Supported response formats: JSON (json.load(r[1])) or plain text (r[1] as-is). Status -1 with an error description on network failure.

See examples/berry/ for copy-paste-ready scripts and the full API quick-reference.

Configuration

All settings persist to NVS and survive reboots.

Compile-time tunables (max clients, in-flight slots, retry timing, LED GPIO, SPI pins, …) live in main/mqtt_broker.h, main/Kconfig.projbuild, and sdkconfig.defaults. See docs/architecture.md for scaling past 100 clients.

Building from source

# ESP-IDF v5.5+: https://docs.espressif.com/projects/esp-idf/en/latest/esp32s3/get-started/
source $IDF_PATH/export.sh

idf.py build              # build
idf.py flash monitor      # flash + serial log
idf.py menuconfig         # tweak: "MQTT Broker Configuration"

For the Ethernet gateway build (W5500 + NAPT):

cat sdkconfig.defaults sdkconfig.defaults.eth > sdkconfig.combined
idf.py -D SDKCONFIG_DEFAULTS="sdkconfig.combined" reconfigure
idf.py build

Testing

The test suite runs against a live broker (there is no host-build today — tests target the real radio + Ethernet stack on a flashed device).

pip install paho-mqtt requests ntplib jsonschema

# Everything (MQTT + NTP), default host 192.168.22.100
make test

# Targeted
BROKER_HOST=192.168.1.100 make test-broker        # 116 MQTT/portal tests
BROKER_HOST=192.168.1.100 make test-ntp           # 13 NTP tests

# With portal auth
BROKER_AUTH=admin:secret make test

# Destructive cycle (saves settings, reboots 2-3×, ~2 min extra)
BROKER_TEST_DESTRUCTIVE=1 make test

# Stress: 90 concurrent connections, 500-msg throughput, 255 topics
python3 stress_test.py

~129 assertions cover: protocol conformance, wildcards, retained, binary payloads up to 15 KB with MD5 verification, 50-client concurrency, throughput, latency, duplicate client IDs, keep-alive, QoS-1 inbound + outbound, unsubscribe, all portal pages and JSON APIs, settings persistence, NTP client+server, mDNS discovery, anti-amplification, and rate-limit drops.

Documentation

• docs/architecture.md — tasks, cores, memory layout, flash partitions, QoS internals, Ethernet/NAPT, LED states, network modes, scaling past 100 clients
• docs/api.md — all HTTP endpoints, JSON schemas, $SYS topics, curl examples
• docs/SCREENSHOTS.md — annotated portal tour (desktop + mobile) for every page
• plan/plan-scheduled-publishes.md — timers design, DST/TZ correctness analysis, 10-year-lifetime rationale
• docs/timers-ux-audit-v0.8.0.md — portal UX audit methodology, baseline screenshots, fix sequencing
• docs/portal-latency-analysis.md — measurements behind the 0.6.6 latency work
• docs/qos-persistence-plan.md — roadmap for persistent sessions
• CHANGELOG.md — per-release notes (full text under changelog/)

Roadmap

• QoS 2
• Persistent sessions in PSRAM (no per-message flash writes — 10-year device-life goal)
• TLS / certificate auth
• Sunrise/sunset modes on timers (needs lat/lon settings first)
• cmnd/<host>/Timer<n> Tasmota MQTT command bridge (configure timers from Tasmoadmin etc.)
• Mobile card layout pattern applied to /clients
• Bootloader auto-rollback with an in-app self-test
• Auto-regenerate tz_presets.c from current IANA tzdata in CI

Contributing

PRs welcome. Workflow:

1. Fork & branch (git checkout -b feature/x)
2. Build (idf.py build) and run make test against your device
3. Open a PR with the test output

Acknowledgments

Web portal UX takes cues from the Tasmota project — the dark theme, full-width button menus, fieldset-based info sections, and Information/Configuration/Firmware Upgrade split. Different goal (broker, not device firmware), same taste in layout.

License

MIT. Custom MQTT implementation — no external broker library. Only non-IDF dependency is espressif/led_strip for the WS2812 status LED.