Open Source Runs on-device
Civic-SLM is a domain-specialized fine-tune of Google Gemma 4 E4B — a compact, effective-4B model — for U.S. local-government documents: city, county, and township agendas, staff reports, comprehensive plans, minutes, ordinances, and municipal codes. It’s built edge-first — small enough to run on a laptop or even a phone — to power civic-transparency tools across all 50 states.
Trained on a single Apple Silicon Mac via MLX-LM. Shipped as both MLX 4-bit (the primary on-device artifact) and GGUF Q5_K_M, so it runs on MLX, Ollama, LM Studio, llama.cpp, or any OpenAI-compatible endpoint. Documents are crawled with browser-use — one small recipe per jurisdiction.
This project is open source under the MIT license and the source code is available here.
Why
Local government is where most public decisions actually get made, and the documents that drive those decisions — agendas, staff reports, minutes, ordinances — are mostly PDFs buried on legacy CMSes. General-purpose LLMs can read them, but they hallucinate specifics, miss citations, and don’t know the genre. Civic-SLM is a small, open, auditable model — small enough to run on-device — trained specifically on this corpus so it can ground answers in the source text, extract structured data from staff reports, and refuse when the context doesn’t support an answer.
Pipeline
- Crawl — one browser-use recipe per jurisdiction (San Clemente, CA ships as the demo; recipes are tiny and composable for any U.S. city, county, or township).
- Chunk — Pydantic-validated
DocumentChunkschemas with provenance. - Synthesize — generate training pairs via the Anthropic SDK or a fully-local LLM backend (env-switchable).
- Train — continued pre-training (CPT) and supervised fine-tuning (SFT) on MLX; preference optimization (DPO) is a planned follow-up.
- Merge & quantize — final adapter merged and quantized to MLX 4-bit and GGUF Q5_K_M.
- Eval — every stage reported to W&B and compared against the committed base-model baselines.
Eval-first
The training contract is no training without a baseline. Four benchmarks run against the base Gemma 4 E4B before any fine-tuning starts; those numbers are the bar every subsequent stage has to clear. The goal is edge-first: beat the E4B base and stay small enough to run on-device — not chase a larger model’s ceiling.
| Bench | What it measures | Score |
|---|---|---|
civic_factuality | Q&A grounded in held-out docs | citation exact-match + word-overlap |
refusal | refuses when context lacks the answer | refusal rate (regex + fallback judge) |
structured_extraction | staff report → JSON | field-level F1 |
side_by_side | open-ended municipal prompts vs the E4B base and a Gemma 4 31B reference | Claude or local-LLM judge with A/B position swap |
Results — civic-slm-e4b-v1 vs base (Gemma 4 E4B, MLX 4-bit)
| Bench | base Gemma 4 E4B | civic-slm-e4b-v1 |
|---|---|---|
| factuality | 0.460 | 0.561 |
| refusal | 0.990 | 0.970 |
| extraction | 0.097 | 0.682 |
| side_by_side | — | pending (vs Gemma 4 31B) |
Quickstart
uv sync --all-extras
uv run pytest # unit suite: schema, ingest, scorers, synth, train, llm-backend
uv run civic-slm --helpThe civic-slm umbrella CLI exposes every stage: doctor, crawl, synth, eval run, eval side-by-side, train cpt|sft|dpo, and a local rag mode for dogfooding the candidate model. See the repo’s docs/USAGE.md for an end-to-end walkthrough and docs/RECIPES.md to add a new jurisdiction.
Status
Civic-SLM has completed its first full training cycle. The v1 model (civic-slm-e4b-v1) is trained — CPT → SFT on a multi-jurisdiction civic corpus (San Clemente plus eight more U.S. cities and counties) — merged, and quantized to MLX 4-bit and GGUF Q5_K_M. It beats the Gemma 4 E4B base on 3/3 scoreable benchmarks while running entirely on-device. Remaining before the tagged 1.0 release: the side_by_side benchmark against a Gemma 4 31B reference, and publishing the weights to the Hugging Face Hub.